Fletcher Building is under fire over its search for a new board chairperson after a prominent candidate pulled out of contention.