"I don't want people comin' up to me while I'm at dinner," says Herbert, who doesn't like to be startled by the sudden appearance of a strange face (he is well-protected at fights). "Most of the world thinks Angelo Dundee is the manager of Ali. When I go to a town, I have to call up Angelo to get me into places. But that's fine with me. I only care if Ali and the bank know me."
Mystery implies romance—but not so with Herbert. "You can sum up Herbert in three words on the head of a pin—dull, dull and dull," says a man who has had dealings with him. That must be a comforting description for someone who has spent most of his life caught in the weft of intrigue and accusation, who has been hounded by the dogs of white justice. "We've never had a day's rest," he says. "I've had more bugs put in my rooms and on my phones. Privacy was just a word to me." Herbert was not just a boxing manager back then. He was a prince of the Nation of Islam, the Black Muslims, the son of Elijah Muhammad, the leader of the sect that made a lot of whites, as well as blacks, cringe at what might be hiding behind a phrase like "white devil."
Images of the Mau Mau, of crazed storm troopers, of bloodbaths were widespread in the racially bitter '60s. Over-reaction, maybe, but there was no question that Elijah, the Messenger of Allah, never leaving his compound in Chicago, enforced iron discipline among his people, preached and imposed separatism and militancy. The Black Muslims had two faces. One suggested a placid, committed people determined to exist by means of their will and acumen; they had their own successful businesses—fish stores, bakeries, farms, etc. The other face spread fear in the form of the Fruit of Islam, a coarse, rude and humorless elite corps who wore black leather gloves and overcoats; all of them knew judo.
"Exaggeration," says Herbert now. "My father was not a violent man. He was gentle. He fought hard to make the black man proud of himself. I saw my father fight off all kinds of people. I saw them come and go." In many ways Herbert is much like his father, who was serious and formal, a man who kept a close watch on his time. Herbert does not abide fools or crackpots for long, either. "Herbert did what his father liked," says Hassan. "He communicated well with his father. There's a lot of Elijah in Herbert. He never bucked his father on anything." Once Elijah walked into the family garage and saw Herbert, then a young boy, punching a speed bag. Elijah delivered a stern lecture. "I don't want you around the ring, boxing for any little fat white man with a big cigar," he said. "Don't be around any sports world. Sport is the ruin of our people. Turns them into children who're used and then left broken. Stay out of it."
Herbert's youth was uneventful, except when he was 18. He was working at his first job, painting numbers on office doors, when he was arrested for failing to register for the draft; beginning with Elijah, who did four years in the pen for draft resistance, contempt for the draft runs in the family. Later, Herbert ran a Muslim bakery, the Muslim newspaper Elijah Speaks and a photography studio where he did portraits of Nasser, Martin Luther King Jr., Roy Wilkins and others. "Photography—I'm crazy about it," says Herbert. "I wish I could do it again. I like to play with light and shadows. To develop pictures. See the wrinkles in an old man's face coming out." It was in his studio that Herbert first met Cassius Clay in the early '60s, and no two men working together would ever be more dissimilar.
Clay was not Ali yet but had been recruited for the Muslims by Malcolm X. At the time, the white Louisville syndicate owned Clay's contract. Impressed by Herbert, Clay the convert also knew the advantage of being allied with a member of the "Royal Family," and kept after Herbert to manage him. Herbert was wary of his father and of his views on sports—boxing, in particular. After Clay beat Sonny Liston for the title and became Muhammad Ali, the Louisville syndicate was out. Herbert became the closet manager, seldom seen or heard, except in the hushed hallways of hotels, whispering to other Muslims. Soon Ali made Malcolm X an ex-friend and ex-confidant. Told to be "responsible" by Malcolm (who had broken with the Muslims over ideological differences) before going on a trip to Africa with Herbert, Ali said, "Malcolm didn't seem too responsible to me. Man, did you get a look at him? Dressed in that funny white robe and wearing a beard, and walking with that cane that looked like a prophet's stick. Man, he's gone. He's gone so far he's out completely."
Turning to Herbert, he then said, "Doesn't that go to show, Herbert, that Elijah is the most powerful? Nobody listens to that Malcolm anymore."
Herbert became Ali's manager officially in 1966. By this time, Malcolm X had been murdered, and the lover of photography came to despise being photographed. Herbert had a firm grip on Ali and tried to work on Ali's image, to bring it just a bit closer to Herbert's own conservative nature. The Ali Shuffle, which was introduced against Cleveland Williams in Houston, was forbidden. "It didn't add anything," said Herbert. Ali called Ernie Terrell an Uncle Tom, and Herbert zippered his mouth. And it was Herbert who urged Ali, who was alarmed at the heavy taxes he had to pay and fearful (even now) of becoming another Joe Louis, to fight more often. " Standard Oil doesn't try to sell a small amount of oil each year," Herbert told him.
At the time, two questions persisted in the white press: Were the Muslims bleeding Ali white financially, and had Ali been coerced into refusing to be inducted into the Army? "We never took a dime from Ali," Herbert says now. "He made and still makes donations to the religion, but no more than, say, Catholics or others give to their churches." As for the draft, Herbert says, "Nobody put any pressure on Ali. He made his decision independently. He was a Muslim. He loved my father." Ask those two questions of those who have been around Herbert and Ali, and the responses are all the same: silence. The awful things being said about Herbert Muhammad are the things that are not being said.
"Why don't your friends set the record straight?" Herbert is asked. "You say one thing; why don't they say the same thing, if that's the way it was? Why don't they say anything?"