"People are funny," says Herbert. "Maybe they don't want to get involved. Maybe they're afraid what they say will be misinterpreted."
Take the exile, for example, when Ali's passport had been picked up, and the patriots barred him from fighting because of his stance on the draft. Ali's legal fees had wiped him out. There was still the money that the Louisville group had put in trust for him—15% of all his earnings. Ali tried desperately to get to this money, but he was not legally entitled to it until he was 35. He borrowed heavily—even down to $10 and $20 bills—from his friends. Where were the Muslims and Herbert, who had taken a handsome 40% of all Ali's earnings? "We gave him money, did a lot of things for him," says Herbert, who used to have to meet Ali in secret on street corners when Ali was suspended from the Muslims by Elijah because he persisted in boxing. What sort of things? "That's between Ali and me."
The Ali-Herbert union is curious, unique and exemplary in the ring, or for that matter, in the entertainment business; the two really have nothing in common, except their religion. Ali is a public person, grows restless without the stage; Herbert is private. Ali has no regard for money; Herbert seems to care for little else. Ali is genuinely kind and giving; Herbert is often seen as cold and ruthless. Ali is a physical creature, will be all his life; Herbert is exquisitely sedentary. In the past, there were no notable rows between the two, hardly any harsh words. But Herbert was angry after the fight with Jimmy Young, berating Ali—rightly so—for his poor physical condition, describing him as a disgrace, and yelling at him behind closed doors, "From now on, you're going to listen to me."
Girls and sweets are Ali's implacable temptations. "I can't watch him forever," says Herbert. "Like he says: 'Herbert, you got to sleep sometime, and at night I can git out.' " Most of the time Herbert is an absentee manager. He comes into a town before a fight, and stays in a hotel far from Ali's, while those he has hired to watch the champ cater to his desires. Early on, Herbert was a fixture in Ali's corner, but now the only time you see him is when he takes a seat below it. There he sits with a water bottle in front of him, shifting nervously while the wild dollar arithmetic of Ali's future spins through his head; sometimes he walks out in disgust or fright, as he did in the Young fight before the decision was announced. "I was scared, real scared," says Herbert. " Ali says he's never been more scared in his life."
On the surface, nothing seems to have snarled their relationship. "It's clear between the two of us," says Herbert. " Ali told me, 'Look, Judge, I'll handle the boxing, you handle the lawyers and the promoters.' That's the way it is." Herbert is sensitive to Ali, he is aware of his anti-boss attitudes, so they call each other Judge. Look beneath the surface, though, and you sense unrest, strain in their union. Among the causes are the tarnishing of Ali's name after his bout with the wrestler in Tokyo and the ultimate size of the purse; the frequency of Ali's fights, which he feels indicates panic and greed on Herbert's part as the two of them approach the end of Ali's career.
"Champ," one of Ali's people said to him while Ali was in a California hospital recovering from blood clots in his legs caused by the wrestler's kicks, "Champ, don't you think that a man who is getting one third of all you make, don't ya think he should be here with you? Does he care?" The man, of course, was Herbert, who gets a third as his end of the money now. Ali did not answer, but later complained angrily over the phone that he wound up with only $1.4 million of the $3 million guaranteed in the Tokyo contract. " Ali is unhappy," says Don King. "He says at least he got all his money when I was doin' the promotin'." If Ali is disturbed—he has not said anything publicly—it would add credence to the widely held belief by insiders that the death of Elijah set him free, that he is his own man now, that no longer can the name of Elijah be invoked to make him step smartly into line.
Herbert hates conflict, yet such talk does not bother him. "I was heartbroken over Tokyo," he says. "But I would do it again. All that money for an exhibition. How can you turn it down? But the event got out of hand and became dangerous. And I still don't believe Ali was publicly damaged. His fame is beyond that. As for the money, I'll take the blame. It was the first time that I did not get all the money up front and in the bank. It was a mistake." Herbert is aware of those who are trying to undermine him. "Go ask Ali one thing," he says. "Ask what he'd do if he had a problem and I wasn't around. He'd find me if he had to spend $10,000 in phone bills."
Herbert says there have been a number of fights he never wanted to take. Ali makes these decisions, Herbert says, pointing out that he told Ali after the Ron Lyle bout, "Maybe you ought to pack it in, get out of the game ahead." Says Mickey Duff, a London promoter, "Herbert's being smart. He has to fight Ali a lot. Ali away from the ring would be his own worst enemy. At this stage of his career, with his tastes and money habits, long absences from the ring would be terrible. Ali stays in shape by fighting. It's that simple." Herbert speaks bluntly of Ali's financial condition, noting that he will net about $7.5 million after taxes this year, about $3 or $4 million after other expenses. He says Ali is in fine shape with Internal Revenue, that there is no reason why he should end up like Joe Louis.
"Look, I can't lock up his money," says Herbert. "I wish I could have done what the Louisville group did—take that 15%. They did a good job for Ali. But I can't do that. I'd look to Ali like I'm interfering, that I'm the boss getting in the way of his money."
"Is his future secure?" Herbert is asked.