During the next winter, Durelle let his weight reach two hundred pounds. He started training for the 1959 rematch against Moore about three months before the fight. It was very hard, but he was able to get down to the weight limit. But then the fight was put off. Each time it got harder to make the weight limit. He got weaker and weaker. The rematch was no contest. Durelle was knocked down three times and then knocked out in the third round.

A possible reason Durelle did so poorly in the second Moore contest was a boating accident he had before the fight. He hurt his head in the accident. After this, he could not walk very well. Also his speech was hard to understand. For years, people thought he was drinking again. Then he met Dr. Jewett in Fredericton. Dr. Jewett treated Yvon for over a year and Durelle got much better.

After the two Moore fights, Yvon felt so bad about boxing that he no longer cared. In 1959, he decided to fight George Chuvalo for the Canaian heavyweight title. He needed the money and there was more to be made in the heavyweight class than any other. Durelle did not train at all for this fight, and was badly beaten. The day after this contest, Yvon retired and gave up his British Empire light-heavyweight title.