In 1953, he could no longer make the middleweight limit. He had to become a light-heavyweight. Light-heavyweights can weigh up to one hundred seventy-five pounds. It took only until September, 1953, for him to win the Canadian and British Empire Light-heavyweight Title. He defeated Gordon Wallace in the title match in Moncton by knocking him out in the second round. In the rematch, Yvon was an easy winner again. But later in 1953, he lost his title to Doug Harper in a fight in Calgary. Many who saw the fight thought it was a very poor decision.
1954 began as a good year for Durelle. In July, 1954, he won his British Empire title back by knocking out Doug Harper in Newcastle. He had also fought eight rounds with Floyd Patterson in New York City. Patterson had won the heavyweight boxing title at the 1952 Olympic Games. He would soon become the world heavyweight boxing title-holder. Durelle showed no fear of this great boxer and nearly knocked him out in the eighth round.
But, for some reason, things started going wrong for Yvon over the next three years. For one, he had a very poor manger in Harry Shepherd. Yvon had gone to Europe to fight in England and Germany. He lost most of his fights there because his rough way of boxing wasn’t permitted. Not only did he lose the matches, but also he got very little money.
Upon returning to Canada in 1955, Durelle once again fought Floyd Patterson in Newcastle. The week before this fight, he broke his hand. Also, the night before the fight he was caught in a storm at sea. He spent the night on board his boat. He was very cold by the time he got to shore in the morning. Yvon went ahead with the fight anyway. He took a very bad beating and couldn’t come out for the fifth round. Many of his fans couldn’t believe it! They thought he was a quitter.
For the rest of that year, Durelle kept boxing because he needed the money. But each fight he would hurt his hand again. At one point, he had three wins, but had lost eight times. He decided to take the winter off. His hand healed well. The next year was a better year for boxing wins. Just the same, Yvon was still not making much money. It seemed that everyone was being paid first and there was nothing left for him. He was very unhappy and was drinking too much. He made up his mind to retire. He felt he wasn’t going anywhere and he no longer cared to box.