Sounds like you have been treating him well! probably too well.
Remember ex race horses love routine, if you are going to change anything do it slowly, if he is hacking out and decides he wants to go in a different direction then don't let him, even if you have to get off to lead him, don't...