Sounds like pinworm ,which doesn't show up in a worm count , these worms are becoming more and more common and very difficult to get rid of , would suggest you contact your vet for more info on how to treat as can be a long process.
Have you tried him in a roller to see what his reaction is ,give him plenty of time to work things out to start with as he may remember pain/issues from being girthed with a saddle on , i would also do this in an enclosed area ie sand school or fenced off part of a feild so both him and you are...