I had a windsucker and he always looked like he was about to give birth, even when he was at his fittest!! It was pre-children, so I used to hack him for *hours* plus a fair bit of XC and SJ, but I could never shift the big tummy!!
I wouldnt worry about it too much. If he looks ok everywhere else (on top of his bum, neck etc), then I wouldnt kill yourself (and him!) trying to shift what possibly cant be shifted.....