if he's got a bit of weight to him then I think you'll be just fine - there is no need to be a pea on a mountain!! OH is a 6'1'' beanpole and rides his 16hh cbxtb - he looks perfect on her, she has a good depth of girth and 8 1/2 inches of bone.
Might depend on how he feels to ride - I'm only 5ft5 (about!) but my boy is 14.2 New Forest but has good depth of girth and solid shoulders so don't feel like I'm sitting on something small! I've known smaller ponies ride like 15.2 and horses than ride like ponies.......good luck!
It all depends on what you are used to, at 5'7 I wouldn't buy anything smaller than 16.1hh BUT that's what I am used to - never even had a horse smaller than 15.3hh so...