Check out the Exmoor Pony Centre at Dulverton (google them for their website).
Also if you are going to Minhead you will be driving right past the Quantock Hills. We have "wild" ponies here as well living on the hills. In June there should be lots of foals around - they are lovely to watch. If you park at Crowcombe Gate car parks ( on the road between Nether Stowey and Crowcombe) you can often see them.
Hi, I live in Minehead, and, as Silver Snaffles says, there is a herd of Exmoor ponies on North Hill, Minehead. Where I most often see Exmoor ponies is as follows: drive up Porlock Hill and there is a car park on the right hand side, which has a lovely view over the bay and quite often has a Styles ice-cream van parked in it, and there are often ponies round that area. Failing that, if you take the nearby left hand turn off to Exford, there are often ponies along that road, either on the right where there is a big, flat, open area half-hidden by a low stone wall, or on the left where the little humpback bridge goes over a stream. Good luck! They are so gorgeous, especially the foals in June, and not TOO camera-shy!