Probably a bit late now but if I were you I would have got back on him today rather than leaving it too long which would only worry you more. It sounds like you just scared him which caused him to buck rather than him being naughty. I know for a fact that if I scared my young mare when I was getting on, that she would react in the same way.
Next time, I'd go and unlock the paddlock to the gate first. Then get the pony, so you don't have to faff with the lock with him, just open the gate, which should be a lot easier and much less fuss. Also by then you'll both be calmer when you come to mount.
The pony is getting used to you (4 times or a month every day -either way, it's still a very new relationship) and getting used to a new place. It also sounds like you're using a saddle that isn't a good fit.
I would relax. Take him for a walk on the lead. Let him get used to his surroundings. Get the saddler out. When you do get back on, don't push either yourself or the pony. Take a nice relaxing ride without scary gates or ditches. Even just a ride around the yard itself.