According to the rules for affiliated its a no, they list acceptable colours and grey isnt there. Possibly, a a low level and/or unaffiliated if they're a very light grey you might get away with it.
Personally though light grey looks like grubby white so I still wouldn't
"77. Breeches/Jodhpurs Breeches or jodhpurs are to be plain white, cream, beige or canary yellow except when worn with official services uniform, when they may be of the uniform colour."
I don't know if the rules were the same in the early noughties, I suspect they were, I used to wear (lightish) caramel coloured breeches, never a problem. They probably qualified as beige, as would a warm light grey I'm sure.