Chainsaw it and paint the stumps with the correct stuff and dig up any small bits. Really it's a specialist contractor job as its such a beggar to get rid of. And it will spread like wildfire so the longer you leave it the worse it will be.
If you've a lot, it is worth getting someone in with a mini digger. I got rid of about an acre by using an old plough and the 3 point linkage to lift the roots out of the ground, then letting the grass grow back. The gorse still grew through but heavy liming and fertilising, followed by hard grazing finally got rid of it. I have a nice paddock now. But do consider the shelter gorse gives. Sometimes it will save a life! I'm thinking of planting it in some corners.
I have quite a bit on the edge of woodland - and I just cut, and cut, and cut again! It works - but you have to keep at it. Spot spraying the bad areas also helps - I use Grazon - but you have to DRENCH it in late spring.
I love gorse! My favourite smell too, I even have a gorse candle! Was actually trying to work out where I can get some, and if it's illegal to pinch one from rough ground
I can see them being a pain in the wrong place though.
We had a racehorse trainer on our yard and she used to feed her racehorses it as an odd treat. She gave some for my horse to try the one day, he loved it!
Gorse is highly nutritious. It is called whins in some parts of the country. And they used to have whin mills for breaking it down so it could be fed to cattle. The mill was just a circular stone trough with a beam pivotting on a central axis. There was a wheel shaped stone on the beam which travelled in the trough as a pony pulled the beam in a circle, crushing the whins it went around. Might be worth putting it through a garden shredder and seeing if your horses will eat it -- but only try a small amount first for fear of colic!
mine adore it. We ride through gorse every day and they often stop and eat it, it is especially yummy at this time of the year.
It is controlled around here by burning. It doesn't get rid of it but keeps it under control.