Landowner/occupier responsibilities with respect to injurious weeds can be found here https://www.gov.uk/guidance/prevent-the-spread-of-harmful-invasive-and-non-native-plants
There is a link on the page detailing how anyone worried about injurious weeds growing on adjacent land can raise a complaint. Action is only usually taken if the weeds are threatening land used for keeping or grazing livestock, farmland used to produce conserved forage and land used for agriculture.