black & white - at my old yard we had a double in the jump field that was black & white spots. ALWAYS used to catch people out in lessons as well as in the open comps the yard ran.
We have a half circle shaped red and white filler which has a sunbeam type pattern on it that really seems to catch a lot of horse's eyes. Mind you we have nothing in black and white so I wouldn't know if that's worse.
It's not fillers that get my horse, but he struggles with fences which don't have a strong enough groundline. We always end up chipping in and climbing it!
Fillers don't bother mine no matter colour shape or pattern. HOWEVER a really scary jump with a hole cut out of the middle caught us out xc today definite eyes on stalks moment lol
One of mine used to always spook if decorated with yellow flowers!
And ive found sometimes one of those new plastic brush fillers sometimes suprises them if in middle of bright course...you wouldnt expect it but mine always looks at brushes!
i heard that any filler with two really contrasting colours, like a really dark and really light i.e. black and white is very difficult for a horse to pick out. and also plain white tends to catch some out.
Like MandyMoo says, Ive always thought that horses had problems seeing something really dark against light so yellow and black for example. And I think plain white can be really bad for shining in the sun - one of mine who never refuses SJ and rarely ever spooks at ANYTHING was terrified of white jumps wings which lined the edge of the arena we were in at the weekend but it was very sunny and it was glinting off them.
I have a horse that is terriefied of any filler with the Union Jack on it... very unpatriotic. My other (now semi-retired) horse only hacks, but will jump across the road if there are daffodils on the verge!
I seem to remember being told at uni that horses only have certain rods and cones in their eyes and so can't see all colours. I think they only have red and yellow, so in theory anything blue will just look grey, but as others have said contrasting patterns are probably scariest.
Our Blue and Black Fillers (or just the blue and black poles/planks alone) cause the most problems at our shows. I painted a union jack filler and they didnt find that scary, or the yellow with black stars or purple and orange spots! Seems to be the more simple one (blue and black has one black stripe across the middle!!)