I had similar concerns for the first two years of ownership but managed to cure it last year. I had an issue where one of the pads was slightly askew in the caliper and stuck so not all of the pad was in contact with the disc. This also meant that after stops the hot pad was much less efficient giving a vague feeling and leaving material on the disc giving the 'warped' feeling. That said, it passed the MOT at least twice so couldn't have been affecting efficiency too much.
A caliper cleanup, fluid change plus some Performance Friction pads (everything else standard) by Dan at Autolusso Penrith made a night-and-day difference and my GTV now has brilliant brakes. If you can't make the ABS kick in at 50ish with a stamp on the pedal, I'd say you need a similar overhaul. If you've got the money a caliper refurb by someone like Bigg Red would be a nice treat. IMO, you don't need 330mm brakes, just properly working 305 with good pads.