Have you had a wheel alignment done with laser/photographic equipment and if so, what was the caster and camber on the front wheels?
Sounds like you have insufficient caster. This is the angle by which the steering axis is inclined rearwards - in other words, the amount the lower balljoint is forward of the upper strut mount (or wherever the swivel takes place - e.g. upper suspension arm (156) or halfway up the strut (164) - the GTV/Spider is conventional and swivels in the upper strut mount). Camber is much more universally understood and is the angle the wheel makes with the vertical, when viewed in line with the tread. Negative camber is when the top of the wheel is further 'in' than the bottom, which is normal but usually exaggerated after lowering the suspension (even when the factory lowers the suspension...) Usually, specification is for several degrees of caster, and about negative one degree of camber. No point getting too finicky over the exact values, as it is often so far off...
In pretty much any modern car, caster and camber are not officially adjustable. But I have dealt with this problem many times in the past. Anything can be adjusted if you have a big enough hammer OR the patience to think it through. I find with most of my cars there's nowhere near enough caster, particularly on the left front. E.g. on my 164, should be 3 degrees plus/minus 0.2 degree, mine was under 1 degree. Such variation (>100%) is cause for concern (to me anyway). No-one else seems to care and no wheel alignment shop would bother to fix it
In theory, replacing the lower arm and the strut would fix. In practice, it usually doesn't. So, the next solution is to slot some holes, somewhere, as required to effect a change. Then pay for another alignment check and hope that you got the right amount...
I haven't checked how the lower control arms attach on the GTV but I think it is similar to 156/147/GT with the bushing beside an immovable ridge in the subframe. Hard to make much of a difference, so you will probably be forced to modify the top strut mount.
On a 164 I usually find too much negative camber (2.5 degrees instead of 1 degree), so the modification needs to address that, too. The one I came up with was to move the front control arm bush inwards by replacing spacers (6mm) with washers (3mm) on the bush mounting. By moving the control arm inwards and forwards at its outer end, that simultaneously increased the caster to 2.5 degrees and increased the camber (i.e. reduced negative camber) from nearly -3 degrees to -1.5 degrees. I've owned four 164s and three of them needed this modification
On my 156 with exactly the same discrepancies, I managed to add washers behind the upper arm and grind aluminium off the front of the mounting, to move the upper arm backwards. Because of the angle the arm is on, this also did a little to reduce the negative camber. Steering feel is more 'natural' now.
On my Uno with - you guessed it - too much negative camber and too little caster, I filed out one of the stut lower bracket bolt holes, to allow the hub carrier to swing slightly outwards. With the bolt tight, it has never moved (I put paint around it to allow a quick visual check). Of course, this doesn't help caster (should be nearly 4 degrees on an Uno, mine has 1.5). Slotting the top strut mount is going to be my next move - unfortunately, quite long slots will be needed (my guess - about a centimetre up there to achieve 1.5 degrees, which still won't be enough really, but every little helps).
On my Spider, I had a fraction of positive camber on the right front wheel. Positive camber is a disaster, especially when there is negative camber on the other side... so, again, it was a case of slotting one strut bolt hole in the lower bracket, to swing the hub carrier inwards and easily get it within spec (and the same as the other side). This reduced a strange side-to-side wobble around steering centre. Because the modification was simple, I was able to get it done in about 15 minutes while it was still on the alignment machine.
Although these are different models, these are all front-wheel-drive cars with similar geometries, so you can see that one or more techniques may apply to yours.
After all this, tyres certainly help. My Spider's steering feel improved noticeably after fitting Yokohama A-drive R1s.