Tracking, toe in /out, relates to both wheels, and therefore should not affect the hands off the wheel test, for pulling, very much, imo. Castor, will, most definitely,worn bushes could, as may any distortion on associated components, or body components the whole of the suspension, hubs wheels are attached to.
I'm way past sell by date, but I would be inclined to finally get the checks done at a VBRA bodyshop, with full geometry facilities, and the correct chassis alignment equipment.
It may well be something more straightforward, regarding tyres or mechanical components, but I have little faith in some of those out there, who may have the latest equipment, but are only as good as the 'technicians' who use it. Hope you get it sorted, and that it is a relatively simple fix eventually.
PS Still referred to as a chassis, although it is monocoque in construction? Contradicts my post in another thread somewhat?