I'm very sure that the way you have described the handling - it's either something to do with drop links, ARB, bushes or the car is needing alignment.

Just before I had lot's of my suspension bits replaced it was exactly as you describe - crashing through pot holes and terrible handling. Once the ARB and drop links were replaced - the transformation was amazing.