There is a rattle on the 156 which is a recognised problem fixed under warranty by alfa. A small aluminium clip (i think) expands and contracts depending on temp and causes a rattle. I had this problem and got it sorted by a local dealer.
Yes, the dash has to come off! Its not a very comfortable feeling knowing an already dodgy dash is being pulled out but all was okay after. It was an official recall so even though the car is used, I would think you can still get it done.