Hmm. Legally speaking taking the deposit back probably wasn;t the best of ideas.
It's not like it's going to get to that stage though, so maybe doesn't matter.
Worth pointing out to the guy that the original buyer forfeited his side of the contract by failing to contact, whereas you yourself have an intact contract with him for the purchase of the car? (Don't tell him the stuff we've said about taking your deposit back!)