A pic would help.
If it is a plastic cover that is removable then have you tried one of the trim dressing products which are spoken about extensively on this forum? e.g. Aerospace 303. I don't see why it would not be any less effective than on any other plastic or rubber trim. It is particularly good for UV damaged areas
A product I recently purchased is Chemical Guys Fade to Black. This is an aerosol spray and is great for treating the honeycomb grill below the front bumper which is always a baitch to dress.
If you remove the speaker grill, clean it up well and give it a couple of coats I'm sure the change will be dramatic.