I would start with something like 2000 grit paper, you may need to move to 1500 or 1000 but once the marks are removed then start to use the lighter paper i.e. 1000, 2000, 3000 and then polish. The Meguiars PlastX is a great plastic polish which would be able to remove 3000 grit marks.
Tape off the surrounding area of the headlights to avoid any damage to the paint or trim
Polishing by hand is entirely possible but hard work. A drill would be quicker but can be a tad awkward.
When wetsanding the lights, use the block at 45 degrees to stop the paper creating deeper marks. When polishing with the drill, keep the pad moving to stop over heating the plastic
There is probably loads of videos out there for doing the headlights. It may sound daunting but in reality it is a straightforward process and it does not take very long to get the headlights looking perfect again