Yes they will, because there are drugs, and there are drugs. Those with the most money and most advanced labs will win, and the playing field will be even more uneven.
If everyone was taking them they'd get stupidly cheap and in the end, there'd be no advantage.. Like the OP said, most 'using' athletes are one step ahead of the testers and only get caught out when they mess up.
Anyway, some athletes all have advantages over others naturally, which is down to genetics. A lot of the stuff these athletes are putting in their bodies are produced naturally, for example "doping" is re-injecting your own red blood cells that have been removed some weeks beforehand back into your body, improving the oxygen carrying capacity of the blood. Some people have naturally high levels of red blood cells, more than someone with average levels would have even after a doping session..
Athletics is more and more becoming about science, training techniques are becoming universal and perfect, so it is getting less and less about how much effort you put in rather than how much of a "best fit" human being you are to begin with.
Look at long distance-running, usually dominated by smaller Africans, eg Kenyan's, they are naturally superior to most others at long distance running, down to genetics. World championship 100m, 200m, 400m, hurdles etc, most countries are represented by a bigger-built black guy, whether they are from England, America, etc. Genetics gives them a massive advantage, its all to do with their capacity to deliver massive power in a short space of time, its for similar reasons why Nordic types pretty much win every world's strongest man event.
I'm not truely suggesting all performance drugs should be legalised, I'm just playing devil's advocate. But its clear the current system isn't working, so here's a radical alternative.