I dont know much about diesel engines but this wikipedia article explains it pretty well
EGR in spark-ignited engines
The exhaust gas, added to the fuel, oxygen, and combustion products, increases the specific heat capacity of the cylinder contents, which lowers the adiabatic flame temperature.
In a typical automotive spark-ignited (SI) engine, 5 to 15 percent of the exhaust gas is routed back to the intake as EGR. The maximum quantity is limited by the requirement of the mixture to sustain a contiguous flame front during the combustion event; excessive EGR in poorly set up applications can cause misfires and partial burns. Although EGR does measurably slow combustion, this can largely be compensated for by advancing spark timing. The impact of EGR on engine efficiency largely depends on the specific engine design, and sometimes leads to a compromise between efficiency and NOx emissions. A properly operating EGR can theoretically increase the efficiency of gasoline engines via several mechanisms:
Reduced throttling losses. The addition of inert exhaust gas into the intake system means that for a given power output, the throttle plate must be opened further, resulting in increased inlet manifold pressure and reduced throttling losses.
Reduced heat rejection. Lowered peak combustion temperatures not only reduces NOx formation, it also reduces the loss of thermal energy to combustion chamber surfaces, leaving more available for conversion to mechanical work during the expansion stroke.
Reduced chemical dissociation. The lower peak temperatures result in more of the released energy remaining as sensible energy near TDC, rather than being bound up (early in the expansion stroke) in the dissociation of combustion products. This effect is minor compared to the first two.
It also decreases the efficiency of gasoline engines via at least one more mechanism:
Reduced specific heat ratio. A lean intake charge has a higher specific heat ratio than an EGR mixture. A reduction of specific heat ratio reduces the amount of energy that can be extracted by the piston.
EGR is typically not employed at high loads because it would reduce peak power output. This is because it reduces the intake charge density. EGR is also omitted at idle (low-speed, zero load) because it would cause unstable combustion, resulting in rough idle. The EGR valve also cools the exhaust valves and makes them last far longer (a very important benefit under light cruise conditions)
Since the EGR system recirculates a portion of exhaust gases, over time the valve can become clogged with carbon deposits that prevent it from operating properly. Clogged EGR valves can sometimes be cleaned, but replacement is necessary if the valve is faulty.
In diesel engines
By feeding the lower oxygen exhaust gas into the intake, diesel EGR systems lower combustion temperature, reducing emissions of nitrous oxides. This makes combustion less efficient, compromising economy and power. Diesel EGR also increases soot production
, though this was mitigated in the US by the simultaneous introduction of diesel particulate filters. EGR systems can also add abrasive contaminants and increase engine oil acidity, which in turn can reduce engine longevity.
So basically the only benefit, apart from less NOx pollution, is lower temperatures inside the combustion chamber which increases components life. Sounds like direct loss of power at some regimes.
I also understood that diesel engines needed it before particles filters appeared so they would pass NOx regulations.