Commodity ness is the problem
If servers weren't built from commodity hardware there would be more than a few percent of power savings to be made.
For example feed in a few hundred volts d.c. and you can
* throw away the rectfier (which you'd still need somewhere of course for mains input but can be done better)
* have smaller input capacitors
* throw away the power factor correction section
* feed straight from a simplified inverter from solar P.V. or wind generator it being easier to output d.c. than synchronised 50Hz. The inverter now doesn't need those big capacitors either.