Languages all have their own tricks
An important factor is whether the language allows (or entices) you to use constructs that defeat compiler optimisation. For example, much of the speed of old-fashioned Fortran came from the absence of anything like pointers - so the compiler could more accurately assess the scope within which a variable might be referenced.
C's use of pointers is probably the main thing that makes it slower than old fashioned Fortran, but if you code carefully and don't use pointers, that gap will close. With C++ you're one more step removed from knowing whether the compiler will be able to optimise what you write, so more frequently it can't.
To give another example from Java, garbage collection can be a real problem but can often be mitigated by avoiding unnecessary object creation/destruction. Unfortunately, this is again something that a compiler is unlikely to manage on its own as it's part of the logical design of the software and at a higher level than compilers work at.
So while compiler optimisation is always a good thing, good software design also lies at the heart of run-time efficiency. Those who say you can't beat the compiler at optimisation may be right at the level of loops and method calls, but at a higher level it's easy to design something so it runs slowly in any language you like. Knowing what the language does fast and what it does slowly is where the solution lies. So there's no real substitute for experience with the particular language in question.
Of course, this does make C++ programmers superior beings, as few are able to gain much experience with this language without shooting off both their feet at some point.