"As most of the people who develop software find, however, very few people know exactly what needs to be built ahead of time, least of all the actual users and customers."
Erm...to a point but I've got a few ranting points on this about why waterfall has a bad rep
1) More often than not so called requirements are simply vague notions and ideas of in the head of some user*. There actually isn't much in the way of empirical research or data to support the requirement apart it seems like a good idea.
The most recent example I've had "our new online shop must allow discount codes to be applied to the shopping basket". Putting aside the mechanics of it for a second (which havent been at all thought through), if you ask the question why does the shop need to do that, i.e. what are you hoping to get from that feature - the person looks at you blankly and basically says "because". Normally the "because" is they've read it on a Gartner blog and its what everyone is doing*.
2) Often businesses want to see something created as quickly as humanly possible with minimal cost/effort/disruption. So when Waterfalling, the requirements engineering and testing phases are always compressed to be short as humanly possible because it doesn't deliver *anything useful* and is disruptive on user's time. How many of us have been in situations where the users try to use the testing phases to deliver new requirements?
So in round agile sounds perfect...but wait you're never given time to iterate the MVP to keep adding all those vague requirements. Many of the projects given to me have finite end points where you have assigned something else completely unrelated.
In a nutshell management want you to be "agile" but only in the sence of working faster/harder/longer, the end users want you to telepathic and turn their vague ideas into requirements. So you are left with a broken waterfall project lifecycle model, descoping everything and upsetting the users because their next best idea doesn't materialise.
I know above is preaching to the choir but I need to share my burden...