"....If you start a project not knowing what you're making then often you end up with a sprawling mess of unfinished features...." Totally agree on the management of the project being key, but sometimes they collude with salesgrunts to create what we used to call "A Never-Ending Story". If the salesgrunt can bill for coders by the hour, he will often make a lot more money on one of those dithering, prototyping, Scrum-style projects, especially as he will save on having to pay up front for the consultants skilled to do the right requirements analysis. Project managers and coders, especially contractors not wanting to have the pain of looking for a new contract the minute they finish the current work, may often go along as it is financially better for them too. For the salesgrunts, the longer the project drags out and the more rehashing of requirements happen, the more commission he stands to make, and the bigger the pat on his back will be from his senior management when he books the larger revenue.
I have worked for a consulting company where we had direction from management to "seek out opportunities to maximize revenue by expanding project scope" with the unspoken directive that revenue was more important than delivering what the customer actually needed. In Scrum it is the Product Owner (aka Customer Representative) that is the key to making sure the customer does not get taken for a ride, but it also the position that seems to most often get assigned to a junior with little experience and no authority.