Your papers please
"Could it be said that the Windrush data was no longer "needed"? In hindsight, no"
The old aphorism "Absence of evidence is not evidence of absence," applies here. Where the person in the Home Office made a big mistake was, having ordered the destruction of the documentation, they then claimed that people didn't have the documentation to prove they landed legally. Of course they didn't have the documentation, YOU THREW IT AWAY, DUMMY!
Throwing away the documentation as unnecessary, then claiming it was necessary to justify residence is self contracting. It shows either incompetence or malice. Which gets me to my second point.
"The Home Office" didn't make and carry out this policy, individuals did, a mix of politicians and civil servants. The claim that "I was just obeying orders" doesn't stand up. They took the money and probably were promoted for a job well done. Now the truth is out, they should bear responsibility for their actions. If you get an order in your job that you think is wrong, get another job. I expect they will hide behind the shield of anonymity as usual.