Re: a sufficiently motivated person could in fact find out exactly how I voted.
And how would "a single private individual of average means" get their hands on an entire set of ballot papers?
That's a separate issue. My comment was only intended to state the obvious: modern technology makes correlating ballot papers and electoral rolls, which used to be a formidable challenge when the system was designed, a fairly trivial and cheap exercise the moment you can access the records. So realistically, the only thing which stands between an inquisitive person or government department and the knowledge of how you (and everybody else in the same district) voted is a court order.
There is a close analogy here with wiretaps and mail interception: when listening to your conversations and tracking your calls required a three-person crew in a white van parked on the street outside your house, it was an expensive tool which could only be used on a small number of targets and in relatively serious cases. If on the other hand getting the itemized list of the people you called requires couple keystrokes, and slurping all of your conversations verbatim is not that much harder, it will be used routinely, and for cases which are clearly frivolous. It may still technically require a court order, but this does not appear to be such a high bar anymore - especially if you invoke the right scare words, or shop for a friendly judge.
Perhaps it is time to redesign the system a little, to put more emphasis on keeping voters' intentions private, and making the pervasive snooping by the government a little harder?
I won't be holding my breath waiting for it to happen, though.