The serial number on the ballot paper (with your vote on) can indeed be matched to the serial number on the stub which it was torn from. I can't remember how it was done in the London elections (with the electronic counting system), but I think the serial number was noted so that it could be matched. Technically, they could match up all the ballot papers and produce a list of how everyone voted. In practice they only bother with votes for candidates that are considered contentious or seditious or whatever.

This came up on one of the UK Usenet groups in the 90s, and someone said they knew someone (yeah, yeah, I hear you all say) who was responsible for matching up the ballot papers cast for the Communist Party candidate in a particular constituency, and forwarding the names and addresses of the voters to MI5, CIA, BOSS, Mossad etc.