Re: Lack of a secret ballot is a greater problem
I worked as a central poll supervisor in our last federal election, which includes training for every single role in a polling station. These are very well defined, and *nothing* happens without two people supervising it... and specific people are responsible for the ballots for each poll.
The people who verify your identity have no record of your ballot, and it is illegal for anyone to mark a ballot in any way that makes it identifiable.
Advanced votes or votes at a different location are enclosed in an envelope with all the data to identify the voter, and to detect any duplicate votes. The inner security envelope that holds the ballot is not marked in any way, and is only opened after the polls close, by the poll clerk and assistant poll clerk for the poll in which your vote counts. These people never see the outer identity envelope which is filed elsewhere for audit purposes. The ballots are mixed in with all the other identical ballots before counting starts, so no one knows if a given ballot is received from an advanced poll, remote poll, or regular poll - all they know is that other people confirmed that the voter lived in the poll area that they are counting.
Counting generally involves a poll clerk, assistant poll clerk or other electoral officer, and scrutineers from any of the parties that wish to send one (usually the biggest three to five parties in a riding), all of whom can observe (but not touch) the ballots and monitor the counting.
Everyone working on the election is sworn to secrecy, and there are penalties prescribed for breaches.
Interestingly, only the poll clerk can decide on whether a ballot follows the rules for validity or not. No one, not the Central Poll Supervisor, not the Returning Officer (whose powers vaguely resemble those of an old time captain of a ship at sea - the RO is the final authority for the conduct of the election in their riding - and no government official or politician can over-rule them) can tell the poll clerk what decision to take, though they can point to the regulations and instructions that may be relevant. The appeal of a poll clerk's decision is in the hands of a judicial review, if needed.
There's a lot more, but it is quite possible to build a secure 'double blind' electoral process where nobody can tie a vote to a person.... and cheating the system is somewhere between extremely difficult and next to impossible.
For example, no ballot is valid unless it is initialled by the poll clerk - who must do it exactly the same way on each ballot, as they hand it to the voter. The ballot boxes are sealed with signed and serial numbered seals to prevent opening them, and they are never unsupervised by the poll clerk or assistant poll clerk once they contain ballots, until they are opened and counted.... then they are resealed and placed in secure storage by the RO.
All the ballots are counted and signed for at each stage where they change hands, and the numbers have to balance before and after the vote.
I should dig out my training books and review the process, because I know I am missing some of the subtle details and precautions... but at the time I was impressed at how well everything was tracked and audited while maintaining complete secrecy of the actual votes.
Fraud protection is very carefully decoupled from the actual votes, and is tied to the act of being identified as a voter and being handed a blank ballot.