The analogy of having public water taps on streets is flawed. If a council has provided a tap for public use then you're allowed to use it. If my neighbour has an outside tap with a sign saying 'free water, help yourself' then it's reasonable to assume you can use it. If I have an outside tap visible from the road that doesn't have a sign saying it's for public use but it's not secured with a lock and key, that doesn't mean you should assume you can use it. The fact that it reflects light into your eyes doesn't allow you to use the water that I have paid for.
The principle is the same for wifi, if you're not sure whether you're allowed to use an open wireless network then don't. If it's been intended for public use then I'd expect that to be reflected in the SSID or to see some signs up somewhere saying 'free wifi for customers.' (Incidentally, this raises the moral question as to whether you should be allowed to use the open wifi provided for customers of the cafe you live above when you've never been in there, but that's for another time.)
If someone leaves a bag full of money on a table in a pub and I take some of it then I'm stealing. Just because it's open and I can see it doesn't give me the right to take what isn't mine. Just because your laptop connects automatically to an unsecured network doesn't give you the right to start using someone else's bandwith. If you see you've connected by mistake then disconnect. It's not difficult. I expect the commenters saying it's the sole responsibility of the network owners are the same people who defend virus and malware coders saying that if people are too stupid to protect themselves they deserve to be infected. Newsflash: not everyone is as computer literate as you! Some people might be unaware that it needs securing! Shock horror! Should we ban them from having computers and keep the internet a pure utopian society for IT professionals?!
And does anyone seriously believe the police dropped the investigation of murders and rapes to come and arrest these people? Come on now, stop being silly.