Oh, that's OK then
Sadly that utopia will never happen. AVs will never be able to cope with complex urban environments without the roadway being entirely segregated. The likely outcome, as with jaywalking laws in the US, will be for pedestrians and non-AV users to be corralled away from the AV roadway, in the same way that they're banned from motorways.
It seems to me that the best approach would be for a mandatory safety parameters to be baked in to all AV control systems but the issue there is that it would remove a differentiator for car makers (e.g. all cars would be the "Ultimate Driving Machine", not just BMW).