Road maintenance comes out of local council budgets so we do all pay for the roads, and while vehicle owners also pay VED, etc, it's not entirely true that road users pay for the majority of the cost of the roads (a big top rate earner pays more in income tax than most will spend on fuel/insurance/etc - should they get priority over one other individual?). Remember also that most cyclists also own cars, pay VED, insurance, fuel, but choose not to drive everywhere (I include myself in this category) - it's not fair to use the "I pay my road tax" argument in this situation either.
I believe the best estimate (though there is a lot of uncertainty) on the amount of damage done to a road is something like it being proportional to the fourth power of the axle load of the vehicle. So, if you want everyone to pay proportionally for their damage to the roads, peds and cyclists alike would owe about a penny over their lifetime for every pound of VED that covers the wear caused by a car. Short of global road tolls, this is probably one of the fairer ways of paying for road maintenance.
Of course, the road haulage industry wouldn't like this much as they would pay something like 1000-10000 times the "damage tax" compared to a car....