It's literally impossible to remove all the cariogenic bacteria from your teeth
In simple terms: the bristles of the tooth brush are too big to get into all the little nooks and crannies.
i.e. there is a very complex series of fissures on the occusal surfaces of teeth. (The biting surface). This is why dentists do fissure sealants. Also, the interproximal areas cannot be properly cleaned (with tooth floss and stuff). [by properly - I mean 100% cleaned, flossing does help though]
(You need to look at a tooth surface with a microscope or a big magnifying glass to see what I mean)
There are loads of bacteria on your tooth surfaces (not all bacteria are bad'). When sugar is there, they 'eat' it and produce acid. It's the acid that 'rots' teeth.
If you don't want your teeth to rot, simply try and always eat stuff like chocolate at meal times. (Cos you produce saliva which buffers the acid & protects your teeth).
What do you think is worse for your teeth? Eating a packet of smarties at once? Or eating them slowly over an afternoon (maybe having one every few mins)
Answer: Eating them all at once is better for your teeth (surprisingly)
You might want to Google the Stephan Curve (look for Dr S Hogg's Oral Health website at Newcastle Uni Dental School which explains this)
(also avoid flavored mineral water - this is deceptively cariogenic)
[Thanks to Prof Jimmy Steele, Prof Russell & Dr Hogg who explained this to me before I emigrated from England]