VLC (and ffmpeg and so on) get by because a) they're based outside the US and b) they're too small to be worth suing. Actually, overall, small non-professional projects provide a net *benefit* to owners of codec patents.
Suing VLC or ffmpeg would be entirely pointless. Any codec patent holder could sue either project into oblivion in a wet weekend, but they choose not to. Do the cost/benefit analysis yourself. Benefit? Basically zero. The project stops existing. Perhaps, oh, three people who formerly used it actually have the money and inclination to pay you the codec licensing fee. All the rest just go use Theora or wait for the next grey encoder to pop up, which it inevitably will.
Costs? Many and significant. First, giant piles of terrible publicity: you're the big bad wolf attacking the plucky little guy. Second, boring practical costs - paying lawyers ain't cheap, and the authors of VLC and ffmpeg are not exactly hitting it rich at the low, low price of $0 per copy: you may get award seventy gazillion dollars in damages, but you will not see a single one of those dollars.
Third and finally, you actually do net damage to your customer pool. The existence of amateur-hour projects for casual users gives your product a boost. MP3 benefited (and benefits) enormously from casual private use for which MPEG-LA chose not to charge. When big rich operators go into MP3 encoding or decoding - because of the big ready-made user base - MPEG-LA bloody well *does* choose to charge, and charges lots of moolah. MPEG-LA don't come knocking on your door when you encode some songs with a grey-market application, but they charge commercial encoder makers, decoding implementations, MP3 player manufacturers and every other significant commercial operation a ton of money.
h.264 has benefited from much the same effect; a large amount of enthusiast video - ripped TV shows, fansubs and so on - is in h.264 format, meaning many people are set up to play the format and many tinkerers who will grow up to have real jobs are familiar with encoding using it. Both of those things help the format grow into areas where it is used by operators sufficiently big and rich to be required to pay for it.
Mozilla is certainly big enough and significant enough that it'd be worth suing, if it decided to just implement patented codecs and hang the consequences. It's too big to benefit much from the 'plucky little guy' effect in the court of public opinion, it's big enough to have enough money to at least break even on the legal costs, and it's big enough that there's actual strategic value in stopping it from using the codec without a license.