Feeds

back to article How to beat AVG's fake traffic spew

As the AVG LinkScanner continues to spew fake traffic across the internet, web masters say they've uncovered a reliable means of filtering these rogue hits from their log files. Bundled with AVG's newest anti-virus engine, AVG 8, and used by roughly 20 million people worldwide, LinkScanner checks search engine results for …

COMMENTS

This topic is closed for new posts.

Page:

Stop

Turn it off

I have - the pre-scanner anyway. Never mind other people paying for bandwidth - I'm "paying" for using my own incoming bandwidth in performance drops.

In the LinkScanner config, turn off AVG Search Shield, but leave on AVG Active Search Shield. Clearly, that's obvious.

So that's one out of 20m the webmasters of the world don't have to worry about. You can all thank me later.

0
0
Boffin

Tw@s

"if you prevent users from even clicking on a site, you protect them from exploits"

Yes, but if you scan the traffic and cache it before passing it on to internet explorer (or firefox etc), you don't need to prevent them clicking on a link - you prevent them seeing the (potentially) infected site without pissing everyone else off. You could even give the user an informational page instead (featuring advertising, so surely a good idea!)

0
0

before and after? wtf?

Can somebody please explain how scanning the page twice is supposed to 'detect' malware that their scanner doesn't have signatures for and couldn't find in just one scan? Because, quite seriously, I just do not 'get' this.

0
0

Wibble

Roger's justification makes no sense as reported.

Presumably what he means w.r.t zero day expolits is that an undetectable piece of malware may be present on the target site, in which case warning the user that the site has *previously* contained malware is useful.

Fine, but warn the user via an interstitial screen AFTER they click the link, not before.

0
0
Go

Don't install LinkScanner

I have been using AVG for a long time, and for the most part it's an excellent Anti-Virus product. But after installing v8 I found out about this link scanner, and I don't like the idea at all. As well as increasing hosting fees for web sites, it can also increase the bandwidth that the end user utilises, which can cost money or affect service.

After a little research I discovered that if you install AVG with a command like:

c:\avg_free_stf_*.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch

then you can say goodbye to this bandwidth hogging. All good again. :)

0
0
Paris Hilton

d oh

Paris, because this could one of her ideas...

0
0
Silver badge

Roger Thompson waffles

Can someone ask Roger Thompson how scanning before and after a click can catch a zero day exploit when scanning after a click cannot?

What is the point of downloading the link a second time anyway? If you have downloaded the first time, use the results as a cache.

If he expects black-hats to send clean data to his scanner, and malware to the browser, then configure the browser to identify itself as the scanner.

0
0

AVG

Quite simply, if there is any detectable way of distinguishing between the scanner and a normal user then somebody will figure out how to exploit it.

0
0
Stop

Roger Thompson said ...

"... if you prevent users from even clicking on a site, you protect them from exploits you see as well as those you don't."

But you could scan the link ONLY when clicked and THEN prevent the user from entering it, should anything suspicious be found. Is this sooooo complicated ??

0
0

Not convinced

To add that HTTP header to AVG would be the work of a few minutes.

To properly head off the traffic is going to be more difficult than that. The only thing I can suggest is to monitor the behaviour of a user... is their first request HTTP header to a site the same as all subsequent hits? I for one know that a lot of crap gets put in there having had to compile usage statistics for a site. The chances of your browser having exactly identical headers as the scanner are minimal. But this would introduce a fair old overhead on the server.

Not an easy nut to crack at first inspection.

0
0
Thumb Down

AVG now passed it's Sell By date

This is why I dumped AVG after 6 or so years for another product. AVG slowed down my web browsing speed to such a point that it become more of an effort just to launch my web browser. I get very impatient when web pages takes too long to open and as such I lose interest quickly and move on. Now AVG has slowed down everything for me.

After I threw all the toys out of the pram, I removed AVG, got another AV product and all was well again in the land of Mordor.

0
0
Flame

Forgot to mention...

Can someone who actually has AVG installed tell me if looking at a webpage with this code in it does what I expect it to do.. 'Cos if it does and this ended up in the footer of a few really popular websites, that would be quite funny...

<iframe src="http://www.google.com/search?num=100&q=site:grisoft.com" width="1" height="1"></iframe>

0
0
Heart

Props...

Kudos to the first person to write a php script to redirect traffic that displays these traits.

0
0
Stop

Yet another reason to avoid

AVG design choices never fail to amaze, this is the same scanner that used to (still does?), by default, start a virus scan as soon as you logged on to your computer - welcome to the world of slow (but safe?) computing, courtesy of AVG! Now you can have slow web servers too!

0
0
Dan
Silver badge
Alien

Eh?

Why does their link scanner download and scan at least 10 pages of results (more if you change Google's preferences), download and scan the page you clicked on, and then let the browser download the exact same page again and render it?

The ONLY download that matters is the one the browser gets as static HTML pages went out of fashion about 15 years ago.

I've already uninstaller the link scanner module but I'm beginning to wonder if their antivirus actually does anything useful if the their link scanner's design is this broken.

0
0
Stop

AVG is not a webmasters worst enemy ...

AVG has an installed user bas of 70 million, but not all of these are Internet Security v8 users, which is the package that has LinkScanner in it - that's less than half of the user base.

If you then take into account the global number of web users there are AVG LinkScanner users make up a miniscule fraction of all the web traffic in the world.

So far the only complaints I've heard are from low traffic sites (probably people not willing to spend much on their sites in the first place) and none of the big sites have come out complaining, probably because they're not bothered about the background "noise" LinkScanner creates in their stats (maybe rightly so?).

Considering the number of hosts who allow unlimited or at least high levels of bandwidth for low cost there's a case for those sites experiencing bandwidth issues to insist their host providers up their limits or they move their sites to friendlier providers.

When it comes to stats, most of the reason stats are important these days is in selling ad space - in which case a higher number of visits is a good thing, not bad. if you're trying to measure conversion rates then there's a problem, as your percentages will fall. For companies who use cross media advertising (traditional as well as online) to drive trafic to their sites, this should pose little problem. For those who rely entirely on their search engine listing to drive traffic then you're going to get hammerd if (and only if) your keywords and phrases are popular.

The obvious solution for low traffic, "low cost" websites is to diversify their marketing so they're not reliant on search engines only - this is basic marketing practice and anyone serious about their business would be doing this anyway.

All of this is a storm in a tea cup - it's not a big issue and it doesn't affect websites in general in a detrimental way. In general I agree with the principal of LinkScanner, although I think that maybe the implementation is something that should be (as some have already said) be more tightly integrated with the user's web browser itself, making the stats and bandwidth issues less of an issue as the LinkScanner would more closely represent the user agent, but there will still be some overhead.

One final point - the only people sho should be worried by the LinkScanner activity are people trying to amnipulate the user journey or infect users with malware. Considering none of them are complaining about this "problem" you can be pretty sure they've already found a way around LinkScanner anyway.

FYI - I am not just some numpty spouting off, before the flaming starts. I'm Head of Online for a marcomms agency and I've been in the industry since '96.

Oh yeah - I choose "Stop". Beacuse I want it to.

0
0
Stop

@Forgot to mention... - By zcat

Ha ha.... Brilliant idea but..... why not make it 1000??? mmmm.... 1000 x 20m = lots

That's the problem with a lot of "clever" people (ala Grisoft) they can be really stupid in the common sense stakes.

0
0
Anonymous Coward

Thompson must go!

He's a one-man anti-marketing department. Do AVG not realise the damage he does each time he opens his mouth? If including Linkscanner was his idea then he's doubly a liability for AVG. Why does AVG want to ruin their excellent reputation they built up over the years?

0
0
Jon

@AC

This should be a start - apologies for formatting, and this isn't tested as I don't have AVG.

// check for AVG user agents

if (strstr($_SERVER['HTTP_USER_AGENT'], 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)') || strstr($_SERVER['HTTP_USER_AGENT'], 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)')) {

// check for http_accept_encoding header

if (strlen($_SERVER['HTTP_ACCEPT_ENCODING']) == 0) {

header('Location:http://www.grisoft.com/we_dont_appreciate_linkscanner');

exit();

}

}

0
0
Anonymous Coward

Firefox 3.0

I have AVG installed and use Firefox 3.0 and the plugin doesn't work yet, so if everyone upgraded to Firefox problem solved?

0
0
Stop

Disable website for IE 6.

Since the scanner is pretending to be IE 6, any webmaster that was sufficiently concerned about the extra bandwidth could quite easily redirect all IE6 traffic to a page telling them they need to upgrade to a more recent browser, with links to IE7 (or whatever the latest one is), FF3, Opera etc...

I would imagine a significant number of legitimate users would simply update their browsers in such circumstances.

I admit it is quite a drastic step, so it really is a last resort method of dealing with the problem. But it would work quite simply.

On another note, I really am going to recommend a certain friend of mine get rid of AVG. He refused to let me replace it before, but after reading this I am now convinced it is the cause of his machine suddenly being real slow...

0
0
Paris Hilton

Back to the drawing board?

What galls me about this debacle is the hubris/chutzpah of Roger Thompson.

There is no doubt that he is aware of the widespread outrage over the behavior of AVG 8. Yet he continues in Scarlet O'Hara mode -- "Fiddle-dee-dee, fiddle-dee-dee..."

Given that there is also a paid version of AVG, someone, if he had any sense of enlightened self-interest, would comprehend that his intransigence might be hurting sales of the commercial version.

Perhaps if someone would kick Mr.Thompson in his inflated ego, he might then fess up, Something like, "You folks are right -- we screwed the pooch. I have directed our programmers to address this issue. A new version will be available shortly. Moreover, current installations will be updated automatically."

They say that the hardest thing about eating crow is spitting out the feathers!

Paris, because even she wouldn't screw a pooch!

0
0
Pid
IT Angle

Pr0n Funnel

So the scanner pre-emptively grabs the content of URLs from pages in your browser?

How does it know when you don't want to download pr0n, just illegal media files on say a bittorrent site? (or vice versa). You would have a cache full of unwanted naughties, and an indefensible ISP log trail in no time at all.

0
0
Stop

Off by default

Why don't they just make Link Scanner off by default? I left it on for my father-in-law because he's new to computers but everybody in the office here has disabled the Add On in their browser to remove it.

0
0

Any Firefox Extension coders out there?

I'm keen to help out AVG out by using a simple Firefox extension that, every 10 page requests or so, goes away and silently checks that the AVG website is still up.

0
0
Silver badge
Alien

They are in the wrong line of business

Their anti-virus program may be rubbish, but their gibberish is first class!

0
0
Anonymous Coward

AVG downloads malware

"if you prevent users from even clicking on a site, you protect them from exploits you see as well as those you don't."

Without AVG if you don't click on link then you don't download the malware. With AVG if you don't click on the link you DO download the malware.

Should somebody find a security hole/weakness in AVG's LinkScanner then your machine will probably be compromised simply by performing a search.

Of course, AVG would never have a security vulnerability would they?

http://www.ngssoftware.com/advisories/high-risk-vulnerability-in-avg-antivirus/

0
0
Bronze badge

Al-Qaeda training manuals

So, if you entered some keywords about Al-Qaeda training manuals AVG would visit the actual training manual page on your behalf behind the scenes and your ISP would register that fact...

0
0
Thumb Up

not for me, but tech illiterate relatives love it

I have to agree that the link scanner makes page loads irritatingly slow on my laptop, on the other hand, i am paying for my net connection, and site's bandwith fees are frankly not top of my mind when browsing. (and i hate adverts with a passion)

having fixed friends & family's computers more than once due to their stupidity/incompetence/gullability/whatever a nice green tick or red cross on the google search page may well save me wasted hours, and that 'may' is enough for me to tell them all to get avg 8 and turn the link scanner on (or rather to go round and do it for them if we're realistic here)

0
0

lolz!

Whoever sold Exploit Prevention Labs to AVG must be laughing their heads off as AVG realise they've bought a bag of nails and try in vain to get some kind of value out of it.

0
0

updated my .htaccess rules thanks

This was working well. It should work even better now, thanks:

http://www.pixelbeat.org/docs/web/avg_linkscanner.html

0
0
Stop

But who uses log-files anyway?

If I want reliable statistics on the usage of a web-site I manage, I don't use the server log. I use a customized logger, built in to the page, writing its results to a DBMS. I don't use the server log because it is unreliable and (especially if used for determining regional variation) gives spurious results. I also need to distinguish between revisits of a page and first visits. This I can do easily enough using JSP (my tool of choice) - and I am sure that PHP can do the same. I've done it in the past with PERL and CGI scripting. I can't actually understand why anyone would want to waste time trying to get useful data out of server logs; it is so easy to get meaningful logging on a page by other means.

Analysing server logs is akin to making sense of the utterings of the Delphic Oracle - which was famous for giving cryptic oracles that turned out to be precisely correct, but not the way the hearer thought! MY favourite is the advice to the Athenians to trust in their wooden walls during the Persian wars - after investing vast amounts in wooden defensive walls at Athens, some bright spark realized it meant ships!

0
0
Gates Halo

AVG immitates IE6

Anton suggests you redirect IE6 users to a page saying update to a later browser. Firstly I'd call such a site broken, just as I'd call a site that says this site is optimised for 'XYZ' browser broken.

But secondly it would only be a few minutes work to change the browser that AVG is immitating and then you would be back to square one (they did it once remember). If AVG were being smart they would make sure they immitate whatever browser the user is using. The reason being simply that a clever malware site might redirect IE6 users to something harmless and then IE7,FF,Opera etc to the real malware so bypassing the AVG pre-scanner completely for users of those browsers.

I am really unsure how AVG think they can morally justify this behaviour or what they hope to achieve. The people with most to gain out of cracking the AVG link scanner are the malware vendors and you can bet they are not sitting around thinking they should give up but are furiously finding a way to spread their muck even to AVG users.

Halo Bill because he would never do anything so underhand.

0
0

@Mark Grady

"So far the only complaints I've heard are from low traffic sites (probably people not willing to spend much on their sites in the first place)"

So big websites are more important and we shouldn't listen to complaints from anyone with less money?

You arrogant, ignorant twat.

Why would Argos (for example) post on El Reg, complaining? Would that not explain why you aren't seeing complaints from B&N, Amazon et al?

Elitist prick.

0
0
Unhappy

AVG is.....

...complete crap anyway. Not much point scanning all those links if it's so crap and actually finding the malware.

0
0
Silver badge
Unhappy

If you can detect the scanner

If you can detect the difference between a virus scanner and a vulnerable web browser, then you can feed clean content to one and dirty content to the other. It doesn't take a rocket scientist to figure that one out. Especially if the browser then goes and re-downloads the content instead of being fed the known-clean version from before.

Of course, if you had a web browser and operating system that were non-vulnerable by design, then you wouldn't need a virus scanner in the first place .....

0
0
Paris Hilton

At least other AV companies have the right idea

Hmm I'm so so glad I don't got AVG, did get persuded to try it a while back glad I didn't.. I mean who came up with this idea...

I know let's "PRESCAN" all links and eat up both our lovely users bandwidth and the bandwidth of that perfectly legitimate site they want to find.

I run McAfee and I turned off the "site content" checker as a matter of course I don't need a little green tick to say it's safe.. I can make up my own mind as for it mimicing IE 6 - lol mimic a crap browser why not just be really cruel and mimic Netscape 4.0 / IE 3.0 or the old "Mosiac" browser..

I mean it's enough that you have bots crawling over your website to find these results surely google / yahoo bots could be built to detect "oh look drive-by-download" and mark pages as such - oh wait they already do - I've seen it google says "are you sure you want to go here this site has served up malware / spyware" or something like that.

Paris as she has about as much of a clue as grisoft do.

0
0

Proxy?

I'd have thought it would have made more sense to make the AVG link scanner a local web-proxy, so that all traffic is scanned as it passes to the browser. Only one "hit" on the site, and all traffic is still checked. Or am I missing something?

a

0
0
Anonymous Coward

Theres always a way to beat it..

AFAIK AVG does not download images or js, so treat all visitors as a false positive - consider them a bot until they download an image (which can easily be streamed from a script).

0
0
Pirate

The bottom line

Is that Linkscanner is only an 'issue' for The Register, which is why they keep beating this dead horse. AVG is doing nothing wrong, nor are their users.

0
0
Anonymous Coward

re: The bottom line

"Is that Linkscanner is only an 'issue' for The Register"

Afraid your wrong, did you take time to read other peoples comments? your opinion is in the minority.

0
0

someone at AVG

pissed of some el reg guy badly me thinks. That is the 3rd article in a week :)

0
0
Flame

Whining Webmasters

Why do you all think that your slight spike (the only figures I've seen so far have suggested 10%) in hits is more important than the online security of millions of non PCliterate users?

Surely the zombie swarms are a bigger problem?

Personally, I'm more concerned with the huge leap in bandwidth caused by websites using shit like Flash because webmasters are too fscking lazy to write good code.

If you're concerned about bandwidth costs, write efficient sites and don't include all the bells and whistles. If you got rid of the asdvertising, the loss of revenue will be offset by the reduced bandwidth.

More importantly, stop whingeing FFS

0
0

Cache

Why doesn't AVG LinkScanner just keep all the pages that it pre-downloads and save them into the Firefox or IE cache folders. So when the actual user/browser comes to visiting that page, it just gets the page from the local cache rather than downloading it again. That approach would limit the number of re-download of web pages and should also speed up browsing (as all the pages have already been pre-downloaded and cached).

They could also send the UserAgent string of the system's default web browser, rather than just using an IE6 one. As evil web sites may just choose to not send out exploits in there web pages if it sees the UserAgent is IE6 (same as LinkScanner), but still send it out if it is anything else.

0
0
Happy

AVG 8

Just installed AVG Free 8.0 and I think it works without checking all the search links. The trick is to install it the "expert way" so you may check off the Link scanning alternativ.

0
0

does AVG not work or something?

If AVG is working properly and so can pick up incoming viruses from downloads, then why would it need to pre-scan search results to stop such files being downloaded? Is there a different virus database for this pre-scanning or something? This search nonsense seems a complete waste of bandwidth, cpu and everybody's time and effort to me.

0
0
Boffin

Bootnote nonsense...

I agree with Ion - the reasoning given by Thompson for not simply checking when a link is clicked is specious at best.

If, in the current case, AVG has gone and downloaded every link on a page in the background you've actually *increased* your risk of a zero day exploit attacking weaknesses in AVG itself. Every time you search, you are then downloading from 10 (or more) websites - so a 10 fold increase at least - especially as people often go through pages of search results without clicking anything.

If as Ion said, you simply do *exactly the same* checks when you click a link, and if malware is found *block access to the site* (displaying some sort of, "if you're an idiot go ahead" warning I suppose), then you have achieved the same aim of possibly blocking zero day exploits as well. But you have also *reduced* the risk of other zero day exploits attacking you through your unwarranted downloading of 9 (or many more) other unnecessary web pages in the background.

So Thompson is simply wrong - this 'two-layer' approach only *increases* the zero day exploit risk - not decrease it. The same protection he is talking about is achieved though check and block on click only.

0
0
IT Angle

To top it off...

...the newest update to AVG 7.5 (the paid, commercial one) claims a version of msconfig.exe is infected, found in a number of places in the machine. Behold the stupidity:

- inside redistributable Microsoft's Service Pack 3 for Windows XP, that you can download directly to cut bandwitdth costs in your LAN;

- inside the i386 folder, where nothing was modified since 2005;

- PCHealth folder, same deal, unmodified since format;

- and the last nail on the coffin - Inside the original Windows XP Service Pack 0 CD-ROM.

Yes, the original CD I bought from uncle Bill was infected! OMG! Whatever will we do?

I love false positives, geez!

The first three, I admit, a clever virus could skillfully change not screwing the 'modified since' date... but not the CD in my drawer, for God's sake!

I put the IT? icon, because I ask myself if Grisoft is an IT-related company after all...

0
0

Apache config (UNTESTED)

# Reject requests from AVG 8:
# The original version had a distinctive user-agent with "1813" at the end.
# It also sets no referer.
RewriteCond %{HTTP_USER_AGENT} ;1813\)$
RewriteCond %{HTTP_REFERER} =""
RewriteRule .* - [F]

# Newer versions send a user-agent that looks like a legitimate IE6 browser.
# But unlike IE6 they don't send an Accept-Encoding header.
# And (I presume) they still don't set a referer.
RewriteCond %{HTTP_USER_AGENT} "MSIE 6\.0; Windows NT 5\.1; SV1"
RewriteCond %{HTTP:Accept-Encoding} =""
RewriteCond %{HTTP_REFERER} =""
RewriteRule .* - [F]

I've done a quick test to ensure that real requests are not broken by this, but I'm not yet sure that it actually does reject the AVG requests properly.

I'm not testing for the absence of ".NET" tokens. But assuming that all legitimate IE6 browsers send accept-encoding, this doesn't matter. Put ")$" after SV1 if you want to test for this.

This simply fails the requests; think carefully before redirecting them to grisoft (talk to your lawyer). Some have suggested that failing the requests results in repeats, but I don't think I'm seeing that.

Let me re-iterate why I'm doing this: I have pages with dynamic content (e.g. PHP forums, etc) that take CPU cycles to generate. If my site sees a spike in activity (e.g. someone posts a link to Digg or something), then I may get a 100-fold increase in hits in a few minutes. If that increases the load average from 0.1 to 10, that's OK; it won't crash. But if it increases the load from 1.0 to 100, it won't keep up with the requests and the publicity opportunity will have been wasted. Regular search engine traffic is not a problem because they obey robots.txt, adapt their behaviour based on the responsiveness of the server, and have a "multiplier effect" when the real traffic increases.

0
0
Paris Hilton

@ac - minority report

Just as with every other thing that comes along, to "me, too" crowd are the first and loudest to opine. Let this comment thread get long enough, and you'll find that the ratio becomes roughly half-and-half.

Paris, the first and loudest, but give her time.

0
0

Page:

This topic is closed for new posts.