back to article How to beat AVG's fake traffic spew

As the AVG LinkScanner continues to spew fake traffic across the internet, web masters say they've uncovered a reliable means of filtering these rogue hits from their log files. Bundled with AVG's newest anti-virus engine, AVG 8, and used by roughly 20 million people worldwide, LinkScanner checks search engine results for …

COMMENTS

This topic is closed for new posts.
  1. Andrew Bolton
    Stop

    Turn it off

    I have - the pre-scanner anyway. Never mind other people paying for bandwidth - I'm "paying" for using my own incoming bandwidth in performance drops.

    In the LinkScanner config, turn off AVG Search Shield, but leave on AVG Active Search Shield. Clearly, that's obvious.

    So that's one out of 20m the webmasters of the world don't have to worry about. You can all thank me later.

  2. Anonymous Coward
    Boffin

    Tw@s

    "if you prevent users from even clicking on a site, you protect them from exploits"

    Yes, but if you scan the traffic and cache it before passing it on to internet explorer (or firefox etc), you don't need to prevent them clicking on a link - you prevent them seeing the (potentially) infected site without pissing everyone else off. You could even give the user an informational page instead (featuring advertising, so surely a good idea!)

  3. zcat

    before and after? wtf?

    Can somebody please explain how scanning the page twice is supposed to 'detect' malware that their scanner doesn't have signatures for and couldn't find in just one scan? Because, quite seriously, I just do not 'get' this.

  4. John Latham

    Wibble

    Roger's justification makes no sense as reported.

    Presumably what he means w.r.t zero day expolits is that an undetectable piece of malware may be present on the target site, in which case warning the user that the site has *previously* contained malware is useful.

    Fine, but warn the user via an interstitial screen AFTER they click the link, not before.

  5. Paul R
    Go

    Don't install LinkScanner

    I have been using AVG for a long time, and for the most part it's an excellent Anti-Virus product. But after installing v8 I found out about this link scanner, and I don't like the idea at all. As well as increasing hosting fees for web sites, it can also increase the bandwidth that the end user utilises, which can cost money or affect service.

    After a little research I discovered that if you install AVG with a command like:

    c:\avg_free_stf_*.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch

    then you can say goodbye to this bandwidth hogging. All good again. :)

  6. echox
    Paris Hilton

    d oh

    Paris, because this could one of her ideas...

  7. Flocke Kroes Silver badge

    Roger Thompson waffles

    Can someone ask Roger Thompson how scanning before and after a click can catch a zero day exploit when scanning after a click cannot?

    What is the point of downloading the link a second time anyway? If you have downloaded the first time, use the results as a cache.

    If he expects black-hats to send clean data to his scanner, and malware to the browser, then configure the browser to identify itself as the scanner.

  8. Andy Worth

    AVG

    Quite simply, if there is any detectable way of distinguishing between the scanner and a normal user then somebody will figure out how to exploit it.

  9. Ion Iliescu
    Stop

    Roger Thompson said ...

    "... if you prevent users from even clicking on a site, you protect them from exploits you see as well as those you don't."

    But you could scan the link ONLY when clicked and THEN prevent the user from entering it, should anything suspicious be found. Is this sooooo complicated ??

  10. Jamie Davis

    Not convinced

    To add that HTTP header to AVG would be the work of a few minutes.

    To properly head off the traffic is going to be more difficult than that. The only thing I can suggest is to monitor the behaviour of a user... is their first request HTTP header to a site the same as all subsequent hits? I for one know that a lot of crap gets put in there having had to compile usage statistics for a site. The chances of your browser having exactly identical headers as the scanner are minimal. But this would introduce a fair old overhead on the server.

    Not an easy nut to crack at first inspection.

  11. Juan
    Thumb Down

    AVG now passed it's Sell By date

    This is why I dumped AVG after 6 or so years for another product. AVG slowed down my web browsing speed to such a point that it become more of an effort just to launch my web browser. I get very impatient when web pages takes too long to open and as such I lose interest quickly and move on. Now AVG has slowed down everything for me.

    After I threw all the toys out of the pram, I removed AVG, got another AV product and all was well again in the land of Mordor.

  12. zcat
    Flame

    Forgot to mention...

    Can someone who actually has AVG installed tell me if looking at a webpage with this code in it does what I expect it to do.. 'Cos if it does and this ended up in the footer of a few really popular websites, that would be quite funny...

    <iframe src="http://www.google.com/search?num=100&q=site:grisoft.com" width="1" height="1"></iframe>

  13. Anonymous Coward
    Heart

    Props...

    Kudos to the first person to write a php script to redirect traffic that displays these traits.

  14. Conor Murray
    Stop

    Yet another reason to avoid

    AVG design choices never fail to amaze, this is the same scanner that used to (still does?), by default, start a virus scan as soon as you logged on to your computer - welcome to the world of slow (but safe?) computing, courtesy of AVG! Now you can have slow web servers too!

  15. Dan Silver badge
    Alien

    Eh?

    Why does their link scanner download and scan at least 10 pages of results (more if you change Google's preferences), download and scan the page you clicked on, and then let the browser download the exact same page again and render it?

    The ONLY download that matters is the one the browser gets as static HTML pages went out of fashion about 15 years ago.

    I've already uninstaller the link scanner module but I'm beginning to wonder if their antivirus actually does anything useful if the their link scanner's design is this broken.

  16. Mark Grady
    Stop

    AVG is not a webmasters worst enemy ...

    AVG has an installed user bas of 70 million, but not all of these are Internet Security v8 users, which is the package that has LinkScanner in it - that's less than half of the user base.

    If you then take into account the global number of web users there are AVG LinkScanner users make up a miniscule fraction of all the web traffic in the world.

    So far the only complaints I've heard are from low traffic sites (probably people not willing to spend much on their sites in the first place) and none of the big sites have come out complaining, probably because they're not bothered about the background "noise" LinkScanner creates in their stats (maybe rightly so?).

    Considering the number of hosts who allow unlimited or at least high levels of bandwidth for low cost there's a case for those sites experiencing bandwidth issues to insist their host providers up their limits or they move their sites to friendlier providers.

    When it comes to stats, most of the reason stats are important these days is in selling ad space - in which case a higher number of visits is a good thing, not bad. if you're trying to measure conversion rates then there's a problem, as your percentages will fall. For companies who use cross media advertising (traditional as well as online) to drive trafic to their sites, this should pose little problem. For those who rely entirely on their search engine listing to drive traffic then you're going to get hammerd if (and only if) your keywords and phrases are popular.

    The obvious solution for low traffic, "low cost" websites is to diversify their marketing so they're not reliant on search engines only - this is basic marketing practice and anyone serious about their business would be doing this anyway.

    All of this is a storm in a tea cup - it's not a big issue and it doesn't affect websites in general in a detrimental way. In general I agree with the principal of LinkScanner, although I think that maybe the implementation is something that should be (as some have already said) be more tightly integrated with the user's web browser itself, making the stats and bandwidth issues less of an issue as the LinkScanner would more closely represent the user agent, but there will still be some overhead.

    One final point - the only people sho should be worried by the LinkScanner activity are people trying to amnipulate the user journey or infect users with malware. Considering none of them are complaining about this "problem" you can be pretty sure they've already found a way around LinkScanner anyway.

    FYI - I am not just some numpty spouting off, before the flaming starts. I'm Head of Online for a marcomms agency and I've been in the industry since '96.

    Oh yeah - I choose "Stop". Beacuse I want it to.

  17. Steve Renouf
    Stop

    @Forgot to mention... - By zcat

    Ha ha.... Brilliant idea but..... why not make it 1000??? mmmm.... 1000 x 20m = lots

    That's the problem with a lot of "clever" people (ala Grisoft) they can be really stupid in the common sense stakes.

  18. Anonymous Coward
    Anonymous Coward

    Thompson must go!

    He's a one-man anti-marketing department. Do AVG not realise the damage he does each time he opens his mouth? If including Linkscanner was his idea then he's doubly a liability for AVG. Why does AVG want to ruin their excellent reputation they built up over the years?

  19. Jon

    @AC

    This should be a start - apologies for formatting, and this isn't tested as I don't have AVG.

    // check for AVG user agents

    if (strstr($_SERVER['HTTP_USER_AGENT'], 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)') || strstr($_SERVER['HTTP_USER_AGENT'], 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)')) {

    // check for http_accept_encoding header

    if (strlen($_SERVER['HTTP_ACCEPT_ENCODING']) == 0) {

    header('Location:http://www.grisoft.com/we_dont_appreciate_linkscanner');

    exit();

    }

    }

  20. Anonymous Coward
    Anonymous Coward

    Firefox 3.0

    I have AVG installed and use Firefox 3.0 and the plugin doesn't work yet, so if everyone upgraded to Firefox problem solved?

  21. Anton Channing
    Stop

    Disable website for IE 6.

    Since the scanner is pretending to be IE 6, any webmaster that was sufficiently concerned about the extra bandwidth could quite easily redirect all IE6 traffic to a page telling them they need to upgrade to a more recent browser, with links to IE7 (or whatever the latest one is), FF3, Opera etc...

    I would imagine a significant number of legitimate users would simply update their browsers in such circumstances.

    I admit it is quite a drastic step, so it really is a last resort method of dealing with the problem. But it would work quite simply.

    On another note, I really am going to recommend a certain friend of mine get rid of AVG. He refused to let me replace it before, but after reading this I am now convinced it is the cause of his machine suddenly being real slow...

  22. Martin Maloney
    Paris Hilton

    Back to the drawing board?

    What galls me about this debacle is the hubris/chutzpah of Roger Thompson.

    There is no doubt that he is aware of the widespread outrage over the behavior of AVG 8. Yet he continues in Scarlet O'Hara mode -- "Fiddle-dee-dee, fiddle-dee-dee..."

    Given that there is also a paid version of AVG, someone, if he had any sense of enlightened self-interest, would comprehend that his intransigence might be hurting sales of the commercial version.

    Perhaps if someone would kick Mr.Thompson in his inflated ego, he might then fess up, Something like, "You folks are right -- we screwed the pooch. I have directed our programmers to address this issue. A new version will be available shortly. Moreover, current installations will be updated automatically."

    They say that the hardest thing about eating crow is spitting out the feathers!

    Paris, because even she wouldn't screw a pooch!

  23. Pid
    IT Angle

    Pr0n Funnel

    So the scanner pre-emptively grabs the content of URLs from pages in your browser?

    How does it know when you don't want to download pr0n, just illegal media files on say a bittorrent site? (or vice versa). You would have a cache full of unwanted naughties, and an indefensible ISP log trail in no time at all.

  24. Sean O'Connor
    Stop

    Off by default

    Why don't they just make Link Scanner off by default? I left it on for my father-in-law because he's new to computers but everybody in the office here has disabled the Add On in their browser to remove it.

  25. OneArmJack

    Any Firefox Extension coders out there?

    I'm keen to help out AVG out by using a simple Firefox extension that, every 10 page requests or so, goes away and silently checks that the AVG website is still up.

  26. Christoph Silver badge
    Alien

    They are in the wrong line of business

    Their anti-virus program may be rubbish, but their gibberish is first class!

  27. Anonymous Coward
    Anonymous Coward

    AVG downloads malware

    "if you prevent users from even clicking on a site, you protect them from exploits you see as well as those you don't."

    Without AVG if you don't click on link then you don't download the malware. With AVG if you don't click on the link you DO download the malware.

    Should somebody find a security hole/weakness in AVG's LinkScanner then your machine will probably be compromised simply by performing a search.

    Of course, AVG would never have a security vulnerability would they?

    http://www.ngssoftware.com/advisories/high-risk-vulnerability-in-avg-antivirus/

  28. Nigel R

    Al-Qaeda training manuals

    So, if you entered some keywords about Al-Qaeda training manuals AVG would visit the actual training manual page on your behalf behind the scenes and your ISP would register that fact...

  29. Anonymous Coward
    Thumb Up

    not for me, but tech illiterate relatives love it

    I have to agree that the link scanner makes page loads irritatingly slow on my laptop, on the other hand, i am paying for my net connection, and site's bandwith fees are frankly not top of my mind when browsing. (and i hate adverts with a passion)

    having fixed friends & family's computers more than once due to their stupidity/incompetence/gullability/whatever a nice green tick or red cross on the google search page may well save me wasted hours, and that 'may' is enough for me to tell them all to get avg 8 and turn the link scanner on (or rather to go round and do it for them if we're realistic here)

  30. Aidan Samuel

    lolz!

    Whoever sold Exploit Prevention Labs to AVG must be laughing their heads off as AVG realise they've bought a bag of nails and try in vain to get some kind of value out of it.

  31. Pádraig Brady

    updated my .htaccess rules thanks

    This was working well. It should work even better now, thanks:

    http://www.pixelbeat.org/docs/web/avg_linkscanner.html

  32. Paul Cooper
    Stop

    But who uses log-files anyway?

    If I want reliable statistics on the usage of a web-site I manage, I don't use the server log. I use a customized logger, built in to the page, writing its results to a DBMS. I don't use the server log because it is unreliable and (especially if used for determining regional variation) gives spurious results. I also need to distinguish between revisits of a page and first visits. This I can do easily enough using JSP (my tool of choice) - and I am sure that PHP can do the same. I've done it in the past with PERL and CGI scripting. I can't actually understand why anyone would want to waste time trying to get useful data out of server logs; it is so easy to get meaningful logging on a page by other means.

    Analysing server logs is akin to making sense of the utterings of the Delphic Oracle - which was famous for giving cryptic oracles that turned out to be precisely correct, but not the way the hearer thought! MY favourite is the advice to the Athenians to trust in their wooden walls during the Persian wars - after investing vast amounts in wooden defensive walls at Athens, some bright spark realized it meant ships!

  33. Nick Askew
    Gates Halo

    AVG immitates IE6

    Anton suggests you redirect IE6 users to a page saying update to a later browser. Firstly I'd call such a site broken, just as I'd call a site that says this site is optimised for 'XYZ' browser broken.

    But secondly it would only be a few minutes work to change the browser that AVG is immitating and then you would be back to square one (they did it once remember). If AVG were being smart they would make sure they immitate whatever browser the user is using. The reason being simply that a clever malware site might redirect IE6 users to something harmless and then IE7,FF,Opera etc to the real malware so bypassing the AVG pre-scanner completely for users of those browsers.

    I am really unsure how AVG think they can morally justify this behaviour or what they hope to achieve. The people with most to gain out of cracking the AVG link scanner are the malware vendors and you can bet they are not sitting around thinking they should give up but are furiously finding a way to spread their muck even to AVG users.

    Halo Bill because he would never do anything so underhand.

  34. Mark

    @Mark Grady

    "So far the only complaints I've heard are from low traffic sites (probably people not willing to spend much on their sites in the first place)"

    So big websites are more important and we shouldn't listen to complaints from anyone with less money?

    You arrogant, ignorant twat.

    Why would Argos (for example) post on El Reg, complaining? Would that not explain why you aren't seeing complaints from B&N, Amazon et al?

    Elitist prick.

  35. Bob Hoskins
    Unhappy

    AVG is.....

    ...complete crap anyway. Not much point scanning all those links if it's so crap and actually finding the malware.

  36. A J Stiles
    Unhappy

    If you can detect the scanner

    If you can detect the difference between a virus scanner and a vulnerable web browser, then you can feed clean content to one and dirty content to the other. It doesn't take a rocket scientist to figure that one out. Especially if the browser then goes and re-downloads the content instead of being fed the known-clean version from before.

    Of course, if you had a web browser and operating system that were non-vulnerable by design, then you wouldn't need a virus scanner in the first place .....

  37. Gordon Grant
    Paris Hilton

    At least other AV companies have the right idea

    Hmm I'm so so glad I don't got AVG, did get persuded to try it a while back glad I didn't.. I mean who came up with this idea...

    I know let's "PRESCAN" all links and eat up both our lovely users bandwidth and the bandwidth of that perfectly legitimate site they want to find.

    I run McAfee and I turned off the "site content" checker as a matter of course I don't need a little green tick to say it's safe.. I can make up my own mind as for it mimicing IE 6 - lol mimic a crap browser why not just be really cruel and mimic Netscape 4.0 / IE 3.0 or the old "Mosiac" browser..

    I mean it's enough that you have bots crawling over your website to find these results surely google / yahoo bots could be built to detect "oh look drive-by-download" and mark pages as such - oh wait they already do - I've seen it google says "are you sure you want to go here this site has served up malware / spyware" or something like that.

    Paris as she has about as much of a clue as grisoft do.

  38. fifi

    Proxy?

    I'd have thought it would have made more sense to make the AVG link scanner a local web-proxy, so that all traffic is scanned as it passes to the browser. Only one "hit" on the site, and all traffic is still checked. Or am I missing something?

    a

  39. Anonymous Coward
    Anonymous Coward

    Theres always a way to beat it..

    AFAIK AVG does not download images or js, so treat all visitors as a false positive - consider them a bot until they download an image (which can easily be streamed from a script).

  40. Anonymous Coward
    Pirate

    The bottom line

    Is that Linkscanner is only an 'issue' for The Register, which is why they keep beating this dead horse. AVG is doing nothing wrong, nor are their users.

  41. Anonymous Coward
    Anonymous Coward

    re: The bottom line

    "Is that Linkscanner is only an 'issue' for The Register"

    Afraid your wrong, did you take time to read other peoples comments? your opinion is in the minority.

  42. john doe

    someone at AVG

    pissed of some el reg guy badly me thinks. That is the 3rd article in a week :)

  43. Anonymous Coward
    Flame

    Whining Webmasters

    Why do you all think that your slight spike (the only figures I've seen so far have suggested 10%) in hits is more important than the online security of millions of non PCliterate users?

    Surely the zombie swarms are a bigger problem?

    Personally, I'm more concerned with the huge leap in bandwidth caused by websites using shit like Flash because webmasters are too fscking lazy to write good code.

    If you're concerned about bandwidth costs, write efficient sites and don't include all the bells and whistles. If you got rid of the asdvertising, the loss of revenue will be offset by the reduced bandwidth.

    More importantly, stop whingeing FFS

  44. Chris

    Cache

    Why doesn't AVG LinkScanner just keep all the pages that it pre-downloads and save them into the Firefox or IE cache folders. So when the actual user/browser comes to visiting that page, it just gets the page from the local cache rather than downloading it again. That approach would limit the number of re-download of web pages and should also speed up browsing (as all the pages have already been pre-downloaded and cached).

    They could also send the UserAgent string of the system's default web browser, rather than just using an IE6 one. As evil web sites may just choose to not send out exploits in there web pages if it sees the UserAgent is IE6 (same as LinkScanner), but still send it out if it is anything else.

  45. Art Ross
    Happy

    AVG 8

    Just installed AVG Free 8.0 and I think it works without checking all the search links. The trick is to install it the "expert way" so you may check off the Link scanning alternativ.

  46. Craig Wallace

    does AVG not work or something?

    If AVG is working properly and so can pick up incoming viruses from downloads, then why would it need to pre-scan search results to stop such files being downloaded? Is there a different virus database for this pre-scanning or something? This search nonsense seems a complete waste of bandwidth, cpu and everybody's time and effort to me.

  47. Anonymous Coward
    Boffin

    Bootnote nonsense...

    I agree with Ion - the reasoning given by Thompson for not simply checking when a link is clicked is specious at best.

    If, in the current case, AVG has gone and downloaded every link on a page in the background you've actually *increased* your risk of a zero day exploit attacking weaknesses in AVG itself. Every time you search, you are then downloading from 10 (or more) websites - so a 10 fold increase at least - especially as people often go through pages of search results without clicking anything.

    If as Ion said, you simply do *exactly the same* checks when you click a link, and if malware is found *block access to the site* (displaying some sort of, "if you're an idiot go ahead" warning I suppose), then you have achieved the same aim of possibly blocking zero day exploits as well. But you have also *reduced* the risk of other zero day exploits attacking you through your unwarranted downloading of 9 (or many more) other unnecessary web pages in the background.

    So Thompson is simply wrong - this 'two-layer' approach only *increases* the zero day exploit risk - not decrease it. The same protection he is talking about is achieved though check and block on click only.

  48. Luiz Abdala
    IT Angle

    To top it off...

    ...the newest update to AVG 7.5 (the paid, commercial one) claims a version of msconfig.exe is infected, found in a number of places in the machine. Behold the stupidity:

    - inside redistributable Microsoft's Service Pack 3 for Windows XP, that you can download directly to cut bandwitdth costs in your LAN;

    - inside the i386 folder, where nothing was modified since 2005;

    - PCHealth folder, same deal, unmodified since format;

    - and the last nail on the coffin - Inside the original Windows XP Service Pack 0 CD-ROM.

    Yes, the original CD I bought from uncle Bill was infected! OMG! Whatever will we do?

    I love false positives, geez!

    The first three, I admit, a clever virus could skillfully change not screwing the 'modified since' date... but not the CD in my drawer, for God's sake!

    I put the IT? icon, because I ask myself if Grisoft is an IT-related company after all...

  49. Phil Endecott Silver badge

    Apache config (UNTESTED)

        # Reject requests from AVG 8:
        # The original version had a distinctive user-agent with "1813" at the end.
        # It also sets no referer.
        RewriteCond %{HTTP_USER_AGENT} ;1813\)$
        RewriteCond %{HTTP_REFERER} =""
        RewriteRule .* - [F]
        
        # Newer versions send a user-agent that looks like a legitimate IE6 browser.
        # But unlike IE6 they don't send an Accept-Encoding header.
        # And (I presume) they still don't set a referer.
        RewriteCond %{HTTP_USER_AGENT} "MSIE 6\.0; Windows NT 5\.1; SV1"
        RewriteCond %{HTTP:Accept-Encoding} =""
        RewriteCond %{HTTP_REFERER} =""
        RewriteRule .* - [F]
        

    I've done a quick test to ensure that real requests are not broken by this, but I'm not yet sure that it actually does reject the AVG requests properly.

    I'm not testing for the absence of ".NET" tokens. But assuming that all legitimate IE6 browsers send accept-encoding, this doesn't matter. Put ")$" after SV1 if you want to test for this.

    This simply fails the requests; think carefully before redirecting them to grisoft (talk to your lawyer). Some have suggested that failing the requests results in repeats, but I don't think I'm seeing that.

    Let me re-iterate why I'm doing this: I have pages with dynamic content (e.g. PHP forums, etc) that take CPU cycles to generate. If my site sees a spike in activity (e.g. someone posts a link to Digg or something), then I may get a 100-fold increase in hits in a few minutes. If that increases the load average from 0.1 to 10, that's OK; it won't crash. But if it increases the load from 1.0 to 100, it won't keep up with the requests and the publicity opportunity will have been wasted. Regular search engine traffic is not a problem because they obey robots.txt, adapt their behaviour based on the responsiveness of the server, and have a "multiplier effect" when the real traffic increases.

  50. Alan W. Rateliff, II
    Paris Hilton

    @ac - minority report

    Just as with every other thing that comes along, to "me, too" crowd are the first and loudest to opine. Let this comment thread get long enough, and you'll find that the ratio becomes roughly half-and-half.

    Paris, the first and loudest, but give her time.

  51. James Pickett
    Stop

    Unforgiven

    "AVG has promised a fix"

    Sorry, guys, you're too late. I've replaced it with ClamWin, which is simple, effective and GPL'd.

    Go thou and do likewise.

  52. Chika
    Alert

    A couple of comments

    @Anton Channing

    I'd direct you to the comment by Nick Askew. Indeed, a number of perfectly good browsers which are quite capable of rendering a site have to imitate IE's headers purely because there are still idiots out there that insist on writing for specific browsers only and will lock out anything else. That doesn't excuse AVG at all, but it does give you an idea that the problem isn't entirely their fault.

    Having said that, I gave up using AVG about six months ago as I found that it was a bit of a hog wrt resources, and that it was woefully underachieving at actually catching viruses, something that came to light when I tried an alternative brand. This doesn't mean that I advise users of AVG to stop using it outright, since even AVG is better than nothing at all.

  53. James Pickett
    Happy

    @Mrak Gardy

    "trying to amnipulate the user journey"

    So that's what they're doing!

  54. Dave Silver badge
    Alert

    Facts

    I work on behalf of one of the larger local authorities. Not quite Amazon, but we count hits in the millions all the same.

    Speaking of which, they have doubled over the last month.

    Because of the infrastructure we could not implement any of the filtering suggestions mentioned so far, even if any of them did have any long-term viability.

    As we host our own servers, if this gets much worse we will need to buy an additional pipe/upgrade the existing one, and no, we are not running on DSL or even ADSL or ISDN...

    Our analytics package is important to us, but it is licensed per page view...

    We still use IE6 for all internal users, since IE7 broke so many applications, so we can't just shun them, no matter how nice that would be for me.

    I am not a shill for the register, but nor am I one of the AC's who are clearly paid by AVG. I haven't posted about this before, because my organisation does not want any bad publicity, but that does not mean that we are happy with what is going on, in fact I believe that our Lawyers are busy right now.

  55. tony72
    Flame

    LinkScanner is a great idea

    Numbskulls who are too dim-witted to realise that they can turn LinkScanner off are exactly the kind of people who might benefit from this kind of protection, so it's a brilliantly self-selecting feature.

  56. Anonymous Coward
    Anonymous Coward

    whinge

    I am optimistic that the market will win in the end and AVG man will eat humble pie.

    Probably the only reason they continue this crap software is because people are complaining about it, so many programmers think they know best...even when they know they are wrong. Look it's nothing personal, just stop this crap scanner for the following reasons:

    1. people who use it dont like it

    2. Web masters dont like it

    3. it does not help online privacy when web logs show users visiting nasty sites, when they didnt. Remember there are now rules for ISPs to keep this data...

    4. It is technically stupid. As the comments here show, it doesnt even do what it is intended to do correctly.

    btw. I am one of those poor website owners, and it's not so much my bandwith being eaten up, but I rely on my stats to spot trends, which this thing pisses all over.

  57. Jolyon Ralph
    Thumb Down

    Even if it wasn't a shit idea, it'd still be a shit idea.

    Ok. Firstly, Linkscanner is shit for many reasons. Because it's screwing up web statistics (i'm not too worried about this on my sites, except for the very horrible side-effect of artificially inflating the IE6 figures, making it a little longer before it can be consigned to the dustbin of history), and because it's eating bandwidth from customers.

    Secondly, even if that wasn't bad enough, it simply won't work. It's so ridiculously trivial for a malware author to get around* that you have the very serious problem of a false sense of security - a link doesn't get indicated as malware so you assume that it's safe.

    For example, linkscanner could be fooled by sending different content (infected or clean) based on various rules that could be reasonably defined to detect whether it is a linkscanner visit (blank referer, IE6, or various other methods), or even with the same content you could fool it - for example a simple javascript redirect to an infected page would almost certainly not get detected by linkscanner. Any but the dumbest of malware authors will work around linkscanner in minutes.

    It's smoke and mirrors. No real security benefit, sorry.

    Linkscanner should at the very least use the user-agent string that the users browser is using and nothing else. That is best as it avoids a way for the malware site to easily detect linkscanner content.

    But I think considering the collateral damage that this system causes I think it's probably best the whole thing is silently dropped.

    Jolyon

    (former AVG fan)

  58. Anonymous Coward
    Stop

    Good idea badly done...

    I think linkscanner does have its place, but like other people say the search results from google or wherever should be passed to the scanner only when clicked. It could do this by a simple redirect and a script that initiates the extension like it does already.

    Or alternatly you could put a link next to each search result with scan written on it so that it only scans when you ask it to, maybe it could even give you stats back from the site such as KB/sec how many popups and images and if there is flash, javascript and video and other potential annoyances on the pages.

    Make it do this with all links through a right clock menu might be useful as well, but whatever.

  59. Jolyon Ralph
    Boffin

    @Chika

    >a number of perfectly good browsers which are quite capable of rendering a

    >site have to imitate IE's headers purely because there are still idiots out

    >there that insist on writing for specific browsers only and will lock out

    >anything else.

    A perfectly good browser would let you specify this on a per-site basis as needed.

    Jolyon

  60. Mark Grady

    @Mark

    "Elitist prick" ... first time I've ever been called that - thanks! I'm really proud of that.

    BTW, notice you were to shy to put your full name against your comments.

    I agree, Amazon, et al, wouldn't post their comments here but you would have heard about it somewhere if it was causing them problems. Luckily I don't just read El Reg but cast my net much wider whn I qualify my comments.

    I'm not saying people with less money are less important (in fact I challenge you to find anywhere in my comment where I did) but if they're paying scalar bandwidth usage charges they're with hosts that are less than suitable for them. I went on to say that much in the next paragraph.

    As for "arrogant, ignorant twat" - I'm really careful not to be arrogant, for example I didn't put down anyone else's point of view out of hand because most of the comments made so far had been valid, to the point, sensible or just enquiring. I'm definitely not ignorant - my clients include micro businesses all the way up to multinationals so I have to be very aware of lots of different needs and budgets. As for twat, yes I am, but not on this.

  61. Steve Mann

    Marginal Websites "don't matter"????

    I have to agree with Mark on the "arrogant twit" front here.

    In the course of my leisure time I have recourse to a number of small, free sites that are of no interest to a casual user but will score high on Google if the right keywords are chosen. In one case that would be exaclty two words. It is entirely unreasonable that these sites should be "slashdotted" by a poorly written piece of software. If the bandwidth (carefully calculated by the webmaster-owners) is exceeded, the sites will be shut down for up to a month. How is this a fair or even a justifiable "solution" to the malware menace?

    "Poorly written"? Yes. When I started in the IT industry as an entry level programmer, I would have had my fingers broken for coding any algorithm that followed every path "just in case" instead of concentrating on the actual job to be done. And that, despite all the whingeing from AVG spokesdrones is what is happening here.

    There is absolutely no reason why this software could not do the exactly the same job it is doing now, but on clicked links only. The AVG user would be no more exposed to exploits than with this witless, poorly thought-out and outright toxic method of "clicking" every link "just in case".

    Only the fact that AVG bought in this "technology" makes the situation and AVG's seeming thick-headedness make any kind of sense. They must be desperate to not invest any more cash in the thing.

  62. James O'Shea
    Thumb Up

    @Mark

    You're both reasonable, and correct. This is just a storm in a teacup. I, for one, am not merely supporting AVG in this, I'm moving from AVG Free to paid-up versions for all Windows machines under my control. Yes, LinkScanner could be better implimented... but by 'better' I mean 'made impossible for someone at the other end to know that it's not a real user'. One reason why I'm paying up is to give them the chance to make it better.

  63. Rachel

    @Tony72

    Disabling/Enabling the link scanner is not that simple. I haven't attempted to reinstall to have the feature removed yet, but I do know if you simply "disable" the link scanner, then the virus program reports an alert icon in your system tray.

    I find it really annoying to have the program reporting that there is a problem just because I disabled a "feature".

  64. Agrado
    Thumb Down

    DON'T TAKE THE ARTICLE'S ADVICE

    The Accept-Encoding header is optional. If you block requests without it you are violating the HTTP standards, and blocking some effectively-random sub-set of site visitors (only some of which will be AVG's link scanner).

  65. Anonymous Coward
    Pirate

    The justification for the pre-scanning

    To try and explain Roger Thompson's argument about zero-day exploits. As I understand it, AVG have noticed that a number of zero-day exploits will only download the first time you visit a site. This is because the exploit is pretty big, and even malware writers can have bandwidth concerns. So my pre-scanning before you download, and then scanning on the download you get 2 things. The pre-scan downloads the webpage into a sandbox. If an exploit exists that your scanner can't detect, it is ring-fenced and doesn't harm anyone. Then when you go to the page properly, the page doesn't attempt to download the exploit again.

    Unfortunately, just like anything else, it won't be long before the malware writers change their policy on only downloading an exploit on first access to the site. Then loads of bandwidth is being wasted for no gain.

  66. dave lawless
    Dead Vulture

    Windows - an Epic FAIL

    Anti virus & anti malware are the worst indictment for an OS I can imagine.

    If I wrote an piece of commercial software that needed a 3rd party anti-virus suite to keep it running I would be mortified. For shame.

  67. Anonymous Coward
    Stop

    Linkscanner INCREASES zero day exploit risks...

    ANY download from a site will put you at risk of a zero-day exploit whether that's by clicking in the browser, or by linkscanner downloading something in the background. If you download it to your machine, it's a risk. By scanning every link returned by search results (sometimes this could be many pages of results), you increase your zero day exploit risk significantly. Whereas as a user you may only click on one link in a search result, linkscanner may have looked at 30 or 40 (or many more) websites - all of which were download in the background, and all of which could exploit holes in your client/linkscanner/AVG/OS etc.

    If on the other hand, you simply 'pre-scan' the link ONCE IT IS CLICKED, you still have the effect of helping to prevent zero day exploits, by displaying a warning at that point and blocking the malware site.

    There is no benefit to link scanner pre-scanning links that the user has no interest in - only an INCREASED risk of zero-day malware. (Plus all the other problems it causes bandwidth wise, processor load, analytics etc). It's nothing more than a badly thought out marketing gimmick that increases the risk to the client rather than decreasing it.

  68. steward
    Pirate

    AVG *is* a black hat

    " But the company may worry that divulging too much information would feed the black hats."

    AVG generates traffic on the web and accesses computers, not for the purpose of information distribution for which they were intended, but to test computers -even when the user looking things up on Google has not tried to access the page-.

    AVG is no better than a dDOS scam, at least as far as information providers and bandwidth providers are concerned.

  69. Anonymous Coward
    Stop

    @The justification for the pre-scanning

    "To try and explain Roger Thompson's argument about zero-day exploits..."

    This is still not a justification for downloading EVERY SINGLE LINK in search results. You can still achieve exactly the same effect by only 'pre-scanning' a site once the user has clicked the link.

  70. Anonymous Coward
    Alert

    The most remarkable aspect of the posts here

    is how well it highlights how functionally illiterate most IT people are.

    "There" is not the same as "their".

    "There" is not the same as "the're", which is a contraction for "there are".

    "Gots" is not a word.

    .

    .

    .

    .

  71. Anonymous Coward
    Go

    Link scanner is great

    First.I can let proxy/whatever to change headers and all filtering is wasted(and let it download some images,not big deal...).

    Second.Thank you really much for such flood of good reasons for upgrade.

    (I am gonna put this in some of my sigs.So nice from you...)

    Beside I am admin of good number of computers on various places and it is not possbile to lock them down completly.(Linux is out of question!)

    And then I have seen only few high disk space hosts with limited bandwith.

  72. Anonymous Coward
    Happy

    @The most remarkable aspect of the posts here

    "There" is not the same as "the're", which is a contraction for "there are".

    WRONG, LOL

    I think the contraction you were looking for was "ther're"

  73. Anonymous Coward
    Happy

    Hmm...

    Actually I don't think there is a grammatically acceptable contraction for "there are".

    (I love it when a discussion inevitably degrades into a "grammar/spelling police" investigation rather than the topic in question)

  74. Anonymous Coward
    Flame

    @"Functionally illiterate AC"

    > "There" is not the same as "the're", which is a contraction for "there are".

    Complete bollocks... the contraction for "there are" is "there're" - the apostrophe indicating the missing letter.

    There is no such word (neither as a contraction nor otherwise) as "the're"!

    (Unless, of course, it refers to a new breed of space beings in an episode of Stargate...?)

    Numpty...

  75. Keith Williams
    Alert

    @rachel

    I find it amazing that you fail to consider that by informing you that a portion of your installed anti-malware is not functioning no matter. suppose a clever cracker managed to smuggle malware onto your computer which then proceded to disable your AV or parts thereof, would you not like to know?

    If you use the advanced installation you can chose not to intall many parts of AVG, acheiving your goal of not scanning links or email or whatever, while maintaining your AVG icon in its informative mode.

    I would also like to remind all the whinging webmasters of the good old dialup days when we would "accelerate" our web browsing experience by pre-loading every page linked to ensure rapid availability when we clicked on a link - so everything linked on every viewed page woudl be sucked down your limited pipes.

    I am a Free AVG user and have been very happy with it over the years.

    Batten down the hatches because obviously the web is going down the tubes

  76. Sarah Bee (Written by Reg staff)

    Re: Hmm...

    (I love it when a discussion inevitably degrades into a "grammar/spelling police" investigation rather than the topic in question)

    It's a beautiful thing. I feel quite emotional. *snif*

  77. shaun

    second one worder of the day

    comodo

  78. Mark

    Re: The justification for the pre-scanning

    Aye, so AFTER the user clicks on a link and BEFORE it is given over to the web browser, you SCAN it.

    See, it's still being scanned BEFORE the webbrowser gets it.

    Pre-view-scanning.

    Unless pre-scanning scans it without loading it onto the computer at all....

  79. Daniel Brandt

    I don't care about Accept-Encoding

    I'm not acting on the Accept-Encoding information. I like the htaccess file shown at www.avg-watch.org and I'm not going to change my recommendation based on this new information. I guess you could say that I don't trust AVG to be consistent about this, even if it is reliable information at this point in time.

    But my htaccess file does have an extra condition, in that it allows the HEAD requests from AVG to pass through without redirection. These requests are only 11 percent of my approximately 7,000 AVG requests per day. I discovered yesterday that the effect of AVG's HEAD request is such that if it is redirected, then AVG detects this and does not immediately follow with a GET request from that same IP address. This means that AVG's HEAD request is being used to detect redirection. I've lost any capacity to trust AVG, and I'm not willing to give them information about what my sites are doing or not doing with regard to LinkScanner. That's why I let the HEAD requests through unmolested. They're only a few bytes anyway. The GET request that immediately follows that HEAD request from the same user does get redirected.

    Yes, I'm probably redirecting some non-LinkScanner users, but they must have a MSIE 6.0 user-agent that is truncated after the SV1, and they have to be coming into my site without a referrer. Both of these conditions are relatively rare. The first one is rare because there's usually lots of junk added by most browsers to the simple user-agent used by LinkScanner, and the second one is rare because my sites that redirect LinkScanner are not the sort of sites that users often bookmark for further reference. (If they come into my site from their own bookmark, then they'd come in without a referrer.) Put the two conditions together, and I don't think I'm redirecting very much legitimate traffic.

  80. Anonymous Coward
    Happy

    Just to throw another wrench in the works

    My office browses through a proxy. IF I ran AVG8 and Link Scanner it would also be through the proxy. My proxy sends a false user agent and strips referrers so all your header based solutions fail. This is basic security. Having Link Scanner cache the page would be the ONLY workable solution.

    And as to all the webmaster tards concerned about your precious log files - I delight in filling them up with crap. If you think you are getting meaningful stats you are kidding yourselves. If you ever saw a 404 for a URL like "yoursite.com/fuck/you/and/your/goddamned/cookies" it may have been me. If you saw it a thousand times its because I put it in a WGET loop. Lets see your stat_anal on that.

  81. auser

    This is a slightly bugged feature...

    The original idea behind the scanner was to put an icon on the search result page that shows if the site is dangerous or not, so the user doesn't have to click on every page to see if it's good or not. Imho this feature should be selectable independently from the normal link scanner, like precaching is only an option and not mandatory for most browsers.

    This feature only works in ie and firefox. There is no opera support, mainly because opera has a similar feature, but they use a central repository of known good and bad sites, so the browser can check a site without connecting to it. Apparenly google has the same feature too, with a clickthrough screen. Recently microsoft has added this to their newest browsers, but only as an option, because it can be used to record user requests.

    Imho avg should just go and do it the right way and check the pages when they get to the computer after the user has started downloading them. It might even pick out the bad data from the good one, so a user could visit an infected page without seeing anything dangerous.

  82. James
    Paris Hilton

    Upgrade time

    Looks like a good excuse to block all the IE6 users

    Paris - because she'd have upgraded by now

  83. Steve Renouf
    Thumb Down

    Literacy @AC

    '"Gots" is not a word'

    Nor is gotten but it doesn't stop 'mer'cans from using it...

    "There" is not the same as "they're" (they are) either.

    "There's (there is) is not the same as "theirs" (belongs to them) either.

    And any number of other fundamental grammatical errors that one increasingly sees with each generation of school-leavers.

  84. James O'Shea
    Thumb Down

    @upgrade time

    What, may I ask, do you think prevents AVG from changing their headers so that Linkscanner pretends to be IE7? Or IE8? Or some version of Firefox? Or, indeed, any damn thing they want it to look like? How about adding a few lines of code which randomly change headers so that it looks like a different browser each time it scans a site?

    It _cannot_ be blocked from your side. Like it or lump it.

    It would not be necessary if your side practiced basic security. The more webmaster whinge about it, the more I think that it is not merely necessary, but sadly overdue.

  85. Anonymous Coward
    Happy

    @ Steve Renouf

    The OED would beg to differ:

    Gotten : Obtained, acquired, won (chiefly with accompanying adverb). Now rare, exc. in ill-gotten

  86. Phil Endecott Silver badge

    Re: DON'T TAKE THE ARTICLE'S ADVICE

    > The Accept-Encoding header is optional.

    True. And it would be a very bad idea to simply block requests that don't have that header. But we believe that Internet Explorer 6 always sends it. So rejecting requests that have the IE6 user-agent and don't have the accept-encoding header will only affect users who are sending a fake user-agent string. So do both tests and it's not a problem.

    > violating the HTTP standards

    No. I'm returning a valid error code, 403, saying that I decline to serve the request. I am using the correct method specified by the standard for telling the browser that I do not want to talk to it. The HTTP spec does not say that I'm not allowed to discriminate between requests.

    (BTW I'm pleased to see that awg-watch is now offering a download of the IP addresses of AVG users. Now the fun really starts!)

  87. Anonymous Coward
    Anonymous Coward

    Re: The justification for the pre-scanning

    "... a number of zero-day exploits will only download the first time you visit a site. ... So my pre-scanning before you download, and then scanning on the download you get 2 things. The pre-scan downloads the webpage into a sandbox. If an exploit exists that your scanner can't detect, it is ring-fenced and doesn't harm anyone. Then when you go to the page properly, the page doesn't attempt to download the exploit again."

    So how exactly does the malware know its the 2nd or 3rd time you've downloaded the page? It can't be by a Cookie because LinkScanner would have to inject the cookie into the browser. It can't be by IP address because many users will either go through a firewall and be NAT'ed or go through a proxy. And it cant be by re-using the TCP socket because otherwise LinkScanner would have to somehow pass the socket to the browser. As a result, when you click on the link the malware will still think its your first visit and the sandbox will have served no purpose.

  88. Anonymous Coward
    Flame

    @Mark Grady

    ...and the rambling concluded with: "FYI - I am not just some numpty spouting off, before the flaming starts. I'm Head of Online for a marcomms agency and I've been in the industry since '96."

    Mark, I have one simple request: next time you spout your dimwitted remarks, can you please be more brief? Doing so would allow for some doubt regarding your level of ignorance - believe me, that's a good thing!

    Any bot / automated traffic other than from the top 3 search engines costs website operators money. It's bad practice and has to be nipped in the bud.

    @zcat: potentially a nice, creative and simple solution. Kudos!

  89. Miguel Santoro
    Flame

    @storm in a teacup

    What's troubling is the name calling and what I can only call a sense of entitlement by the flock of rabid webmasters who've seen spikes in their traffic bills because of this product.

    If you're a webmaster, and your profit is based on sales and/or advertising minus your operating costs (which include traffic) and because of this software your profit has gone down, then what's happened to you is unfortunate, but in a couple of words: "SO WHAT?" If you can't pay your bills you go out of business. Tough.

    It's not that you're unhappy about the software, but rather that people are visiting without generating revenue. By this token, you should also be unhappy about people who surf your site while using adblockers, or those users who hit your site without making any purchases. Personally, I wish for a pox on all of you who are making such a big deal about increased costs based on this product. You aren't entitled to money from every user who generates traffic on your website, and you certainly can't be naive enough to think that all these scripts flying around here will actually stop this (or any other) software from scanning your websites.

    As to the software maker, I commend them on trying to keep their users safe. Perhaps they'll sort out their software so it generates less traffic, and blends better with the users browser choice. Even if they don't, their goal is to sell a product that protects users, not reduce traffic to sites indexed by Google. If you webmasters are serious about keeping your traffic costs down, you can simply stop the Googlebot from indexing your site...

  90. Mark

    @Miguel Santoro

    Bollocks.

    USERS of AVG's product are complaining. Some ISP's have 3GB/month download caps. Enough to browse the web and read email. Unless you're spammed to buggerey or that piece of shit link checker goes downloading 10x as much.

    For fucks sake, the big ISP's all complain about how using an unsecure wireless point to download is "stealing internet access". How? Because it's using up their bandwidth on things they did not ask for.

    Rather like link checker.

  91. Dave
    Stop

    AVG declares war on webmasters

    AVG shocked and surprised webmasters are cranky. While smirking, AVG spokesperson says they should bend over and "take it like a man".

    AVG is committing seppuku. They are making enemies of people, some of whom where recommending AVG a few month's earlier. Attacking (a subset of) your products evangelists is never a good idea.

    Besides, Linkscanner is a dumb concept anyway as implemented. Every firewall product worth anything implements that kind of scan using a transparent proxy.

    Anyway, when AVG finally dies, the party is at my place.

  92. Anonymous Coward
    Anonymous Coward

    Can't just block IE6

    No you can't just block all IE6 and tell them to upgrade because IE7 requires XP or later. Plenty of people are still using Win 2000 or earlier and/or using machines that they do not have admin rights to be able to upgrade the browser.

  93. Steven Knox
    Happy

    @Jolyon Ralph

    ">a number of perfectly good browsers which are quite capable of rendering a

    >site have to imitate IE's headers purely because there are still idiots out

    >there that insist on writing for specific browsers only and will lock out

    >anything else.

    A perfectly good browser would let you specify this on a per-site basis as needed."

    The perfectly good browser DOES: http://www.opera.com/support/usingopera/operaini/index.dml#ua

    Opera FTW (Again!)

  94. Phil Endecott Silver badge

    @Miguel Santoro

    > Personally, I wish for a pox on all of you who are making such a big deal

    > about increased costs based on this product.

    Miguel, that's pretty offensive. Just think of, oh, oxfam.org. Do you want them spending money on bandwidth bills or on poor people in the developing world? Or my friend who runs a fan site for an obscure author and pays the bills from her own pocket? Or even the bbc, who pay it from the license fee that we pay: wouldn't you rather they spent that on programme-making?

    You may see every website as a business where profit equals advertising income minus bandwidth costs, but it's really not like that: many sites exist because generous people pay for them with their own money in order to benefit the community.

  95. John Stag

    AVG is jumping the shark

    What with this, the over-zealous zapping of innocent files (eg. it zaps VNC) and the new barrage of "notification" windows, it seems AVG has forgotten why people don't like Norton.

  96. Mark Grady
    Flame

    @Anonymous Coward

    ANY bot causes traffic, costs bandwidth and creates a visible hit in a site's logs. OK if you use a good stats package you can filter their noise but most people running cheap sites, and I exclude the readers of this august organ explicitly here, almost definitely don't, not even Google Analytics. Do you think hosting providers give discounts on bandwidth charges because Google, Yahoo or MSN's bots have crawled a site? If you do you're more dimwitted than you think I am.

    I'm with Miguel on this one - survival of the fittest. If your margins are that narrow that extra bandwidth is going to nail you to the wall WTF would you do if people really were interested in your site? Ask them not to visit?

  97. Herby Silver badge

    Why this (pre scanning) is bad.

    Compare it to the little kid (it always seems to be one) of your boss who presses ALL the floor buttons on the elevator when YOU want to go to a high numbered floor. You get to stop at all the intermediate floors and see the doors open and close. A big waste of time, money and effort!

    If you have ever been in this predicament, you will readily acknowledge that you have an overwhelming desire to decapitate the offender. The desire is usually related to the number of floor buttons you needlessly endure!

    p.s. It really doesn't do any good anyway!

  98. Anonymous Coward
    Stop

    not big...not clever - and stop reporting!

    their 'lets scan every search result' idea isnt big or clever...as most people

    say, your system is basically connecting to all search results...which could be some dodgy territory which you. as a human, would never click on

    yourself after reading the basic synopsis presented by eg google.

    however, by fueling how their search system can be mitigated you are only egging them on to fix their product so it does look like a real visitor...after all,

    every single malware writer in the world wants to avoid being found out by AVG link scanner - AVG is on huge numbers of machines....and AVG8 will be on even more in the next few months - thats a big target to lose out on!

    so, the more you report, the more AVG will fix their software so NO work arounds can be done. the real fix? we COULD rely on eg google to feed a different REFERER tag perhaps to any results such that you can see if your site ranked 1 or 34 and therefore weight the chance that the visitor was because you were searched for..or you could weight the visits if they came from any searches anyway. you could also add extra code to the page that logged if a visiting 'reader' did a mouseover event or somesuch....scrapers wouldnt (right now!).

  99. Miguel Santoro
    Heart

    @Phil Endecott

    Phil, would you rather that <pick a straw man> website infected with one of these millions of exploits went and infected the PCs of innocent users? Think of all the users that will be harmed. Think of the downstream effects of the exploited machines harvesting user banking details and the loss of time and money to these people.

    > You may see every website as a business where profit equals advertising income minus bandwidth costs, but it's really not like that...

    Not all websites make a positive profit, mine certainly doesn't (and I don't use ads, or sell a product, and I get 800GB of traffic a month for $4.99). But in putting information on the web I am aware that I cannot control who can and cannot visit what I have placed there. (I can certainly try, but I have a suspicion that this would bite me in the end.)

    If what AVG is doing works, and users are saved then from AVGs perspective and from their _paying_ users perspective, this would have been worth it. And I have to be honest here, all these tech-savvy people testing their scanner for them for free is an absolute goldmine, with or without the publicity.

  100. Anonymous Coward
    Anonymous Coward

    Malware

    "Do you think hosting providers give discounts on bandwidth charges because Google, Yahoo or MSN's bots have crawled a site?"

    No. Google, Yahoo and MSN obey instructions in the standard robots protocol and if you tell them not to access your site then they don't. And I know of webmasters who tell Yahoo and MSN to get lost because their robots use much more bandwidth than any traffic they send.

    But the "LinkScanner Tax" is apparently compulsory. AVG expects other people to pay for their useless product, which is so easily fooled it is a danger to anyone who uses it.

    LinkScanner blatantly lies about its identity in order to use my resources so that AVG can make money from increasing my costs and causing me problems.

    Malware.

  101. John F***ing Stepp

    Okay, Da old guy checks in.

    Kind of crap programming that I would find hard to do while drunk.

    This is about the stupidest idea that I have ever heard.

    But then; as I have said; I don't run AV.

    Really; the last AV setup I encountered required IE. Hello?

    If you don't run IE and don't download that stupid ocx ff plugin (who does this?) then you probably can't get a virus in any case.

    If it were up to me it would be baseball bats all around; cause some one is for sure not a team player.

  102. Chad H.
    Thumb Down

    This really effects everyone

    All that extra traffic running around the net is sure to slow down everyones browsing eventually. If we're seeing 5% increases on small sites with most of AVGs customers still not upgraded to 8, how much extra traffic is there going to be once most of their customers upgrade.

    For those who said webmasters should write shorter code: Well why shouldnt AVG write code that doesnt require all this crap. Just setup known good list, with every site searched once, and not again for a few hours.

    And to the Grammar Police: First person to write an english correcting post in an argument on the internet looses.

  103. Anonymous Coward
    Anonymous Coward

    Re: @Phil Endecott

    "If what AVG is doing works, and users are saved then from AVGs perspective and from their _paying_ users perspective, this would have been worth it."

    There is no positive side to what AVG is doing. The task they say that LinkScanner accomplishes can be done just as easily after the customer clicks a link, it simply has to scan the page before passing it to the users browser.

    They claim that it protects users from "Zero day exploits" by malware that only infect on the first visit to a web site. Well unless LinkScanner passes cookies from its scan to the browser I doubt this is true. But if it is then they could fetch the first visited page of a site twice AFTER the user clicks a link. This, by the way, would protect the user not only from sites in search results, but also from any site they visited.

    As for malware writers, I'm fairly sure they can count past 1 and add a bit of code that sends the zero day exploit on the 2nd or 3rd page visited instead of the 1st.

  104. Ivan
    Boffin

    It's only pretending to be IE6 and we can prove it

    "The Accept-Encoding header is optional. If you block requests without it you are violating the HTTP standards, and blocking some effectively-random sub-set of site visitors (only some of which will be AVG's link scanner)."

    As already stated: if the user-agent string for an "addon" such as this is exactly the same as IE6 (even though the "addon" is plugged into Firefox), then a web server should assume that the user-agent is IE6. Because we also know that IE6 does send "accept-encoding", then we can also conclude that this "addon" is only pretending to be IE6 and we can deal with it.

    BTW: My logfiles also show that this "addon" neglects to send the "accept" field too, which the real IE6 also sends. So that's two reasons why it should be dealt with.

    Of course, this won't necessarily work once the Grisoft people read this, as they're probably going to change things again.

  105. zcat
    Linux

    Summary..

    @steve: google limits the results to 100. Plus you don't want the page too big, regular users won't be happy. 100 results is small enough that nobody's likely to notice it. I'm also thinking AVG may update their software to not scan their own site, in which case searching for 'suspicious' websites might be a better strategy. Anything that's dodgy enough to have government-run 'honeypot' websites in the first 100 results would be ideal.

    @nigel: See my second post, and substitute in the appropriate search query.

    @mark: 3G cap is a pretty standard plan down here in kiwiland, I've spent most of this week removing AVG and replacing it with Avast after I have to explain to people why they're now hitting their 3G monthly cap in the first week. I'll probably be suggesting Clamav+Winpooch once I've had a chance to properly test them, but clamav by itself doesn't provide real time scanning.

    I'm really pissed off because I've been recommending AVG (free or commercial, as appropriate) for the last few years. When they pull this shit it makes me look bad too, because I recommended them.

  106. Anonymous Coward
    Anonymous Coward

    So to summarise

    My understanding of their reasons for pre-scanning is to detect any kind of malware, with the concept that if they find any kind of malware, then there is likely to be more. AVG obviously feels that this would help reduce the risk of users being exposed to zero day exploits.

    However, the issues that so far have been identified with this scheme are:

    - this increases the chances of AVG users being exposed to zero day exploits in AVG due to the increased number of links followed.

    - ISP will log AVG traffic as though it was a real user... this leaves scope for causing legal issues for end users (surely something AVG would be keen to prevent).

    - it eats up the users bandwidth (without necessarily making it obvious to the end user)

    - it eats up the bandwidth of websites - a real issue for small hobby sites with bandwidth caps

    - messes up the log file analysis for the website owners (I don't run any websites and hate onlne advertising with a passion, but I would defend the right of website owners to analyse their logs for whatever purpose).

    - the same safety (albeit without the pretty ticks and crosses) can be achieved by using a local proxy to do the prescan only on links clicked by the user (and without the bandwidth / log analysis concerns).

    However, here is another even more serious one that I've not seen anyone mention:

    - (eventually) could provide a defense for pedo's looking at kiddie porn or would be terrorists looking up bomb making techniques - assuming that they are meticulous about cleaning their browser history and cache. They will be using the excuse: "Honest I didn't download any kiddie porn officer! it must have been my anti-virus tool!"

  107. Martin Nicholls Silver badge

    This is why..

    Some people think there's a soul on this planet that actually uses IE6. Well not just *this* but things that pretend to be IE6 when they really aren't.

    It's also why you shouldn't EVER trust server logs for visitor metrics. Use javascript to tell you about real visitors, server logs are utterly useless.

  108. Anonymous Coward
    Anonymous Coward

    @ Martin Nicholls

    Excellent advice Martin. And how do you count this with javascript disabled?

    Doh!

  109. Martin Nicholls Silver badge

    @Anonymous Coward

    You account that a percentage of your users is going to do that - still more accurate by a long way and hopefully the same degree across all browsers (it isn't, really, but it inflates your IE, and more importantly REAL IE6 users to enough of a degree that it's more useful than log parsing).

  110. Steve K Silver badge
    Thumb Up

    @ Rachel

    Disable Link Scanner AND choose "Ignore Component State".

    Problem solved - Link Scanner is off, and you don't get messages saying AVG is broken.

    Steve

  111. Graham

    how about

    after the initial avg get(s), doing another http that identifies the previous get(s) were from avg.

    that way you can't do any filtering on a website based on it being an avg request but you have half a chance of filtering out avg requests from your logs.

    would increase traffic that little bit more but the second get need only be a minimal http get that a website could identify and send back a blank page to.

  112. Anonymous Coward
    Joke

    Bigger problems for Linkscanner

    There are other ways of differentiating LinkScanner from IE6. If you plonk this in an external JS file and link it from your pages, you'll see what I mean.

    var avg = { load: function(libName) {

    document.write('<script type="text/javascript" src="'+libName+'></script>');

    } }

    avg.load("/js/empty.js");

    - IE6 will run it and subsequently request "empty.js"

    - Linkscanner will run it and subsequently request "'+libName+'"

    Or put another way ....

    - IE6 will run it and request "infected_file.js"

    - Linkscanner will run it and request "safe_file.js"

    You can bet that this is a harder problem for AVG to fix than adding a dummy Accept-Encoding header. Sheesh - I can't believe anyone still thinks Linkscanner is a good idea ... it's not, it's a joke ...

  113. Mark

    Re: Can't just block IE6

    However, you CAN tell them to get the latest Opera or FireFox browser.

    I'm *sure* Microsoft won't care that AVG has caused people to be recommending another browser...

  114. Doug Lynn

    AVG is still good, if you don't like linkscanner turn it off

    Hi, been a user and reseller for years and my system is fine, stop persecuting AVG for being innovative, or they might stop giving you a free AV.

  115. Eddie Johnson
    Paris Hilton

    Y'all are still missing the boat

    @AC recommending avg.load("/js/empty.js")

    Again, you CAN NOT rely on people having JS enabled, cookies enabled or any other option enabled. More and more people are becoming security conscious, or having a security conscious friend drill this stuff into their thick head. I personally run with a *.js block rule at the proxy. I don't need no stinking scripts.

    LinkScanner offers absolutely no technical advantage. Why would they have a better chance of spotting malware on a preemtive scan than on the actual download to the browser? They wouldn't! This is nothing but a marketing ploy to establish a feature where they are "ahead" of the pack. Its straight up FUD, plain and simple.

    EOT.

    Paris. Because only she would be dumb enough to still be finding "solutions" to this.

  116. Mark
    Paris Hilton

    @Doug

    Reseller.

    So you making money from selling AVG8 isn't any sort of conflict-of-interest..?

    PS they are completely free to stop offering a free AV product.

    If I give someone a lolly for free and then punch them in the nuts, must I be forgiven because I may not give away free stuff?

  117. Anonymous Coward
    Boffin

    Has exploit hit AVG?

    Yesterday ( July 2 ) noticed my AVG8 had last updated and scanned on June 28. Tried manually updating with questionable results, could not reset automatic updates, scan took f o r e v e r, went to AVG site, couldn't successfully register to post in Free forum so had to be satisfied with reading. Seems many users there ( beginning around June 26 or so ) were having same difficulties. Finally found info that fix was to download newest installer ( released July 2 ) and choose "Repair" but off-site link was older installer and on-site link was miserably slow ( 2-5kbs ) - gritted teeth and said I'd give 'em one more day - same results today. Removed AVG & installed Avira AntiVir Personal. Too much grief. I think someone targeted AVG8 and they were too slow in coming up with a fix and too limited in making it available to those of us who wanted it. The Register should dig in and find out what happened to those of us who put our trust in AVG but won't again anytime soon. Betcha AVG won't let on - the moderators in the forum didn't, the most revealing thing there were "servers are overloaded" comments, which was definitely true, but no info as to why. I think their boat has been sunk.

  118. Sirikan
    Thumb Up

    A resolution

    You guys might be interested in this. We just got a response from the CEO of AVG Australia.

    http://forums.whirlpool.net.au/forum-replies.cfm?t=1006623&p=47&u=76869#r929

    "It's Peter Cameron here. I am the Managing Director of AVG Australia / NZ and I thought that is was important to communicate with users of Whirlpool tonight. I have spent the last two hours reading your posts and have heard your concerns loud and clear.

    I can categorically assure every member here that AVG has heard you, and that we fully understand the issues that have been raised here and that this message has been loudly communicated to the AVG Technologies development teams in both the USA and Europe. I assure you that you currently have the attention of the Chief Technology Officer who is the person responsible for the design and development of the AVG product range.

    As a matter of urgency, AVG is evaluating the LinkScanner technology so that we can ensure we maintain the level of protection for AVG users, but to minimise the impact on web sites and minimise any bandwidth overheads to both AVG users and web hosts.

    We expect the AVG development team will respond quite soon with an effective resolution to the concerns raised both here on Whirlpool and elsewhere.

    AVG has always worked to provide maximum protection for our users and also for the Internet eco-system (it is one of the reasons behind the AVG Anti-Virus Free Edition product) and we will continue to strive towards these goals.

    Sincerely

    Peter Cameron

    Managing Director

    AVG Australia / New Zealand"

  119. Lloyd Borrett
    Happy

    AVG Responds to and Resolves LinkScanner Issues

    AVG has already responded to resolve this issue. The full response can be seen at http://www.avg.com.au/index.cfm?section=news&feature=104

    An updated version of AVG Anti-Virus Free Edition 8.0 is already available, see http://www.avgfree.com.au. The Search-Shield component of LinkScanner has been modified to only notify users of malicious sites. The equivalent modification to the the AVG 8.0 commercial products will be rolled out on 9th July 2008.

    Once the updated version has been rolled out to all AVG 8.0 users the issue will be resolved.

    As of this date, Search-Shield will no longer scan each search result online for new exploits, which was causing the spikes that web masters addressed with us.

    However, it is important to note that AVG still offers full protection against potential exploits through the LinkScanner Active Surf-Shield component of our product, which checks every page for malicious content as it is visited but before it is opened.

    We’d like to thank the web community for bringing these challenges to our attention, as building community trust and protecting all of our users is critical to us.

    Best Regards, Lloyd Borrett

    Marketing Manager, AVG (AU/NZ)

  120. Anonymous Coward
    Anonymous Coward

    Adwords Links

    Does anyone know if it visits the sponsored pay per click links as well the natural/organic results please?

    That could prove to be quite expensive for the advertisers if every search where their ad gets an impression is getting a clickthrough.

This topic is closed for new posts.

Biting the hand that feeds IT © 1998–2019