back to article AVG scanner blasts internet with fake traffic

Early last month, webmasters here at The Reg noticed an unexpected spike in our site traffic. Suddenly, we had far more readers than ever before, and they were reading at a record clip. Visits actually doubled on certain landing pages, and more than a few ho-hum stories attracted an audience worthy of a Pulitzer Prize winner. Or …

COMMENTS

This topic is closed for new posts.

Page:

  1. Richard Silver badge

    Thanks for a response Pat

    But I *won't* be upgrading to AVG 8 entirely because of this feature, and will no longer suggest it as an option to my friends and family.

    There are a few reasons for this.

    1) Every computer and every NAT router has a limit to the number of concurrent connections it can support. This total varies according to the firmware and model of router.

    I use a bittorrent client a fair bit to download various pieces of open-source software. Due to the way torrents work, these clients tend to utilise a large number of concurrent connections.

    I have to limit this number to get an acceptable browsing experience on the various machines connected to my router - as I want to download as quickly as possible, I limit this to the maximum that gives me a good browsing experience.

    This function in AVG 8 grabs around 10-20 connections every time I visit Google or the other supported search engine pages.

    2) Many broadband customers have download limits, before either being charged quite a lot extra, being cut off, or having their download speeds greatly limited.

    Some of these limits are quite low.

    3) My connection is fundamentally shared with everybody else at my local exchange. If everybody suddenly starts downloading all the hits from any page of Google, my internet experience will suffer.

    4) If a webmaster sees a massive spike in traffic, that's going to cost them a fair bit extra in bandwidth charges. Given that any extra traffic produced by a 'bot' such as the AVG LinkScanner cannot possibly gain them any advertising revenue, all such extra bandwidth expenditure is wasted.

    It's doubly wasted because no human ever sees the result of that bandwidth, so even those sites which don't rely on advertising and are simply providing a totally free service will suffer.

    Many smaller sites may either hit their daily limits early and thus be unreachable for much of the day, or suffer such extra costs that they are forced to close.

    5) When travelling, I connect in many places that have terms and conditions including "You may not use download accelerator products".

    The way such products operate is to visit every link on a page and start to download them while you're reading.

    My bandwidth provider is unlikely to see any difference between such products and the AVG LinkScanner, and therefore may hold me in breach of the T&C of the connection they provide and block me.

    This feature will cost me, personally some hard-earned cash, and therefore I do not want it.

  2. Anonymous Coward
    Pirate

    heres an idea

    for all you that don't like it scanning other sites on search engines in case you click on it and webmasters

    why not install it and Keep searching AVG so it Generates load more traffic on AVG's site and see how they like it ???

  3. Anonymous Coward
    Anonymous Coward

    AVG8 free custom instal???

    I don't understand why people say it makes their browsing slow as it's only supposed to kick in for search engine sites, unless in reality it's working in the background on all sites without displaying any results. My real concern would be if it did start to do this for links on all pages visited.

    Anyway, as has been said, just turn it off. The AVG icon on the task bar gets an exclamation mark over it, probably to try to make you think you're not protected but it's easily ignored. For a custom install, the only components I can turn off are the email scanner and alternative languages so either John A Thomson has a different free version or is using the paid for one, he doesn't make it clear.

    What bugs me most is the drop down notification window, if that can be turned off for good then I'd be 100% happy with AVG8 free.

  4. FoTD
    Flame

    I suggest throwing this back in Girsoft's face!

    Well, until this problem gets bad enough that we can all ban together and start a class action law suite against Girsoft for effectively DDoSing our HTTP servers, I suggest the following. Instead of allowing them to waste our own bandwidth with their ineffective and misguided attempt at blocking malware laden sites, I recommend all of us web server admins simply redirect this traffic right back at Girsoft! Those of us running Apache can simply add this directive, either server wide or per directory:

    RewriteCond %{HTTP_USER_AGENT} ;1813\)$

    RewriteRule ^.*$ http://www.grisoft.com/ [R,L]

    This of course will redirect all hits from this rouge user agent to Girsoft's own servers. Screw em! Let THEM deal with the traffic burden! And perhaps if enough of us do this Girsoft will get the point and scrap this stupid linkscanner BS. This should work until they change the user agent string, in which case it's either back to the drawing board or time to sue their asses! I for one will be more than glad to sign up for any class action suite someone puts together. Their "product" generates malicious attcivity on the Net and should NOT be tolerated!

  5. Nebulo
    Paris Hilton

    Not only that ...

    but since I allowed 7.5 to upgrade to 8, I've found that it now takes me ages to get anywhere via my start menu (my preferred way of starting stuff). Every submenu takes tens of seconds to open, although I've got the delay set to 0 - I assume AVG is scanning every link in every submenu, every time. Still trying to turn it off, but haven't found how to yet.

    Or maybe there's a new sort of polite virus which puts itself in your start menu and waits until you specifically ask it to run? AVG, I used to love you, but I'm rapidly falling out of love right now. You're wasting my time every day.

    Paris, because I'm sure she could find something for us to do while I'm waiting.

  6. Anonymous Coward
    Anonymous Coward

    Newer installer - No need for command-line switches

    I'd been using the command-line switch for a couple weeks on new installs, as removing SafeSearch cured all the problems I was seeing with AVG 8 Free (slow computers, crashing browsers). About a week and a half ago, a newer installer (build 1310) for AVG 8 Free showed up, and the SafeSearch module was now selectable in a custom install.

    So as long as you get an installer of build 1310 or later, there's no need for the ugly command-line switches. Just do a custom install, deselect SafeSearch, and you're good to go.

  7. Simon Neill

    Sod the webmasters....

    ....what about poor little me and my 300MB download limit on VM?

    hey, does it do this for the IPLAYER too? I can't imagine trying to stream 10 iplayer videos, but when I try there are explosions.

  8. Anonymous Coward
    Alert

    It's Not About Analytics

    The story only picked on one aspect of AVG 8's problems and generating fake web analytics is just the tip of the iceberg.

    Why does Roger Thompson think his Link Scanner can protect users against web-based threats while the product plainly announces it's presence and Pat Bitton repeats the same party line?

    The important things being overlooked here are:

    a) AVG 8 is easily detectable. The user agent that AVG uses is detectable by the same malicious sites AVG is trying to block, so those sites can easily spoof AVG into thinking they're safe when they're not.

    b) AVG 8 is attacking popular sites. As many thousands of AVG users upgrade to AVG 8 the software it is literally launching attacks against popular sites that rank well in search engines.

    c) AVG 8 customers can now be tracked. Even after AVG 8 changes the user agent to use something less detectable, many web sites now have exhaustive lists of the IP's of the AVG 8 customers that have installed the product to date. Many IP addresses for many broadband customers are somewhat static and may not change for years so if they continue to use AVG they may become a target at some future date.

    d) AVG 8 doesn't need to pre-scan if it can defend against the threat. If AVG 8's Link Scanner can detect the problem then AVG 8 should be able to stop you from getting infected in the first place. If the Link Scanner can't detect the problem in the web page then you'll assume AVG 8 will still stop the infection anyway. Therefore, if the Link Scanner can't detect something unknown and AVG 8 can't stop the unknown infection then Link Scanning in advance is completely useless and a complete waste of everyone's resources since anything detectable is already being defended against without the Link Scanner.

    Bottom Line, it doesn't work and it's making both users and webmasters angry.

    AVG has completely blown a new release and the reputation management skills shown in the face of this crisis with comments about "breaking eggs" are plainly laughable.

    The solution is simple:

    Disable this BUG until you get a new solution, release an update of AVG 8 that has this disabled, and release it ASAP.

  9. teacake

    @Phil the Geek

    "Us Opera users can retain our smug self-satisfied demeanour content in the knowledge that not only are we running the Saab of browsers "

    You mean poor-handling, over-weight, trading on a reputation for reliability and individuality that is long in the past... need I go on?

  10. tony trolle
    Alert

    not too slow

    think I had it running for 3weeks now, it slows this 1.8Mhz system down less than one second on google but only had one red warning in all that time; russian site :-p

    never thought about the site analytics problem

  11. Shabble

    Security for idiots

    With a firewall on my router, with Windows Firewall, Windows Defender, normal AV, Spybot SD resident, UAC and a degree of common sense this link scanning is entirely unecessary.

    Don't allow dodgy websites to install ActivX, don't run keygens for hacked software and scan your P2P downloaded Rar and Zip files before extracting them. Anyway, a decent AV will catch a virus when it tries to install from a web page. AVG 7.5 used to do this for me when I had it installed.

    This link scanning thing is a gimmick - its the equivalent of buying an SUV to increase your safety. It doesn't really, and just blocks up the roads. Note: the only virus I've found on my computer in the last year was an old keygen for a bit of software I wanted to test out, but forgot about and never used. This was missed by AVG for many months and was only picked up when I switched to Avast!

  12. Andy
    Black Helicopters

    What about accidental paedo or terror browsing

    Presumably if you enter "children" or "al qaeda" into google then AVG will happily visit any dodgy web sites that appear in the results for you, thus ensuring that you move onto the police watch list for axis of evil members.

  13. Vostran
    Stop

    Bunch of criminals!

    "I don't want to sound flip about this, but if you want to make omelettes, you have to break some eggs."

    Sounds to me like a criminal enterprise.

  14. Ross

    Right idea, wrong solution

    Protecting users from malware located on webpages is a good idea - it's probably the biggest attack vector these days what with every one and his dog using a router with built in firewall. The AVG method is using a sledgehammer to crack a nut though.

    A better alternative would be to run a local proxy that scans your inbound traffic for malware. That way only the pages you actually visit are scanned, rather than anything Google deems fit to throw up as a search result. It would save web server bandwidth, your bandwidth, and your CPU cycles. It also still ensures that the detected malware doesn't hit your browser.

  15. Anonymous Coward
    Alert

    whining fools

    your all a bunch of crazed whining fools. this feature is actually a good bit of host protection and has been proved to work - far better than blacklists etc.

    you're all moaning for the same reason - you're all using outdated methods of checking that someone has actually visited your site. relying on a dumb apache log file or the number of times image 'lets take these advertisers to the bank.jpg' got downloaded is foolhardy and wrong. countless times its been abused purposefully but like little blind mice you've carried on regardless. use some other method

    of validating a real human has visited your site. something basic like an

    'onmouseover' event on a news story or somesuch is 100% better than 'oh look, Mozilla 4.0 blah blah from 127.0.10.40 has loaded index.html'

  16. Daniel Feenberg

    Not a deluge yet

    On our rather esoteric web site (www.nber.org) that browser string accounts for 5% of the last week's 5.04 million hits. So it isn't killing us yet, but then what fraction of their users have upgraded so far? It certainly seems tempting to serve up an error page to the user, asking that they disable the "feature", should they eventually select one of our pages.

    Why should they be scanning pages before the user clicks? Perhaps they don't have the proper hooks into the browser? Or don't know how to use them?

    Daniel Feenberg

  17. Rodney Cole

    tenuously linked to AVG thread

    I upgraded our motley collection of PC's to AVG8 recently and haven't noticed much change in speeds, apart from when the actual scheduled scan is taking place. The only Googled site that I have been warned by AVG about had "this site may damage your computer" written on it already.

    I don't really want the sole option of Yahoo! search in the AVG bar because it's soooo inferior to Google, but something happened the other day which I would be grateful for your views on.

    I had just booked some Premier Inn rooms online as is my regular wont and then visited a favourite US blog and followed a link to the LA Times archive site. At the top of that page was, guess what, Lenny f++king Henry gazing back at me from a Premier Inn banner, complete with the standard UK hotel search.

    As it seems unlikely that Premier would be targetting all LA Times readers with banner ads, the only conclusion is that one of the many tracking cookies that AVG recognises but does nothing about is "profiling" my surfing. Cookies go regularly into the CCleaner bin.

    How valuable this is to the advertiser (advertising to an existing customer) is puzzling, or am I in possession of the wrong end of the stick? Or just uninPhormed and paranoid?

  18. Anonymous Coward
    Thumb Up

    How to disable it after installation

    You can disable it in IE if it is already installed by using the Tools->Manage Add-ons feature. This will not cause the red exclamation mark to be shown in the system tray, and will still be displayed as Active in the AVG User Interface.

  19. Anonymous Coward
    Stop

    avg?

    I couldnt find this in the repositories.

  20. Charles

    Alternative advice to Firefox users.

    If you *really* have a problem with AVG's SafeSearch feature but insist on using Firefox, consider trying out the Firefox 3 release candidate. SafeSearch won't work on this version of Firefox, so even with AVG8 installed, SafeSearch stays disabled. In any event, the handy NoScript addon keeps the bulk of trouble (and bloat) out of my way.

    As for scanning ahead for malware, here's a possible angle: multiple payloads (so as to try as many angles as possible). AVG may be able to detect and block one or more of the payloads, but you could still be owned by the unknown or zero-day payloads. By scanning in advance, any site that has even one detectable payload can be blocked, and by blocking you also reduce the likelihood of being hit with an unknown attack.

  21. This post has been deleted by its author

  22. Timbo

    @ FoTD

    Any way in which a "robots.txt" file can do the same ??

  23. Andy

    Effort better spent on patching...

    The poster who cited LinkScanner saving the day, the article also mentioned..

    "People who’ve visited the website over the last week need only panic if they are running a version of Microsoft Windows that hasn’t been patched or a version before Windows 2000."

    Looks like users are buying all these AV products, that use resources (locally and on the Internet), but not patching their systems?

  24. Anonymous Coward
    Flame

    I hate web advertising, But...

    If the AVG8 link scanner can detect the exploits prior to the user clicking on the links then surely this means it can detect the exploits when the user clicks on the link.

    In which case Link scanner is DDOSing the web just to put a pretty green tick or a nasty red cross next to the search results so the user to feel good or bad about going to the site.

    If AVG can't detect the exploit (when the user clicks the link) and prevent it, then why are they charging people for their product?

    I have no love of web marketeers, traffic profilers etc, but a company that thinks DDOSing any website that is turned up in the search results is the way to protect people is even worse.

    An antivirus company should know better!!!!!!!

  25. Martin Maloney
    Alert

    No one has addressed this

    "...Thompson points out that AVG only scans the first page of results on sites like Google - unless the user clicks on subsequent pages...."

    I do serious research on the Web, thus I use google Advanced Search, and I select "100 results" for "Results per page."I doubt that I am alone in this.

    That kinda kicks "...only scans the first page..." in the butt, doesn't it?

    Several months ago, having tired of AVG 7.5's penchant for throwing a plethora of false positives, I switched to avast! Thanks, el reg, for quenching any temptation that I might have had for sampling AVG 8.0.

  26. Kevin Reader
    Alert

    Surely this is old tech done really badly...

    Apart from the fact that they could validate the PAGE you actually CLICK using their "new technology" before serving it to the browser, I have long noted avira'a live scanner doing the following (which is rather better):

    Even several versions ago the avira live scanner checked files as they were written to disk (probably on close) - this seems to include files in the browser cache. This appears to detect suspicious exploits. I would have thought AVG would have implemented something like this or a a similar scan in the download path of the browser. That's what a sane develop would do.

    I suspect that the scan is a) slow and b) processor intensive and they did not want to delay the response to the user's click AND SO they decided to pre-scan the search hits while you are reading them. BUT that means they scan ALL the results even if you do not visit them.

    There is a theory that many people never even press NEXT on google or other searches so the impact is LARGE but not huge. For anyone who does proper web research - or looks for obscure information - it wil be far worse. I regularly wade through pages of google results without clicking more than one link a page - this 'technology' would increase my bandwidth and visits 10-fold. Utter madness and shody development.

    El-Reg: Can we have an icon for "Mental Developers, Extra Stupid PHB", it would suit these sorts of stories more than the existing ones, and is becoming ever more common. (It used to only be Bill's gang).

  27. Stephen Hurd
    Thumb Down

    This should work grea with Firefox!

    So now, with Firefox preemptively downloading all the links on a page, and Linkscanner doing the exact same thing, now every link you don't follow will be followed twice on your behalf! Yay!

    Seriously people, if I want it, I'll click on it. I really don't understand what AVG thinks could be accomplished here that transparent proxying wouldn't... and definitely not one that makes it worth downloading everything (twice? thrice?). ISPs are rolling out the bandwidth caps and the web folks are cranking up the bandwidth any way they can.

    The thing that's even more fun is that (unless AVG is caching what it downloads and acting as a caching proxy anyways) the web site you get to when you follow the link quite likely won't be what AVG scanned anyways. The ads (a common source of malware) will rotate.

  28. Jonathan Richards

    Serves you all right...

    ... for trying to make money with the Internet. Bring back ARPANET, sez I.

    Mine's the moth-eaten Afghan. Ta.

  29. John A Thomson
    Paris Hilton

    Response to the flamers!

    @ Nexox Enigma

    Good grammar and breeding clearly shows out. You may be shocked to find I have founded, run and participated in many community projects both online and offline. I also can't stand texting because of the lack of proper English... it just feels plain wrong, wrong I say!

    @Phil the Geek

    But Phil this type of technolog is coming to Opera and Mozilla products. The Register wrote of it only days ago.

    @ Kanhef

    Sorry to disappoint you, but AVG isn't paying me nor am I doing their dirty work as you are suggesting. Admittedly, I do resell their products, but that's not a new thing and I was a bigger fan of Avast Pro until AVG 8 appeared on the scene. Our old layered security solution was a far more lucrative solution in terms of revenue, but it has been found that AVG 8 is a good and cheap solution for end customers that don't have years of experience of using the web - those very customers that wish to have the additional protection it offers in a single solution and don't mind paying a small amount of money to achieve it.

    I see mainstream web users being better protected through using this technology. We have a good sized customer base here in the UK and even have a customer in Nigeria (anyone tried 196kps down, 64kps up???) using this technology and none of them are complaining about it turning their system into treacle. I wouldn't be recommending it to my end customers if I thought it was a bad product for them. Sure there are situations where it may not be recommending - namely, when the ISP is one of the many big boys that sucks during peak hours... and we all know who they are!

    My attitude is this...

    It is alright for websites to add all kinds of additional marketing and advertising streams, to add multimedia content, to use all manner of high bandwidth items to increase their buzz and marketability, but most webmasters haven't thought too much about the people on slow connections and those end users that eat through their ISP allocated bandwidth to gain the unnecessary parts of the user experience. I realise that these technologies eat up the same ISP bandwidth, but that is the customer's choice when they install / enable Linkscanner. Now the worm has turned and web vendors are complaining. It is unfortunate that some very small business may well not be able to adapt to this changing landscape, but the bad guys have moved on and so must the security vendors to better protect the masses - there are always some casualties in war!

    There are many US based suppliers offering these types of bandwidth allocations. Have a look through websites like http://www.webhostingstuff.com/.

    @ tony trolle

    Try Googling for things like warez, cracks, etc. You'll see a good many more red crossed results. Many web users still believe obtaining illegal software is a good way to save money and don't worry about the consequencies. Perhaps they will take more note of the warnings to stay away from these types of websites when they see security software reporting bad things.

    In this day of legitimate websites being hacked to serve drive by downloads, and all manner of other malware, having Linkscanner and technologies of its ilk is going to be a good tool in protecting us all from end user being infected.

    Q. Is AVG 8 perfect?

    A. No. I can think of quite a few improvements I'd like to see. I've also seen some compatibility issues with a few other security products that disappear when those other products are removed.

    Q. Could Linkscanner work better in other ways?

    A. Certainly, but it cannot be changed overnight into a security product that is going to please the webmasters voicing their concerns.

    Q. Will we see similar products / features from other security vendors?

    A. Very likely.

    Q. Why are you so passionate about this technology?

    A. Because I've seen it working to protect web users that wouldn't know any better and would have their systems infected with all manner of malware. I even seen it detecting trustworthy websites that have been hacked to serve malware. The website vendors involved (3 different companies in one case) had let this website serve malware for weeks to visitors without having a clue that something was amiss. Thousands of website visitors could have been affected during those few weeks. Linkscanner detected the exploit code without even breaking a sweat.

    Okay, I'm stopping now as my position is quite clear and the flamers must be queuing up to get into the comment box :-P.

    Paris, cause she knows good breeding!!!

  30. Adrian Jones
    Pirate

    Another AVG problem

    A few weeks ago, I discovered that several of my mailboxes in Pegasus had disappeared. Very annoying as one of them was my main box, with hundreds of emails.

    I've just realised that it was AVG which deleted them. It's scanned them as an archive, identified a virus (or so it thinks, I'm somewhat doubtful, since I've always deleted anything with a dodgy looking attachment, and one was a simple word document from about 10 years ago!) and moved the whole file to the Virus Vault.

    Which then filled up past its default maximum size, so it's deleted several of them.

    All without bothering to tell me about it. I only spotted it because I happened to be in front of the screen when a scan finished, saw the pop-up appear and disappear and wondered why it had 40+ warnings. (Cookies, it would appear.) I then discovered that there were several "viruses" found in May and that they'd already been deleted from the vault.

    Not very impressed with AVG after all.

  31. George Forth

    Turning it off isn't especially fun

    I don't like it as it slows down my creaky old machine when I search on Google (the creakiness is why I use AVG and not something resource-hungry like Norton). But if you turn it off, it records this as an error rather than a choice. Most annoying.

  32. Matt Bryant Silver badge
    Pirate

    RE: Kanhef

    I have AVG 8.0 running just fine on a seven year-old desktop for my kids. It has an Athlon 1100MHz CPU and only 768MB of RAM, yet runs WinXP a treat and I haven't noticed any drop off in performance in any way at all since upgrading her PC to AVG 8.0. I have long appreciated AVG, especially after several years of McAfee and Symantec screwing up other systems (including those at work), and I would not hesitate in the slightest to recommend AVG 8.0 to friends and family. Seeing as kids are often the ones that get suckered into visiting dodgey sites, the search preview offered by Linkscanner is a brilliant tool.

    Strangely, everyone is up in arms about personal users using AVG's Linkscanner, but I don't hear a peep about the commercial spiders and bots that regularly try and read EVERY page in our work website. So, it's OK for Google and co to trash our work bandwidth, just not personal users? Puh-lease!

    IMHO, thanks to Grisoft for another excellent product!

  33. Bambi
    Black Helicopters

    No worries on AVG 8 ...

    I think just like Norton, McAfee and other 'hoggy' security systems (which is what they have become), folks will be migrating to other things that are less resource intensive. AVG 8 slows down even brand new Vista computers with 4GB RAM and Core 2 Duos!

    AVG 8 could very easily be a thing of the past very soon.

  34. Hate2Register
    Unhappy

    Off with their heads!

    I agree with Aditya Krishnan that AVG should not be pinging sites with gay abandon, skewing traffic (and making it look like I'm visiting sites that I'm not). George Forth points out that if you go into AVG options and turn off the Linkscanner, AVG puts a permanent error message in the system tray.

    Aditya's suggestion that AVG compare search results with a database of bad sites is a sound one.

    This revelation could sink AVG's reputation if they don't fix it soon...

    [an otherwise loyal (ha) AVG user]

  35. Steve

    alternatives

    If you employ the use of a hosts file (mvps for eg), tighten up your o/s internet security settings (the defaults are poor), forgo the use of IE, use an add-on lightweight firewall, use Opera as your broswer, and install avast as your virus scanner, this issue is a moot point.

    And most of the above is also easily accomplished using a distro like openSUSE as an alternative to windoze.

  36. Chuck
    Alert

    Get a clue people

    I am amazed at the supreme level ignorance among so many of the LinkScanner critics here, many of whom probably consider themselves security experts. What a sorry group. I guess that's the crowd you attract with inflammatory statements intended to stir the irrational fears of people (Joseph Goebbels and Karl Rove would be proud, El Reg).

    Get a clue people. I've been using the paid version of LinkScanner since shortly after the company introduced it. Too many times to mention I've been protected from exploitive web sites listed in Google's search results, or contained on trusted sites. Yes, I'm patched, so maybe 80% of these wouldn't have affected me, yet I still don't want to visit a poisoned web page for obvious reasons. And what about the 95%+ of users out there who don't maintain regular patches, or what about all of us who need protection against (albeit rare) zero days before a patch is available?

    Someday, once the masses get more properly educated than some of the ignoramuses on this thread, they will refuse to click on any hyperlink until it has been properly scanned by LinkScanner or a similar real time scanner (Note most of Linkscanner's competitors including McAfee's are NOT real time so they're essentially useless), just as my 65 year old mum knows not to click on file attachments from unknown senders (and even trusted senders) in her email, or how more enlightened novices have learned not to click on malicious ecard greetings.

    If you're a web site operator, don't your visitors deserve to receive some verification of the real time safety of your site? What are you trying to hide? Your ignorance? Your ostrich beak?

    If people here truly studied LinkScanner, how it operates and the thinking behind its (IMHO) clever low impact approach to stopping web exploits, you might arrive at a different conclusion.

  37. Anonymous Coward
    Stop

    Not only advertising

    A major problem with LinkScanner is not only the one-time traffic it generates by accessing pages, but by the fact that LinkScanner's parser just plain doesnt work properly, and can with certain content cause endless loops with user's computers unwittingly registering tens of thousands of hits against a site, as LinkScanner sits there reloading it endlessly.

    How shocked I was when I saw this, on one user's account statistics... (a 6 day traffic period)

    1 2202660 81.92% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)

    2 453439 16.86% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

    3 7634 0.28% Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1

    Thats 82% of traffic, several million hits, on what is normally a fairly quiet shared website, coming solely from a user agent I later discovered was submitted by LinkScanner. In numeric terms, thats around 70GB of traffic in a week to a single fairly small website.

    @Pat Bitton, all traffic hitting our networks from LinkScanner's user-agent is now being redirected to grisoft.com, I fail to see why we or our customers should be paying for your product's traffic when the damn thing doesnt even work properly.

  38. Anonymous Coward
    Anonymous Coward

    Flawed Logic @ John A Thomson

    How can Link Scanner protect users that would otherwise be infected UNLESS the sick truth is that AVG 8 can't stop those particular infections in the first place?

    Either AVG 8 can or cannot protect you without Link Scanner, which is it?

    If AVG 8 can protect you, then Link Scanner is a complete waste and something used purely for marketing purposes.

    If AVG 8 can't protect you and uses Link Scanning to attempt to stop things it can't protect against, I'd be looking for new AV software ASAP.

  39. Anonymous Coward
    Anonymous Coward

    Get A Clue Chuck

    Please explain, just how did the Link Scanner save you?

    Did it detect something AVG couldn't detect or stop from infecting your machine?

    If AVG could stop it, then the Link Scanner was superfluous, no?

    Total hysterical hype.

  40. Phil Endecott

    No referrer set on these requests?

    A quick check of my logs shows about 5% of page requests (and a smaller proportion of bytes) are from this ";1813" user-agent. Interestingly, none of them had a referrer set. Is that also a characteristic to check for? Does anyone know if the ";1813" is ever present in a real request?

    I've encountered two similar issues in the past. One is MySpace users putting images from my site - and it's always the largest files - on their pages. That was using >80% of my bandwidth for a while. Then there was "fasterfox", a firefox extension that would prefetch all of the links on a page. I got a link on the front page of Digg once and the fasterfox users (the vaast majority of the requests) brought the server down. I have these Apache rules to avoid these happening again:

    RewriteCond %{HTTP_REFERER} profile\.myspace\.com [NC]

    RewriteRule .*\.jpe?g - [F]

    SetEnvIfNoCase X-moz prefetch prefetch

    deny from env=prefetch

    My message to AVG: please check for a robots.txt rule, and tell me what robot name I have to check for. If you want to tell your users that "This site refuses to let us check it for viruses and we suggest that you don't visit it", then that is just FINE with me. Or if you'd like to wait until they've actually clicked on the link and then test the content, please go ahead.

    Just to answer one of the posts above: Chuck says

    > don't your visitors deserve to receive some verification of the

    > real time safety of your site?

    Absolutely, my _visitors_ are welcome to do all the verification they like. It's the 90% of verifications that come from people who won't ever click on my link, but just happened to get a page of search results that included it, that I object to.

  41. Oliver Beresford
    Thumb Down

    "it doesn't mask the user's IP address"

    If the reg article is correct in that "it doesn't mask the user's IP address" then I will stop recommending AVG to people and point them all at Eset NOD32 Antivirus!

    I would be very unhappy if I descovered that my IP was being logged as having visited every site that came up in google when I did a search, and I think most people would be against this 'feature' if they understood that this was what LincScanner would do...

  42. Steve

    performance

    Obviously anything that impacts the performance of my computing experience, particularly when a properly configured system protects just fine, is completely unwarranted. Propoganda can come from many directions, particularly say, lobbists with financial interests in the matter.

    The configuration I described previuosly worked just fine on MANY machines and for MANY years, sans exploits and sans LinkScanner.

    I'm just sayin...

  43. Nigel Brown

    Giving it the NOD

    Call me an old Luddite, but as I still have no desire or need to install SP2 for XP then the delights of AVG 8.0 are now denied me. I am currently halfway through the trial periond for NOD32 and am more than happy.

  44. Robert Day
    Dead Vulture

    Another twist of fate

    I have to say I am so very happy I am not ranked #11 on Google's results for, say "shopping". I mean.. with millions using this Link Scanner thing, and hundreds of thousands of them searching for "Shopping" on google, the top 10 results, are getting hundred of thousand clicks / day more than #11, who suddenly pales immensely compared to the top 10. Popular sites are suddenly so very much more popular than they once were..... Wonder how they will deal with THAT angle....

    Dead vulture, for my soon to be dead site, ranked #11 in search results.

  45. John A Thomson
    Stop

    No AV is perfect

    Okay it has been asked multiple times by those who don't understand the principle and thinking of Linkscanner, so let me try to explain.

    Let's take a website like Spy Sheriff by way of an example, AVG AV will detect it the malware being pushed down by the parasites. However, when you use Linkscanner it warns you not to go near it in the search engine results and if using the paid for version it will stop you going there either directly or by clicking on a search result.

    However, take a zero day virus that can avoid detection by any AV initially. It can take AV vendors anywhere from hours to months to devise detection and protection into their products. Linkscanner is looking for the exploits, techniques and typical methods that the malware writers use to actually push the zero day virus out onto unsuspecting computers. Bottomline, AVG and most other AV products probably won't detect the virus, but Linkscanner will detect if the bad guys are using known exploits to push the malware packages down onto victim computers. Users running only AV may become infected depending on their OS and set-up.

    So please can we stop comparing Linkscanner to the protection offered by typical antivirus products. Apples and oranges my friends.

    @Chuck

    I've suggested a similar scheme already to Pat to help with the website owners that don't want to be scanned. Either a robots.txt file (as you've suggested) or some additional meta information could make Linkscanner, and other such products, ignore the scanning and give it a classification of "Scanning refused by website... Use at your own risk! The link will be scanned if you decide to follow it". Having a nice warning, such as we see IE7 doing when there is a certificate problem, could make it nice and easy for people to click through or not. The weblink would be scanned at this point if the user selects to click through.

    Webmasters may find it more palatable to only be scanned if the web visitor is actually going to visit the website. Cautious website visitors may not click through onto perfectly healthy websites and that is a cost to be considered when implementing the "NO LINKSCAN" tag.

    Unfortunately, social engineering techniqies will ensure the bad guys manage to trick some users into visiting malware websites if this type of scheme is adopted. That is why it is important that the pre-scan is done at some point before the browser actually lands on the webpage.

    Please don't ask me to comment again!!! I had enough explaining to people who don't even have the courteousy to understand how the technology works.

    I'm sure Pat and AVG would rather hear your suggestions on possible ways to fix this than your whinging. How about some constructive criticism to help AVG to help you?

  46. FoTD
    Linux

    @Timbo

    "Any way in which a "robots.txt" file can do the same ??"

    No, probably not, and I highly doubt linkscanner bothers to parse robots.txt any way. I assume you pay somone else for hosting and don't have direct control over the web server? You can still use my suggestion if you are hosted on Apache and your ISP allows you to adjust your site settings through the use of a .htaccess file. Add this to your .htaccess file for the site:

    RewriteEngine on

    RewriteCond %{HTTP_USER_AGENT} ;1813\)$

    RewriteRule ^.*$ http://www.grisoft.com/ [R,L]

    This turns URL rewrite module on (if they allow this and have it installed) and redirects all requests from the rouge user agent to Girsoft. If you are hosted on IIS server you will probably have to do something in the way of an ASP script instead, and it would need to check the user agent and issue a redirect before serving up the pages content.

  47. John A Thomson
    Happy

    NOD32

    NOD32 is an excellent security product. I've trialed it a few times, the last of which turned into such a disaster that it was off the computer within one hour! The latest version had some very bad press when it was first released and most business customers stayed with the previous release. They also occasionally have a big issue:

    http://www.sheffieldforum.co.uk/showthread.php?p=3566976

    However, in general it is one of the better security products out there.

    No security product is perfect... no security vendor is perfect... but some are far superior to some others... you know the ones that seem to be bundled in with new systems :-).

    One thing is for sure... the bad guys only need to get something right once, whilst the security vendors need to do it right every time! An impossible mission if you ask me. Layered protection is the best means to preventing the bad guys from succeeding.

    p.s. Have a look at the Linkscanner videos over on YouTube to see some real world incidents and why those using Linkscanner were better protected. They are short and show off some clever techniques used by the hackers.

  48. Anonymous Coward
    Anonymous Coward

    so long AVG, NOD32 rules

    "...Thompson points out that AVG only scans the first page of results on sites like Google - unless the user clicks on subsequent pages...."

    Thank fuck for that. They could have been scanning all of the results returned! Software makers, developers have to stop feeling "they" have to protect us from ourselves. If I want a software to scan links before I visit them I will use Google. Marketing hype, nothing else.

    Got a massage "updates will not be available soon, update to v8". Installed v8, first scan it found over 200 viruses, trojants etc. yeah right! So lemme see. AVG 7 was nor detecting these, nor any other AV for that matter. How much can you trust an AV which turns up with dozens of fps?

    And dont even mention new duplo interface. Removed it, installed NOD32, 10 days into trial, works like a charm. Thanks to AVG for free use of v 7 series that was an excellent "install and go" AV which I recommened at every opportunity to friends and family. However they jumped on the band wagon, messed up a good product that was working perfectly well.

    @Chuck

    "Too many times to mention I've been protected from exploitive web sites listed in Google's search results"

    If you can be owned bu visting a posion site, then you deserve to be. No LinkScanner will save your ass

  49. anarchic-teapot

    Bugger

    So that's why hits to my site have apparently doubled over the past ten days or so, and here was me thinking that trying to put quality stuff online was at last paying dividends.

    What bothers me - apart from all the bandwidth burning, which apart from being unecological is just plain rude - is that the information being provided to (mostly) individuals paying the odd cent to advertise on my site is now seriously distorted.

    Moreover, there are hosting services who are perfectly capable of taking a site offline once it reaches a set bandwidth limit, and do. Not good for small businesses.

    @ Phil Endecott : my sentiments entirely

    @ AVG Look you do a nice antivirus suite, what on earth possessed you to do this shielding in such a damn silly way?

  50. Nuno trancoso
    Dead Vulture

    Blunder...

    If i ever seen something utterly and totally f..... messed up, this is it.

    What amount of REAL protection will anyone gain from scanning pages they WONT visit? Answer, none. Score one to pointlessness.

    IF the AV is doing its proper job, the page/contents will be checked somewhere between the TCP/IP stack and the browser. So, if protection works, it will work on proper time, no need to check in advance.

    Some ppl are on a limited traffic plan. How happy do you think they will be when your AV gets them a nice ISP bill for overshooting the limit "while just browsing"?

    Website owners put up (mostly) w/ search engines and the like because its in their best interest to get listed. Your product offers them nothing but annoyance and increase bills (bandwidth/etc). So they might not take lightly to having to serve content in a pointless way. You can actually see it right here, site admins are looking for ways to shut off YOUR USERS.

    Seriously AVG ppl, if you really need a new "cool feature" to sell you (IMHO braindead) product, by all means, do so. Just dont add one that is bluntly on its way to turn into a PR disaster...

    the bird.... cause its a braindead "feature" too....

Page:

This topic is closed for new posts.

Other stories you might like