Google in mass 404 land grab
So, Google doesn't like other people hijacking their web pages, but it's more than happy to hijack someone else's. On December 8, the ever sneaky Canadian cable and telcom giant Rogers Communications was caught inserting Rogers-centric messages onto third party web sites. Since Rogers's high-speed internet service is co-branded …
You fail to see that there is a basic difference between the two kinds of behaviour. In on case, you have a cable provider moderating html output (something the end user can do nothing about) and in the other, you have a piece of USER INSTALLED software altering html output. If you don't like Google Toolbar, uninstall it. You don't have the same luxury with your ISP.
The 512 byte limit is interesting because Internet Explorer will also replace any server-supplied error page with its own built-in version if it's "too small". I forget what the limit is but it's similar. So in this respect Google is only following in Microsoft's footsteps.
Doesn't IE when it sees a 404 less than 512B it gives you a user friendly error page. So how is that different from Google?
If you don't like your ISP, you can simply move to another one?
This is also the normal behaviour of Internet Explorer - if the error page is too small, it too displays its own page.
(Though whether or not it's 'right' for either to do this is a matter of debate - on the Microsoft side, the page displayed is intended to help. On the Google side, the page is intended to help, and make money for Google.)
It may be different in detail but it's fundamentally still the same. Yes, people have a choice to remove the toolbar. They have a choice to change ISP; there's a lot of hassle involved in the latter but the choice is there most of the time. The problem isn't whether there's a choice or not, the problem is Google assuming that they have the right to alter what the user is seeing without informing them of what they're doing. Without information there is no way to presume consent. It's as simple as that.
I agree with A Gould - it is completely different. If the user wants software that will replace default error pages with google search pages, that's their choice. The only thing I'd do is make sure the user is aware that they're choosing to do this - something thats very hard to do as users don't read things you stick on the screen...
Optional software providing a service is somehow wrong?
Exactly the behavior of adwares.
For A. Gould: yes, you can change isp.
Uninstalling the "toolbar" is far from obvious for most users. At first they do not know why it's there, then they can not make a relation between the hijacking and the toolbar.
What's the difference between this and MSIE displaying custom error pages under the same conditions? It's been doing this for years, and not just for 404 responses.
This is not about the user end message that is quoted here. Of course the user can turn off their web toolbar, but this is not the issue at hand.
The issue is that when someone is directed to my website, and there is no response from my server, google will take over my web page, present its own content, or a competitors.
Hence they are hijacking my website and people trying to access my site without any right to do so
Different, yes but still damn annoying. I'll be checking the 404 output on the site I look after and will be adding sufficient comment-wrapped # symbols to push it over 512 bytes...
FWIW, I have a custom 404 response but I'm pretty certain the html is <512 bytes. For Google to suggest that my brief 404 page is somehow unworthy and that they could do a better job for me is condescending and arrogant.
If someone goes off-piste on my site, I want to be able to suggest what they should do next, not have Google take them elsewhere (potentially to another website).
It seems that for every reason to love Google, there's an equal an opposite reason to hate them and hence the fundamental balance of the universe is upheld...
I'd actually consider installing the google toolbar for this....
are somewhat retarded and self serving. While I'm not a fan of toolbar software, using an opt-in service that was specifically designed to be non-intrusive to lampoon a company you essentially dislike by comparing it to a company that is in fact guilty of taking advantage of its users is completely ridiculous.
The google toolbar can be disabled, uninstalled at will. It's designed to provide functionality for users that desire such functionality, without inconveniencing or aggravating web users or web masters. Tell me what's wrong with that?
What rogers did was forcefully alter the online experience of their paying users.
I'm with A. Gould on this. The difference is fundamental.
In part it isn't about what the purpose of the change is. If I as an ISP search and replace (for example) Huckabee's name with some amusing homonym that would (probably) get me sued for meddling in the democratic process or some such. If I do the same to promote myself that is supposed to be acceptable somehow?
But Google offering a tool that has amongst it's capabilities, the ability to help me find the page I wanted when I get back a result that says 'nothing there', well personally I won't use it, but it's a tool. Use it if you want it. And if you install software without knowing what it does (it's not like it's likely to be a hidden feature) then you deserve what you get.
My one thought against Google with this is that the page they offer should include the returned error page as well as the helper content, rather than just dropping it.
but the I have opted to use their service and think it's fine, as it helps fund a service I pay nothing for
I'd be ever so upset, on the other hand, if my ISP decided to insert their branding into webpages downloaded over a connection I pay for. I think the difference is if you're paying for it, then it's an outrage to have your data played with, if it's a side-effect of a free service then you pays (or not) your money and you makes your choice
It's not fundamentally the same. Most people aren't trying to get 404 pages, so Google is replacing one piece of undesired data with another piece of undesired data. Rogers alters content the user wants to see, creating undesired data where none existed before.
Not that that makes them any better morally.
On top of the above mentioned reasons, I wish to point out that I can change all sorts of things on my computer, including OS, browser and toolbar-plugin things, but in many places, I am limited to one broadband ISP. In many others, I can only choose between two.
The idea here is that ISPs should not be allowed to modify content of traffic just because it passes through their network. (...or, in Comcast's case, should not be allowed to choke out/eliminate traffic based on type/protocol.)
(I know, the cases of evil stuff such as virus activity and malicious TCP traffic, we want the ISPs to step in and help to limit the damage... But there is still a VERY marked difference between these ideas.)
Just flicking through the prior article, Rogers' provide a "I dont want to receive these messages" option which seems pretty comparable to Google's philosophy. I would go a step further and say that an ISP arbitrarily attaching content to the beginning of web content is a little less intrusive than a company inspecting your web content, analysing it, and sending some part of that content on to it's swarm of search-o-mat machines with their bottomless bins of archived logs and their penchant for using one users input to value-add another users experience...
Your search for "Nesmith offlington bengay cooking cough freetrade" returned 0 results.
Did you mean "ben smith of islington being gay is looking for a wee bit of rough trade" ?*
I opt in and out of all sorts of services with my ISP that benefit me or benefit them or benefit us both, and much of that might be considered spam or intrusive in the eyes of the right beholder. I have an advantage with my ISP as we have a contractually defined relationship focussed primarily on the delivery of network services and if I am not satisfied I can kick up a stink or move on. The wunderkind at Google, on the other hand, seem to be doing backflips trying to find new ways to turn my searches for supportive underwear and soft pornography into the next Web 2.0 meme.
* yes, I know it doesnt work this way.. yet..
And NO, I do not know a Ben Smith - of Islington or anywhere else - and if such a person exists I am in no position to make statements on their sexual preference or subsequent peccadillos and I am certainly neither inclined nor empowered to cast judgments on another person's personal life! Live free, and be the best you that you can be, Ben Smiths of the world!
re: Wrong
By Anonymous Coward
no essentially what they are doing is hi-jacking the browsers default 404 page only
in your scenario {a timeout} no 404 recieved so timout message given to user
if your server has any custom error page it is shown
only the default
<html><head><title>404 Errror</title></head><body><h1>404 Errror</h1><p>
the page was not found</p>
<address><a href="http://servername/">servername</a> apache vxx.x</address></body></html>
page tends to be small enough
I haven't seen the "land grabbing" message yet, but I think Google should clearly inform the user that they are being redirected, and provide them with a link in the redirected message to disable this "feature", and include the original 404 message.
Toolbars are evil things, who knows what they might also be hiding. I wouldn't install one on my machines and I discourage friends and family from such stupidity as well. But then I can cope with a command prompt.
There probably isn't a connection in users' minds between a toolbar which gives an easy way to access google services when explicitly requested and a search service.
While I agree this isn't like your ISP messing with data at the network level, it still isn't right. If I send out an email with a slightly incorrect url in it (perhaps reformatted by the email client so that the link is incomplete) I *really* don't want google stepping in and offering to send customers to my competitors, simply because I don't have a high enough page rank. I want the recipients to take a second look at the URL, see what the matter is and correctly enter the URL. I may be slightly incompetent, but that doesn't give google the right to send my customers somewhere else based on my email.
Software should be clear about what it is doing. I've no problem with google having a toolbar to allow easy access to its services because that what people expect, given the user interface, but when it starts slanting users' activities because they aren't fully aware of what is going on, it has gone too far. When I explicitly go to a search engine, I'm thinking about what I'm doing. When people click on URLs they are probably not evaluating what is happening in the same way. That's how malware gets installed or people end up being phished.
This reminds me of people who get angry when prompted "Are you sure you want delete this file?"
We usually judge software on their usefulness (let's stay impartial to presentation), so it's bizarre to read a story criticising a good feature.
Fair enough the lack of prior disclosure is a bit of a boo, but overall I'm with Google on this one.
So you get a Google page instead of a Microsoft one. Big deal.
Isn't this basically what Amazon's recently approved 404 patent is about?:
'A client component runs on a user computer in conjunction with a web browser and detects errors, such as but not limited to "404: page not found" errors, in which a requested web page or other object cannot be displayed. In response to detecting the error, the client component notifies an error processing server, which uses the URL of the failed request to identify an alternate object to display.'
That lawsuit would be interesting.
We must assume that end users are idiots.
They will not know that the toolbar from the company that has promised to do no evil, will be manipulating the content of the requested page.
It will be hard to explain to grandma, why she can not trust the links on "her favorite site", because the links and pages on her favorite site, may not be coming from her favorite site.
This is going to open up a whole new avenue for spam. People could send spam that reads "check out www.theregister.co.uk/my_credit_card_offer", then buy the key words or phrase of "my credit card offer". This will allow spammers to use the reputation of popular or trusted sites, to sell their goods on those trusted sites.
I wonder if google is going to do this with their own site too. I wonder if a person visited http://www.google.com/app, could they be shown a page full of Office live adverts? If not, then why should my site's reputation be stolen by google's advertisers?
This should definitely be an opt in type of system. Can you think of any legit site that would want their traffic stolen in this way? I can't. This seems like a tool for the spammers and advertising engines
I spend some time developing programs to try out features of whatever framework I work with. This quite often results in 404 errors, which I actually want to see. Fortunately, I don't use tool bars and usually turn them all off, seems like yet another serendipitous stroke of luck.
(Akin to not having Flash installed, not quite true, IE has it, but needs the machine rebooting at times if Flash is used. I am sure I could investigate and find a fix, but I can't be arsed, Firefox worked for most things and IE is just there to work with VS).
Amazon: How do the USPO issue patents, any feature is now patentable? Have they any software to implement this or is it just more patents for vapourware?
I'm with you on this one:
Google is just replacing a microsoft feature with a Google one*.
Its application specific, and you can use the "Uninstall programs" to get rid of it
It only works on people for windows, on IE, so install Linux, Firefox, or Safari, ...
- Joel
*bye-the-bye I hate that Microsoft gets to claim "30%" of search from people typing what they want into the address bar. People who use live search don't know what it is.
..is that I have yet to come across a PC, owned by one of my customers or a family member, with a tool bar that the owner could recall having deliberately installed. In fact some of them have row upon row of badly laid-out crud at the top of their browser, seriously diminishing available screen area. Generally they are not even used, the numerous jolly icons just sit there cluttering up the view like gargoyles on a roof, whilequietly ushering in various creepy 'features'.
I have been known to demonstrate the joy of hitting F11 to give a nice big screen to them - to exclamations of joy. Then the next time I see them they ask 'what was that trick for making the screen go big you did - I forgot...' or a panic phone call describing how they can't put an address in as all the stuff at the top of the screen has vanished when they open their browser.
I don't do that anymore, just mumble something about be careful what boxes you tick next time you update Acrobat reader and leave it.
If you voluntarily install a piece of trojan malware like Google Toolbar, you have to expect the people who convinced you to do something so stupid to use said trojan to their own ends. That is, after all, its intended behaviour. If I don't want said behaviour, I don't need to install their piece of invasive crap.
However, when I pay my ISP, I expect them to leave my bitstream the hell alone, and not insert or delete things at their whim. To do so is NOT expected or intended behaviour, and if I caught my ISP doing it I'd be very tempted to emulate the BOFH and his EtherKiller and make sure their isolation was up to spec.
As for IE, they do it in their browser already, and you CAN'T uninstall their piece of trojan malware if by some unfortunate circumstance you're stuck using their crap.
Yes, there is a huge difference.
why google toolbar is being bundled into every and any other bit of software going.
The latest version of daemon tools now asks you if you want to install that trash and has recently knocked itself from my usual list of installed applications because it tries to make a grab at being your default search provider (why? you are an iso mounting software, not a bloody search company!!) regardless of if you select yes or no.
also, on a side note tiscali was in the habit of hijaking typo'd domain addresses and showing you their "did you mean" page, littered with web ads. I a user argue with them demanding to be taken off the service, they claimed he wasn't on it. After about an hour he got a frustrated "oh ok, we'll take you off it then". hold on, you said he wasn't on that service not long ago... within a week of opting out, he was put back onto the service.
Google are American...America does not allow it's own citizens to freely travel to Cuba or to sell products there (although it will happily send people it wants to torture to Gitmo),. therefore no point Google to go there!
It's a lovely place, one of the nicest countries I've ever visited, cutoms far more freindly than LAX !
Whose fault is it if the webmaster isn't technical enough to monitor their own logfiles, and then not have a custom 404 file just on the offchance a link doesn't work? Its a stupid shame to make a site to attract visitors and make it pretty, while forgetting to take care of the basic themes
If google are displaying genuine 'did you means' and not sneakily disguised Sponsored Ads, then I don't see a problem.
Once it leaves you home network it is no longer yours. You are the bitch of every router, ips, load balancer, qos shaper, inspector, and rejecter between you and the destination you think you are going to. Your packets get rewritten, rerouted, proxied and cached. And most of it is done for your benefit.
Likewise with google, except they are being snaky in your turf - your system, your data - and sure you can disable it (just like you can disable the Rogers' ISP thing) but they know very well that the majority of lay users couldnt considering the implications of their "service" even if they wanted to.
You know your ISP (and others) can frig with your traffic so you TOR'ify or VPN but your search toolbar shouldnt suddenly start trapping your returned pages and pumping your data into google central's data-sump...
I know, I've installed it several times -- and it's all Adobe's fault.
You know, when you've got a trusted piece of software that auto-updates you can happily just hit "next" when the installer pops up right?
Wrong.
The Adobe Reader updater now has Google toolbar bundled, and the default position is that it installs it -- you have to untick the box to opt-out. The first time this happened, I didn't even look and the next time I used IE (I don't use it much) I wondered where the Toolbar had come from. Well, the next couple of times I installed an upgrade of Acrobat I remembered just too late and had to manually uninstall.
Anyway, the Adobe Reader installer definitely doesn't tell you everything about Google Toolbar, so it's hardly an informed opt-in service.
I doubt anyone really cares whether you know what's going on -- anyone who bundles it gets a cut of the advertising cash so they just want as many people signed up as possible.
why not make a 404 page on your site with a google adsense search box in it so at least you can get a bit of google revenue for not having your links in complete order.
make it larger than 512 of course and if especially clever preload it with the relevant links
smiley for all that potential extra revenue to be donated by messrs page, brin et al. for having a messy website.
Most people who get the Google toolbar have unintentionally installed it because it came bundled with, say Acrobat Reader or Java.
So it has sneaked onto their computer because, let's face it, most people can't be bothered to go into the specific 'sub-component' checkboxes or read dull licence terms. And usually the firms who produce the software that the user *does* want are trusted names.
So we have what is getting perilously close to the idea of malware (forcing an opt-out is sneakier than an opt-in) and once you're lumbered with it, your webpages will be occasionally substituted without your *explicit* consent.
If this is viewed as the thin end of a wedge it becomes a very scary prospect, and if we give Google the benefit of the doubt (remembering their much-lauded slogan: 'Do no evil') this becomes merely very ill-advised.
Google's defence is also spurious; it should not be up to the webpage author to modify the pages served, simply to accommodate a third party's idosyncrasies!
I've not read everything terribly carefully, IANAL (and other excuses, yada yada), but... Isn't this covered by a patent recently awarded to Amazon?
http://patft.uspto.gov/netacgi/nph-Parser?patentnumber=7,325,045
The issue is not if I track 404, but what I do with them, if my page is <512, then google assume there page is more worthy with their nice sponsored links.
Legal responsibility for any alterations in the bitstream from a site surely lies with the owner of the device that makes the alteration.
If the spooks have a wiretap on your pipe and are altering the stream as well as snooping on it, they need a warrant.
If my ISP wants to alter that bitstream, they need to get their lawyers to sign off on a piece of paper that takes full editorial responsibility for every byte that enters my home. If they insist that their interference is part of the T&Cs, they should be challenged on "unfair contract terms". (I can't see how snooping my private conversations is a requirement of how they do business.)
However, if I install, or allow to be installed, some software on the PC, that's my fault. I own the machine. I have physical access to it. No-one else does. If I can't manage it, that's no-one else's fault. ("Doctor, doctor, it hurts when I do this...")
Of course, all this is much easier if you have the notion of common carrier protection in your legal system, because then you can mention that phrase and remind your ISP why they *need* its protection. It's even easier if your constitution bangs on about personal privacy rights.
I haven't installed the Google Toolbar, because it is not providing any aditional service for me, but if someone uses it, this would be additional service.
If a webmaster doesn't bother to provide any specific information why the user gets an 404 then the page provided by the Toolbar is useful and helpful.
You are suggesting, that this would be negative for the administrator of the website but I believe the customer will see this as additional service provided by the administrator with the help of google.
And if you don't like it, just provide customized 404-pages...
No big deal and in no way comparable to Rogers...
Dear CadeMetz,
This aticle would've read much better if you'd precluded your bias.
Google's "suggestions" are useful and may evolve into something more useful later depending on usage. Regardless of the fact that they have a business intention ultimately, as long as they are trying to SERVE THE USER in first place and then trying to derive some mileage out of it, I don't mind the ulterior motive (branding or otherwise). It's not morally repungent - it's give and take. Only problem might be if the website's 404 had something useful in it - which is never the case. If you claim that google is denying the website that 404'd a chance to do get some weblicity - IMO you are absolutely wrong!
However, what the ISP-you-mentioned did was plain ho*ing - Selling it's sweet spot for money.
Al.
..like me use a tiny 404 page to redirect the browser to an internal search page to help them find what they were looking for.
Good point.
Google for some reason feel qualified to offer advice about "unsafe" websites, through both their searches and the regular updates the firefox will search for, and yet they are incompetent at web security. How many times must gmail be broken into before they stop pretending to be security experts?
Yet another google "feature" I'll have to fix on my websites, just like their crappy precache. Maybe they could stop breaking our sites and try branching out into other crappy services for a change, like email...scripting...voice...data...errr. Ah well, at least when they run out of things to do we can get back to life without google.
The argument about it being user choice is crap.
For starters, Google Toolbar is bundled with so many different things these days that you have to make a concerted effort *not* to have it installed. We, as computer savvy users are aware that you have to watch install apps for the sneaky "Would you also like..." bit but most aren't and will just "I agree > Next > Next > Finish" their way through the setup.
Secondly, even if they do pay attention during setup, the majority of users don't understand what a 404 is, let alone what it's there for (more important than letting them know they/the site has made a whoopsie is providing suggestions on what to do about it). So when Google ask if they'd like to take the nasty errors away, of course they're gonna say yes.
Third, Google don't do anything which isn't about their bottom line in the end. Making money by inserting a handy search box in response to user typos or (heaven forbid) my balls up is not exactly wrong but still pisses me off a great deal. The argument that not very many sites should/do have <512b error pages is bunk too - if it was an insignificant proportion, they wouldn't waste their effort with this scheme.
Finally, the site I run is someone's small-scale livelihood. Like I said way up there, if someone goes off-piste, I need to help them out with getting back to the right place such that they may still make a purchase. If they're presented with a search box already populated with keywords from what they were looking for, the chances are the results of the search will simply redirect them to a bigger fish with a higher mystical PageRank and the customer will be lost.
<Farnsworth> If anyone wants me, I'll be in the Angry Dome </Farnsworth>
10.4 Client Error 4xx
The 4xx class of status code is intended for cases in which the
client seems to have erred. Except when responding to a HEAD request,
the server SHOULD include an entity containing an explanation of the
error situation, and whether it is a temporary or permanent
condition. These status codes are applicable to any request method.
User agents SHOULD display any included entity to the user.
It only says "SHOULD", not "MUST" - although I've found this annoying, developing/debugging web applications (as I do), which send diagnostic information with 404 messages. We had to work around this with MSIE, by padding the page.
