Google's encrypted search casts shadow on web analytics
In adding SSL encryption to its primary search engine, Google isn't just protecting your traffic from anyone sniffing your network. It's also preventing third-party webmasters from tracking the search terms you used to find their sites. That may be a good thing for netizens intent on privacy lockdown. But for webmasters, it …
Will make the job of analyzing various malware sites easier - remember that many of them will only sent payload if you came from google, even going to the point of checking if the search was of just keywords, or URL and discriminating users based on that.
Then why do I get a 'This page contains both secure and nonsecure items. Do you want to display the nonsecure items?' when I go to https://www.google.com.
There is nothing evidently different on the page itself when you click Yes vs No, except that in the 'No' case it takes a little bit longer to load.
This possibly because in the 'No' option, an additional get (https://www.google.com//compressiontest/gzip.html) is done before the page loads fully, while in the 'Yes' option the same is done post page load.
Also the get of https://www.google.com//compressiontest/gzip.html gives me a 'Done but with errors on page' which on double-click gives the following error message:
Line; 71, Char: 50, Error: Access is denied, Code: 0, URL: http://www.google.co.uk
And the privacy report tells me it has blocked a cookie for http://www.google.co.uk
Is this because I'm *cringe* using IE 6/ XP or because of some goof up at the google end?
"Also the get of https://www.google.com//compressiontest/gzip.html gives me a 'Done but with errors on page' which on double-click gives the following error message:
Line; 71, Char: 50, Error: Access is denied, Code: 0, URL: http://www.google.co.uk
And the privacy report tells me it has blocked a cookie for http://www.google.co.uk"
Look at the protocol in the above 2 URLS - it is HTTP, not HTTP*S*
Hence the
"This page contains both secure and nonsecure items. Do you want to display the nonsecure items?'"
The above URLs are the "non-secure" items which you are preventing from loading by clicking no.
So yes, Google has made a boo-boo. :)
Works fine in IE8 - I get no prompt for secure/non-secure items... looks like it's your outdated browser that's having the issue...
"And yet somehow, not knowing this visitor's specific search term is protecting their privacy? Please. The only thing it does is make the life of a web master a much bigger pain in the ass than it was before."
Yeah, it isn't protecting their privacy at all, it's just.. erm.. protecting their privacy by not telling you the referer.
This is only really a "problem" with Google search anyway (and possibly bing, I've never used it). Most of the other search engines I use (ixquick, yauba, scroogle etc) provide a referer that doesn't show the search terms at all.
Privacy FTW :)
...once Phorm tried their product on the market. So perhaps all the e-commerce people who jumped up and down excitedly at the thought of what the Phorm offering might do for them should actually have thought about the unintended consequences.
Other sites could plug ssl more. What google could do is check for http/https versions and refer to the https version if coming from google's https version. Easy does it.
Only we then have to fess up to the fact even more that the ssl certificate infrastructure is horribly broken and not trustable (``verisign^Wsymantec only protects you from those they don't take money from''), and we may have to finally admit it and try and find a better solution. Oh deary deary us.
The future of the internet is encrypted transmissions. So what if it screws up web analytics.
It won't be the end of the internet if they can't analysis how you got to their site.
If Google gave every little girl in the world a pony, you could be sure Cade Metz would find a downside.
...several million tons of pony shit be enough of a downside?
Being a web-master I say screw myself. Why the hell should anyone know how we got there.
It's not normally webmasters who give a stuff about the analytics - we just make sure the damned thing is working properly. The only analytics I'm interested in are load times, the user's OS/Browser combination and any non HTTP 200 responses really.
Anything else is marketing :)
As a web developer I have to say that the concept of analytics and SEO (thats another rant) is a right pain and something I hate. Yes, we design and build our websites to be friendly to text readers, and therefore search engine bots, but it should stop there. I don't want middle management types poncing in to my office like they own the place, insisting we use completely irrelevant page titles because "its good for google". Well bollocks to you, I'll tell you what's good for google: writing the damn page content properly and updating it on a regular basis with the Content Management System I went to great pains to implement for you, so you didn't have to bother me with your shit every 5 minutes and I could actually get on with my job of constructing websites, rather than pissing about with body text on the old ones.
So now we can farm less data from our users? Well as far as I'm concerned, that's a good thing, maybe this company will learn to actually engage with its user-base rather than paying idiots to decide what it is they are thinking and how best to exploit that.
Anon. for obvious reasons.
Would the info be passed from one https site to another https site?
If so then problem solved!
The beta seems pretty fast I must say - haven't used it too much, but it seems to do the job.
ttfn
Of course the detail of what keywords the user used to reach a site are not lost, google has them and I'm sure will be more than willing to start charging webmasters for access to it.
It won't be long before google are adding their own detail to the link to allow tracking and enquiry by webmasters for a *fee*.
Have fun...
M
Clicky:"the only thing this does is make things a pain in the ass for webmasters"
or rather
Clicky:"the only thing this does is put me out of a job"
To me this has only a little to do with privacy. Letting web sites know what you were searching on is a very bad idea because some of them will modify their content accordingly. As long as we have that kind of sleazy behaviour, it is a good idea to block referrers. It will help keep websites honest.
On the other hand, I'm not sure that Google particularly cares how I feel about that. Since they will have this information and web masters won't, could they be planning to sell it? Either way, I win.
In order to protect little miss Cowherd from an assortment of web nasties I have, among other measures, configured my firewall to block URLs that contain certain words.
One of those words, I have discovered, also blocks Google Analytics.
Bonus!
Now the problem. If little miss Cowherd uses Google's encrypted search, then all of the query string will be encrypted and I won't be able to block based on what she is searching for (either deliberately or accidentally).
So, probably going to have to block port (outgoing) 443 to her PC.
configure your network to use OpenDNS and block the resolution of the GoogleAnalytics domain (along with loads of other stuff - pron, adware, etc)
works for me (and all machines on my home network) :-)
I agree. Blocking port 443 (outgoing) also blocks all encrypted forms for logins and views (banking, IRA self-service, medical sites, etc).
Even using OpenDNS, you can either block Google (entirely), or not. I want to filter on the query string, and you can't do that over SSL.
So 443 gets the chop on the kiddies computers.
There isn't a lot that Little Miss Cowherd would need SSL for, and when she does she will have to use the main computer (under supervision).
OpenDNS is a good suggestion though. Will certainly use it to block the pron. (just can't be use to block searches (accidental, or otherwise) for pron using SSL Google).
Why not just lock her up in a cupboard until she turns 18... nothing like having trust in your kids huh
If you don't control your kid's Internet activity you get one group of twatspanners berating you. If you *do* control their Internet activity a completely different group of assholes has a pop at you.
Just can't win.
I am left wondering at what this particular AC was attempting to achieve with that post and why he/she/it is so bereft of anything remotely resembling a pair of bollocks that he/she/it felt the need to post AC in the first place.
Does this mean that all those irritating sites that do no more than provide their own form of search for whatever you were looking for and get in the way of real results will get clobbered? If so, I'll be using https from now on, just to irritate them.
I really hope so. It will make searching so much better not to have to use google to find a search engine that gives more ads and less results, but you still need to read it through for a moment anyway before realising what toss it is.
I hate those rotten bastards! This could be the next best thing to leaving flaming bags of canid excreta on their doorsteps.
I think this place could use a torches-and-pitchforks icon, come to think of it.
Good title, that's exactly what I said when I read your post.
Either you have no idea how ssl works or I'm reading your post wrong.
Please tell me you are actually analysing packets captured "off the wire" from another device on your network or not even that, just captured packets using the same computer should do it and NOT just looking at the https google search query in your web browsers address bar?
Because if you are, then words cannot describe the utter cataclysmic fail of comprehension you have on so many levels of the subject matter that you comment on.
So other sites won't see what users searched for? Boo-hoo. This is useless for privacy anyway; everyone knows that Google is the first and worst offender in this regard, and they still have access to your information. Now they'll do so exclusively.
-dZ.
everything should run on https, fuck the analytics companies.
so the webmasters can't see all my traffic info - boohoo
my isp can't spy on my traffic easily - boohoo
the legalised mafias can't stalk me as I surf the web - boohoo.
Google still has options to deliver search term data to the destination site. But of course, you can always just use HTTPS yourself:
https://secure.grepular.com/Getting_The_Search_Term_From_Clicks_From_Google_SSL
"we would have no idea what people were searching for to get to our site, which is arguably the #1 reason to run analytics in the first place"
Yeah right.
Why do you care what people were searching for?
Answer : So you can tailor your site's apparent responses to get more hits that you don't deserve.
So, this is a bad thing for consumers how?
If sites stuck to the content they really have they might get less hits overall but they'd only get relevant hits rather than the huge number of mis-hits I find when googling.
If you want to know how I got to your web site and what I was looking for, then ask me while I'm there. I may be willing to tell you.
I assume that unless you're not only up to no good but insane, you won't want to offer, or to attract people with, something that you can't provide. (Although Googling "{name of latest product} review" often indicates otherwise. I want a review, you don't have one, you still want my eyeballs...)
Stealing search terms from my browser? I'd prefer you didn't do that, you creepy person.
Some sites that specialise in reference information use the referrer info to colour-code your search terms in the page they send, much as Google does for you if you go for the cached version. That can be a very handy time-saver on big pages.
There you go - at least one purely altruistic webmaster activity that's been stamped in the balls by this change.
PS @ "All sites should go SSL" - sod off. SSL adds overhead, and for a huge percentage of sites (and visitors thereof) there's absolutely no point to it. I agree that it'd be nice if all sites could have it as an option (and search engines could then direct like-for-like to SSL or non-SSL versions of the result), but asking all webmasters to shell out for a "trustworthy" certificate from one of the anointed sellers is bullshit. The trust that underpins SSL would be better entrusted to an NGO such as ICANN than brokered by a cartel of douchebags.
Just create your own certificate. Remember that the certificate is used for TWO reasons - 1 to 'ensure' the identity of the server, and 2 to be able to encrypt traffic. We are only interested in the second reason here, so VeriSign (Symmantec) need not be involved.
I agree about the overhead though - massive decrease in throughput unless you go for SSL accelerator cards.
I Take it that you are perfectly happy to visit a site that thinks it needs SSL, but only has a self-signed cert? Pretty sure all modern browsers are going to warn you that this is dangerous.
is just one of many sites blocked by my nameserver anyway.
At the moment, Google does not see the links you click on from a Google results page. This is true with or without SSL. Hold your mouse over a link, and in your browser's status bar you will see the real URL of the target page.
Compare this to Yahoo. Hold your mouse over a link that looks like the target page, and you will see a long ugly string that goes to rds.yahoo.com. Occasionally Google has done some redirects too, but as far as I know, only on a very limited "research" basis. Yahoo is a major culprit here -- for many years they've had at least one, and sometimes two, redirects on every link. I scraped Yahoo for about three years, and it was not much fun parsing out those redirects to get to the actual URL. Obviously, Yahoo and Google both know what search terms you used. But Yahoo also knows what you clicked on from their results page.
One thing to watch out for is whether Google will be tempted to install redirects on their links on SSL pages. That way they'd know what you clicked on, and could sell this data to webmasters. I actually don't think they will do this because it would be too obvious. It would undermine the public relations value of the SSL option they just rolled out.
Before the current refit of Google search, hovering over the link showed the link you'd expect to go to, but the second you clicked it, it changed to a Google link, with the real link encoded in to the end and your browser would use that instead.
They probably still can but the above doesn't work, possibly instead some strange javascript stuff in the background dynamically changing hrefs, but I'm too lazy to run Wireshark while Google searching to test.
Interesting results to be found if you Wireshark while installing Google Chrome, and then doing anything with Google Chrome after install. Each install has a unique ID. IP addresses? Google don't need 'em when they have your Chrome install's unique ID. No idea if Chromium suffers with this, not tried.
...but Google has placed an "onMouseDown" handler of the form "return clk(this.href,'','','res','8','','0CEIQFjAH')" on every search result and this clk function posts all of the parameters back to Google in a request for an image that returns a "204 No Content"
So they absolutely know what you are clicking on , what number on the page was the result that you clicked on and even the style of result ie is a video result.
You think google don't know what link you clicked on? They almost certainly do. It's pretty trivial to track link clicks by javascript, and google analytics will do it for you. Google used to use redirects for link tracking, but they probably realised people like to be able to copy links from the google search page, and the vast majority of people have JS turned on. There's a few people who block analytics or use noscript, but they're insignificant compared to the majority of people who allow JS.
Google will not give up on knowing what link you clicked through to. That's a major part of their search algorithm improvement. They want as much data on you as possible.
"So they absolutely know what you are clicking on ,.."
IF AND ONLY IF you are allowing their javascript to run...
both ssl and normal versions use this type of redirect despite the fact they show the actual url in the status bar
https://www.google.com/url?sa=t&source=web&ct=res&cd=2&ved=0CCYQFjA&url=http://en.wikipedia.org/wiki/Sex&rct=j&q=sex&ei=_Cf9S7WFJZSqnAOi3cSXCg&usg=AFQjCNHPSeEKdDPWJip-BQmed0EUGYBzGw
"I agree about the overhead though - massive decrease in throughput unless you go for SSL accelerator cards." ... on the server side, only. Heavy caching can reduce this even further. If you have once established an SSL connection to Google, there is no need to do the expensive Public Key session key exchange from this computer for the next couple of months.
Search for "session cache" here:
http://www.rtfm.com/openssl-examples/part2.pdf
Actually, a big hard disk might be as effective as cost-effective as a crypto coprocessor. Also, powerful CPUs are cheap these days. Go for a couple of AMD chips if you need more than a trivial server.
the extra power to do this encrypting at Google's end must add an overhead?
You could just disable / fudge your own HTTP referrer if you're using Firefox and RefControl:
https://addons.mozilla.org/en-US/firefox/addon/953/
Works a treat.
It seems to me that the solution is simple. I'm sure it's one that Google has already thought of:
In Google Webmaster Central for my site, I'd have some additional settings:
"Encode referrer data in GET request" Yes/No
"Variable Name for domain" (text input - default "dom")
"Variable Name for query string" (text input - default "q")
the when google shows my links on the SERPS, instead of linking to:
http://www.inventpartners.com/
It would link to:
http://www.inventpartners.com/?dom=google.com&q=web%20design
... and I can analyse that request string using analytics or logfile analysis.
