back to article Google screws Scroogle

Scroogle, the privacy-friendly Google scraper, has been crippled by changes to one of the dominant search engine's interface pages. The unheralded alterations to google.com/ie yesterday could mean Scroogle has to be "permanently retired", its operator Daniel Brandt wrote. "Over the next few days we will attempt to contact …

COMMENTS

This topic is closed for new posts.
  1. John Lilburne

    Well fuck me

    Google objects to content scrapping?

  2. Gabriel Weinberg
    Thumb Up

    Another Scroogle Alternative

    http://duckduckgo.com/ is another Scroogle alternative. I'm the founder.

    No IPs stored--check out http://duckduckgo.com/privacy.html

    And we also have an encrypted version: https://duckduckgo.com/

    1. Anonymous Coward
      Anonymous Coward

      duckduckgo

      "This site requires JavaScript"

      Any site which requires me to allow you to run code on my box is a definite no-no for me.

    2. A. Coatsworth Silver badge
      Unhappy

      go go gadget javascript!

      the interface is nice, I like the privacy settings and the results (for just a small sample of searches I ran) look ok... but why, oh why, in the name of Cthulhu does it need javascript to work? It's probably th only search engine I've seen that doesn't work over NoScript.

      Well, at least including it in Mozilla's search bar bypasses that issue, but still...

  3. Anonymous Coward
    Anonymous Coward

    "If you have something that you don't want anyone to know, ...

    ...maybe you shouldn't be doing it in the first place."

    ... what ... so if I wanted my vote in the recent elections to remain private, my only choice according to oogle is not to vote at all?

    1. Andrew Jones 2
      Alert

      well obviously...

      Bad example - as your name is on the list - and the number that is written on the voting ballot is added to the list next to your name as you are handed it - then no - your vote is NOT private.....

      It is not strictly public either - as in the whole world cannot see who you voted for - but the information could probably used in the future.

      1. prathlev
        Welcome

        @Andrew Jones 2

        You must live in a strange country. :-)

        Where I live (Denmark) it works like this: One person verifies your ID (typically drivers licence) before taking your voting card (sent to you previously by snail mail) and putting a mark on the list next to your name. Another person, sitting next to the first person, hands you a voting ballot. Noone writes anything down about who gets which ballot.

        Are you sure it's trackable where you are? Most modern countries seem to use an anonymous system.

        1. Pascal Monett Silver badge

          It's the same in France

          It's unconstitutional to know who votes for who in baguette country.

      2. It wasnt me
        Thumb Down

        Thats exactly the point, you fool.

        Number next to name on ballot - no id.

        No name on ballot - no id.

        Vote can be traced if someone has both the ballot slips and the list (key) - you are protected from ballot box stuffing, as votes are auditable if required.

        Its an absolute necessity that the vote is both private and auditable.*

        *Or it was until postal voting introduced the possiblity of large scale un-auditable fraud into the electoral system.

  4. Anonymous Coward
    Anonymous Coward

    ixquick

    ixquick seems to be plastered with Google Adwords. I suspect it to be as effective an shield to Google's all-seeing eye as a particularly revealing hi-viz vest.

  5. EddieD
    Unhappy

    Bollocks

    Bollocks, bollocks, bollocks, bollocks.

    1. Russ Tarbox
      Joke

      Yours or clean ones?

      lol etc

  6. Anonymous Coward
    WTF?

    Scroogle

    Have Scroogle tried looking at:

    http://www.google.com/m/

    It's the mobile version, which hasn't changed - it also bears a striking resemblance to the old Google layout.

    This took me about 1 min to find. How hard did Scroogle look?!?

    1. I didn't do IT.
      Unhappy

      Re: How hard did they look?

      It may be that there is more going on here behind the scenes... As more and more people use this, there is bound to be reciprocation from Google. After all, this is revenue (but only potentially. But huge companies seem to thrive off these virtual, potential dollars).

      I would not put past Google to have a blocker for IP or if a session cookie was not in place that gets pulled down going to the main search page.

      It may also be that Scroogle is just tired of keeping up or keeping the project going.

      And then there is the always popular option of legal threats, etc.

  7. Matthew Collier
    Unhappy

    Scroogle *might* have already explained...

    ...why http://www.google.com/m/ might be no good to them:

    "P.S. Many thanks for all the support emails. A word to those who are emailing us with links to simplified Google pages with just the initial search entry box on them: Sorry, but this doesn't help us. It's the simplicity of the pages with search results that matter. Scroogle uses it's own gateway pages, but we parse out the links from Google's results page before we pass them back to you.

    See the "sample" link in the second paragraph of this page, where you have to mouse-over each link to see the snippet. Results like this were not too difficult to parse. More importantly, they were stable. The query to Google that produced results like this is illustrated by the URL behind the "Next" link at the bottom of that sample (which no longer works, obviously). The parameters in URLs like that could be manipulated for language selection and number of results per page, just like for all Google searches.

    Whether Google cares to restore this simple interface is a question of fundamental corporate policy. Frankly, we've always felt that we'd reach this point with Google sooner or later"

    (from: http://www.scroogle.org/cgi-bin/nbbw.cgi )

    Arse! :(

  8. Anonymous Coward
    Big Brother

    Meanwhile, in other news...

    Sales of tin foil hats have gone through the roof.

  9. Gav
    Thumb Down

    Screen scraping is a a kludge

    Scroggle wrote an interface that depended on screen scraping. Screen scraping is the kludgeyest of data transfer methods, easily broken with very little in the way of error tolerance. It is the method of last resort. No-one using it has any right to expect it to remain stable and functional for any length of time, never mind 8 years.

    1. Anonymous Coward
      Big Brother

      Missed the point much?

      That is all.

    2. Ammaross Danan
      FAIL

      Title

      Sorry, but I myself have written a screen scraper that's ran for a few years now. It is a kludge, it can break on a whim of the developer of the page being scraped, however, with proper maintainence, it is VERY easy to keep up. 3 minutes of analyzing reveals this:

      http://www.google.com/search?q=jobs&hl=en&start=0&sa=N

      hl=en is the language,

      start=0 is the page to view results. So, the "next" button can be set very easily. As for results per page, simply run two or more queries (and assume 10 results per page view) and compile them together.

      Shame Scroogle removed their "open" source. Any by chance have a link to their project page perhaps?

  10. Jeff 11
    Stop

    Google screwed no-one

    Scroogle screwed itself by using the most technically half-arsed possible way of leeching a web service.

  11. bluest.one
    Thumb Down

    !Screwed

    Why don't they just use the API instead of relying on something that Google was likely to change.

    Scroogle haven't been censored or blocked, they just relied on something for their service that couldn't be relied on.

    Typical anti-Google Regbollocks.

    1. Pascal Monett Silver badge

      Regbollocks ?

      Uh, technically they have been blocked, albeit perhaps not intentionally.

      I don't see where El Reg is at fault for reporting this, nor do I see any particular bias in the article.

      Maybe you should have read the entire article before answering ?

  12. Anonymous Coward
    Black Helicopters

    Trackmenot

    Unsurprising, as Google now seem to taking active measures against anyone using Trackmenot to poison their search result history as well.

    Where's our Evil Google icon, Reg, for the millionth time of asking?

    1. The Unexpected Bill
      Alert

      I haven't noticed that so much...

      TrackMeNot hasn't caused me any difficulties in using Google, nor any other search engine. I never got the "automated queries" error even when running it on nearly every computer in my home.

      What Google really hasn't liked is my Tor exit node. And that's just fine with me.

      Yes, I do believe we commenters need an evil Google icon (and if there's room, maybe a "good Google" icon for the times they do something right (i.e. not the sea of changes that have washed over all of Youtube over the past year)).

    2. BlueGreen

      Trackmenot tinhatter AC 19:26

      Any evidence of that? Other than perhaps you setting the TMN query frequency too high which google interprets quite reasonably as evidence of malware at your end and blocks you with a helpful alert as to why.

      Cure: dial it down.

    3. Anonymous Coward
      Big Brother

      Evil google icon?

      We've already had Big Brother for a while.

    4. Ralphe Neill
      Big Brother

      Evil Google

      Another vote for an Evil Google icon!

  13. Russ Tarbox
    Black Helicopters

    Am I the only one that doesn't "get" this whole privacy thing?

    I spend with my debit card, my bank knows where I've been and what I buy. I order things to my address, dozens of companies know where I live and what I buy. My car number plate is logged all over the country at petrol stations, my phone company knows where I've been and who I've called. My search provider knows what I search for.

    Big fucking deal. It's not like anyone real actually looks at this information; it's really used for targeting ads. And get this - it means that when I do see adverts they're actually useful! Sometimes it's even allowed me to find a great deal on an item. Which my bank knows I bought.

    50 years ago, my Grandad's butcher used to know his favourite type of sausages...

    1. Thomas 18
      Thumb Down

      Its about rights

      People should have the right to privacy even if they choose not to use it. You can shop with nothing but cash, use public transport and a pay as you go phone if you so choose. Each of these is an equivalent service even if it is less convenient.

      Your right that nobody looks at the data, that is until you are suspected of something. At which point they start misconstruing all your data.

    2. Paul M 1

      No, you're not the only one

      The trouble with many of the anti-Google brigade is that they assume that what happens on the net is private. It is not. The Internet is a public place and should be treated as such.

      How many times have we seen the Internet likened to a city centre when it comes to allowing children unfettered access? "You wouldn't let your child wander round the red light district so why let them on the net unsupervised?" Well it's the same for adults too.

      If you decide to wander into Ann Summers etc you have to accept that people will see you. Browse to a particular web site and be prepared for that to be available too. Whether anyone cares is of course another matter

      So if you want privacy take responsibility for it - use encryption, anonymizers, proxies etc.

      1. Gaz Jay

        In short...

        If your just an average person going about their life like everyone else and have nothing to worry about, then fine, bollox to your privacy!

        However, if you are conscious of security or a little worried about information that "may" be being stored about you then you also have the right to that privacy that was ignored by others.

        It all boils down to personal choice. But give us the right to choose!

    3. Anonymous Coward
      Grenade

      SCROOGLED

      "It's not like anyone real actually looks at this information"... Not yet. Not unitl you are labled a possible terrorist for attending a political rally, or dont like globalization, or are an activist, environmentalist, tea party member... not yet. Any right you thought you have are slowly being eroded away. Every mention of the word terrorism and they come up with new and creative ways to take away your liberties. Applauded by the lap dog media, it sickens me to see a once great nation built on god given rights to its citizens is constantly under attack from within.

      http://www.inteldaily.com/2010/05/obamas-shredding-the-constitution/

      "I spend with my debit card, my bank knows where I've been and what I buy." - I spend with cash and money orders.

      "I order things to my address, dozens of companies know where I live and what I buy." - I buy COD, or money order and order in a different name.

      "My car number plate is logged all over the country at petrol stations," - My car is registered to a corporation.

      "my phone company knows where I've been and who I've called." - I use VOIP, on a VPN.

      "50 years ago, my Grandad's butcher used to know his favourite type of sausages" - Now google knows what your sexual preferences are by search results. Thanks to your favorite google cookies.

      http://www.scroogle.org/doctorow.html

  14. Anonymous Coward
    Thumb Down

    Here be a Regular Scroogle user

    Scroogle returned consistent results, not biased by IP/country, search history, my hair color, etc.

    And 100 results without having to go to "Advanced search"

    Hope they get this sorted...

  15. pj3090

    Headline is backwards

    So if you think Google is evil, is it more appropriate to a) use a different search engine altogether, or b) keep using Google's services, but in a way that deprives them of any compensation or benefit? If your answer is b, then you are not really being an honest person.

  16. Doug Glass
    Go

    If You Can't Convince 'Em; Confuse 'Em

    Firefox with TrackMeNot. Lock and load.

  17. Displacement Activity

    Que?

    Works for me.

  18. Nebulo
    Happy

    O Happy Day

    Scroogle seems to be back again this morning. Hope it stays there.

  19. william henderson 1
    Thumb Up

    'tis

    up again :)

  20. Matthew Collier

    @Michael 2

    "It's rather moot anyway, your ISP will keep and track your interactions with scroogle and most of the other stuff (like your search results not being manipulated)"

    Not if you use the SSL version of scroogle. They know you're going to Scroogle, obviously, but not what you're searching for, not the results therein (of course, they get to see where you click on, shortly after, but you can't have everything (unless you use TOR or a VPN ;) )

This topic is closed for new posts.

Other stories you might like