The growing abuse of webmail services to send spam has led anti-spam services to throttle messages from Gmail and Yahoo! Over recent months security firms have reported that the Windows Live CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) used by Hotmail, and the equivalent system at Gmail, …
Gmail Spams Itself
Gmail not only sticks mail from Google itself into the Spam folder, but also marks these messages with the anti-phishing message, "Warning: This message may not be from whom it claims to be. Beware of following any links in it or of providing the sender with any personal information."
So, on one hand Google is sending me emails about update my Adwords billing info, yet on the other they're hiding them from my inbox, and telling me not to trust these emails. What to do?
Throttle account registration to prevent spam?
If spammers are using sweatshops to circumvent CAPTCHA, why not introduce a system that causes delays in setting up multiple webmail accounts e.g., from same IP address (space) effectively?
Another approach could be Google et al. to set up a limit against sending too many e-mails to too many recipients. A limit not even noticed by bona fide users could be detrimental for spammers sending large numbers of e-mails. Gmail could also throttle heavy outbound traffic that too many of the recipients report as spam. "Many mail recipients have reported your e-mails as spam, for the next X days you can only send Y e-mails per day, to max. Z recipients each..". A bit of cooperation between the mail providers would be helpful here, too.
Obviously, these measures would not stop spam entirely, but if the spamming process turns costly and complicated, it reduces the profitability = it works.
Gmail Spam Filtering
I'm always astonished at how well Google Mail filters spam - way better than my own mailserver's spamassassin config. I always assumed it was down to their feeding back the results of the mass ranks of gmail users marking mails as spam.
Presumably a similar method can be used to limit the abuse of spam _from_ gmail accounts. If Google see lots of spam arriving from user firstname.lastname@example.org they could automatically disable that account.
Just a thought...
I assume that most gmail accounts are for personal use and generate little outgoing mail, relative to commercial operations. I don't imagine that commercial users would object to a more stringent sign-up process, leaving the rest of us with a limit of, say 100 outgoing messages/day?
As for CAPTCHAS, has anyone tried using faces?
I've often thought this would be an easy way to stop spam. Simply have a limit on the number of emails an account can send in a day.
If you need to send hundreds of genuine emails then you need to get a premium account, which will cost money - even a few cents an email will make spam unprofitable.
IP ban won't work
the spammers are using a thounsands of compromised computers to fill in the email account details the humans only solve the CAPTCHA which is then passed to gmail via the compromised computer.
As they have tens if not hundreds of thousands of computers in their control they could still create thousands of spam accounts per day even if you could only apply for one email/day/ip...
As for CAPTCHAS, has anyone tried using faces?
But how ... show a picture of a "celebrity", and hope the human guesses it.
Maybe a better CAPTCHA would be to show a photo from a large stock library, with one of a number of possible questions : "how many trees are there ?", "Is there a church in the picture".
They really need to
filter outgoing messages as possible spam too. Bounce it back to the sender. Or use the filter to make a note of who is sending lots of suspicious outgoing mail and delete the account if it crosses a certain threshold. Spam filtering should work boths ways. Incoming AND outgoing.
>> show a picture of a "celebrity", and hope the human guesses it.
Some of us manage to maintain our cultural vacuum* with few, or selective leaks. I wouldn't have recognised the Paris icon if it wasn't for the posts.
*I've had mine so long now it came with a free international flight!
spam filter on the output?
Why can't we just spamfilter the outgoing mails? As a bonus, automatically remove the account if signal/noise ratio is too high.
@anyone using faces
The point is that CAPTCHA is broken as they're using human slaves to interpret it. They're human, so it'll work.
Also, WTF is a celebrity? If it weren't for the image of Paris here, I'd have no idea who she is. In fact I've still no idea and care even less. What about cross-cultural problems; show me a picture of President Sarkozy's missus and I wouldn't recognise her (nor him for that matter unless he was wearing a string of onions and a beret).
This is why CAPTCHA uses mangled /Latin/ characters: everyone knows them, even Arabs and Chinese. Or, put it another way, I could even sign up on a Japanese website that had a CAPTCHA even though I don't speek the lingo. And of course that's the weakness; get a sweatshop full of barely literate slaves (or even British chavs) and they will sit there cracking CAPCHAs all day.
Of course the only answer is a universal ID card....
The answer is with Gmail et al
Unfortunately the answer lays firmly with Gmail, Hotmail, et al to validate email accounts.
They need to put newly created email accounts "on probation" where they're limited to sending very few emails a day until, say, a couple of months have passed. If someone needs to send many emails, they can upgrade to a paid account (which should make the web mail providers happy). Once the couple of months probation has passed, and if the email account has done nothing wrong (e.g. all outbound email should be scanned for spam) then the restrictions could be lifted.
Alternatively they should scan all their outbound email for spam.
Probably some stick will be required to do anything: the major spam validation engines need to blacklist the web email services for them to take action.
The odd thing is that it's in everyone's interests, except the spammers, to be more proactive in managing web email services. Gmail, Hotmail et al stand to loose a lot of credibility if this continues, ultimately leading to more people blacklisting their domains.
Thin end of the wedge...
"4.6 per cent of all spam originates from web mail-based services"
Erm, why aren't we going after the 95.4% instead of wasting time on the minority?!?!?!? We kill them off first then start worrying about the minnows.
@IP ban won't work
The point is that the spammers are out to make money. Anything that eats away at their margins will cut down the type of spam they can profitably send and hence the overall volume of spam.
Has anyone made a study of the profitability of spam operations btw? What is a Gmail account valued at these days? I presume the people running these kinds of scams are reasonably canny are are making a good profit for the legal risks they run.
My love is in league with the freeway
Perhaps, instead of showing a picture of a celebrity, GMail could display a Zen riddle, or something that requires a certain amount of judgement and intuition.
E.g. "This is a photograph of a tree. Are you outside the forest?" The correct answer of course is that the question is wrong, there is no outside. GMail will refuse entry to anyone who answers the question too quickly.
Such a system would slow the spammers down, and perhaps encourage them to abandon their desire for money, and indeed their desire in general.
re: Google is spammer heaven
That's newsgroups, not mail, and if it is a single cretin, there is such a thing as the killfile. Use an NNTP server and a Usenet client and you will see that such things still exist.
The problem is single domains (gmail.com, hotmail.com, yahoo.com) with millions of legitimate users. Because of this, it's hard to backlist those domains without affecting those legitimate users. Not only that, but the shear number of users means that users will often have spam-looking usernames, like joebloggs432432.
The world needs to cut down on free mass used email providers, and go back to the days when you got an account where you worked/studied, or from your isp, or even bought your own domain and had it hosted (very cheap these days, and gives you some individuality). Lots of educational establishments used to give out lifetime email accounts, that seems to be less common now as they have to pay per user licensing costs for proprietary email servers like exchange.
Even worse is people using free email providers for business email, how can you take a company seriously when they have email@example.com painted on the side of their van? Registering their own domain would have been cheaper than paying someone to paint their van.
Bring back account creation by invitation only
i acknowledge that its not the ideal solution but i would be happy to have the email account creation by invitation only scheme google used when it rolled out gmail for testing purposes and limiting the number of invitations to maximum of one or two invites a day.
The ability to allow users to create accounts on the fly does allow anyone to freely open account. In an ideal world this would have been fine but we don't live an ideal world or else everyone will be using open source stuff :)
Having an invite method in addition with a system like CAPTCHA will hopefully reduce the number of account creation and hopefully make it slightly more tedious for spammers to use sweatshops in india to follow the process of account creation.
It is also interesting to see how google opts to tackle the problem as more spam accounts from gmail will in effect also increase revenue for google thru the adverts they display with each email. Whether they choose to resolve the issue or follow other major companies with money in mind lets wait and see :(
Paris; coz even she can create a gmail account now
@ My love is in league with the freeway
problem would be then that "educated" people who do not spam at all would end up getting the question wrong, thus screwing themselves over in the process.
So basically your suggestion (maybe a too hard question i dunno) would probably end up ensuring that not many people get to send emails....
What about if i decide to send an email through outlook? - how would you stop that..? ;-)
Spam really grates my cheese
Spam is never going to stop, because the "whole industry" never gets on board, SPF had potential, and my home mail server is setup with it, it doesn't stop it being banned because it's on a public IP, the SPF record is updated within 1-5 minutes of my IP changing.
No harm in thinking solutions though, and we have, from other posters:
- Limit to X emails a day
- Restrict the number of sign ups per day per IP
- Spamfilter the outgoing mails
I'm assuming these sweatshops just create the account, and then automated software is used to send the spam. The simple fact they're using sweatshops shows CAPTHAS are working, so stick one on the "send mail" page, requiring the sweatshops to send each mail individually, dramatically increasing the running cost for spammers, while only adding a couple of seconds onto sending mail for legitimate users.
Say 1 person can send an email every 15 seconds, copy & paste the address, copy & paste the content, add attachments, fill in the CAPTCHA. Over an 8 hour shift, that one person could send 1,920 e-mails, that's a hell of a lot less than automated bots sending God knows how many thousands an hour.
The way the UK government is going, e-mail will be banned soon anyway.
lets think out of the box
if they cant pronounce their "V's" properly or pronounce Guitar as "gee-taar" with a hilly billy redneck accent they that should know it all on the head.
Don't delete SPAM accounts
Don't actually delete the SPAM accounts, just silently bin everything they send. This way the spammer has no idea if they are still getting through, unless they SPAM themselves just to see if it's still working.
There was someone spamming a newsgroup with abusive posts towards the people in that newsgroup, then he started sending total and usless junk messages into that newsgroup and it is still going on now 9 months later.
Sending an email to Google Abuse I got the reply back
"Thank you for your note. Google does not regularly monitor or censor
postings sent to Google Groups, but we do try to prevent wide-scale spam
and other forms of Usenet abuse. Please be assured that the information
you sent to us is being collected and taken into account. While we
understand how annoying off-topic posts can be, we aren't able to pursue
most complaints we receive about them. We are using the information you
provide to make large-scale improvements in preventing abuse. We
appreciate your help in our efforts to increase the quality of Google
Now because of their inability to stop a Google user that newsgroup is now dead, no posts anything in it apart from that rogue person.
Pay to open
I'm not expert in this area but surely if you had to paypal (or other method) a £1 (or similar) fee to open the email account (which is then free to use) then this would help put them off. You could then limit how many accounts could be setup with that paypal account within a set space of time.
You would also have the benefit of being able to prove who you are if your account is hijacked or you forget the password.
"by invitation only scheme google used when it rolled out gmail for testing purposes and limiting the number of invitations to maximum of one or two invites a day."
This is completely useless.
I'm a spammer. I get an invitation (I play nice on a forum and say please please, like people did for gmail accounts).
Next day, I have 2 accounts as I send myself (well, my bot does it) an invitation, being limited to 1 a day.
Next day, I have 4.
After 10 days and an hour (the hour it took to get the first invitation from a forum), I have 1000 accounts.
After 20 days, I have 1,000,000
After a 33 days, I have more accounts than human beings on the planet.
Not such a great idea. If *I* want to invite three friends, I can't.
But it does absolutely nothing else than annoy legitimate users and prevent the amount of spam worldwide from doing more than doubling each day.
Please, please, everyone blackhole mail you don't want.
Most of my spam are responses to mails using addresses in my domain name.
"We have detected that this message is SPAM"
If the filter programs would scan the message body & send replies to the addresses it found there, rather than the from/reply address which is always bogus, my spam box would be practically empty.
ways to beat the sweatshops?
Could this work?
- Credit card authentication. Users are severely limited to say 10 outgoing emails per day unless they verify who they are via a credit card check - the infrastructure is already there thanks to google checkout.
- Phone numbers, the user must give a valid phone number - google calls them (could be an automated call) to verify that it is really their number. If later found to be spamming then they can be traced by finding who bought the number.
It's perfectly obvious what needs doing
1. The captcha test should have to be passed every time you send an email rather than just when you create an account
2. When you send an email there should be a delay of about 10 seconds while it tells you a joke or something to pass the time. Not so bad for the average user but x 1000 spam emails and the spammers capability is severely limited.
The price of freeloading
The price we pay for 'free' services is this sort of shite.
STOP giving away free email accounts. When people have to pay for something then perhaps they'll value it more. I don't think a charge of a few dollars/pounds a year is unreasonable for something people now find as vital as an email account.
Awaiting the responses of "what about the poor/developing countries"...
"Don't actually delete the SPAM accounts, just silently bin everything they send. This way the spammer has no idea if they are still getting through, unless they SPAM themselves just to see if it's still working."
It's still a waste of resources for the email provider - it still has to process a frontend for the user, or an SMTP connection to their client machine.
Just junk the accounts.
Spam filters never used to annoy me, but I have missed an interview for a desktop support role with a City firm because the agent simply used the subject "FW" in his email.
Problem with all the suggestions ....
Everyone is assuming that the spammers are using the "normal" methods of send spam, like actually logging on to Hotmail or Gmail or using Outlook. But they use their own programs (not difficult to write), and so where would the CAPTCHA test come from, or credit card authentication? Also, if people would check on the spam they receive, the From email address is normally different from the Reply email address. The addresses get spoofed. So a better idea would be to send an email reply to all the email addresses held within the body of the email. Any that get bounced means the address is not valid, and if you suddenly get lots of emails from servers you know nothing about, you will now know you need to run a virus check very quickly because you are part of a botnet, or your email address has been compromised.
Mine is the big target on the back.
The solution to usenet spam is just to get a decent newsreader and possibly a proxy as described here http://improve-usenet.org/ Filter out anything from googlegroups and you're back with a nice, clean news service.
It is a bit strange that Google don't appear to care that they're trashing the usefulness of one of their products by allowing Google groups accounts to send messages, but that's their problem.
@ Ed Mozley
oh heck no! I have a hard enough time figuring out some captchas ONCE, forget every time I want to send an email.
It's unfortunate but free accounts are both useful and necessary...
Free accounts are required for many reasons, mainly to do with anonymity:
* signing up to websites to avoid their spam
* signing up to a website to track their spam
* a temporary email address for an advert (e.g. newspaper/website)
* a one-off email address to bait phishers / scammers / marketingdroidtards
* testing applications
And why on earth should I let everyone have my 'real' email address? This is the internet for goodness sake.
In exchange for giving us a "free" email account, they get access to our emails (Gmail) and display advertising. Therefore they, Gmail, Hotmail, etc. have a vested interest in sorting it out.
Has it not occurred to you that this email from Adwords might be a phishing email? Why would Google possibly need you to update your details? Have your ads indeed stopped running?
Paris because... erm... he and she might appear to be well suited!
None of these "ideas" will work
All these ideas about limiting numbers of mails, improving captchas or whatever will work. None of the spammers actually sit there in front of a PC typing in the thousand mails, they use an open relay somewhere on the net and a script.
It doesn't matter what limitation hotmail puts on me, 1 email a day with a captcha to guess and a 10 second delay. The spammer can still send his messages through the relay without a problem.
What we need to do is move to a solution that requires computation to send a mail (like PGP/SMIME signing or encrypting). That way the spammer needs more PC power to send the message. If he wants to send 1 message it takes a second of his CPU time to do it.
Then he needs to buy a botnet :-(.
The "auto-reply" to check validity of an address works for a while, except we're talking about spammers that have registered a valid gmail/hotmail/... account. So the reply will be delivered. Of course you can require a response to that reply to add the sender to your whitelist, but there are always idiots out there who can't understand what they're supposed to do. Most of them are users at work rather than friends, so it's not a big problem. But for the stupid masses to sign up, it's too complicated. (like walking AND chewing, talking AND thinking being president AND not being a dick etc....)
Oh well, looks like we're stuck with spam.
@James Pickett AND @faces
"As for CAPTCHAS, has anyone tried using faces?"
At last, Paris can have a purpose.
@ Hany Mustapha
Quote "Paris because... erm... he and she might appear to be well suited!"
They're not phishing spam emails in my Gmail. They're from Google. Therein lies the humour.
It's very simple. Just like Paris...
"Obtaining a working Gmail account has a number of advantages for spammers. As well as gaining access to Google's services in general, spammers receive an address whose domain is highly unlikely to be blacklisted, helping them defeat one aspect of anti-spam defences. Gmail also has the benefit of being free to use."
Haven't I read that exact phase 3 times now in ElReg GMail stories?
Only Workable Solution
Appears to be to start blacklisting service providers until they clean their acts up.
When users start getting "Your mail was rejected because your service provider does not maintain their system against UCE properly" then they'll start voting with their feet.
instead of using letters and number. the server could generate a 'smart' picture.
Ik could generate s composite image on the fly.For example :afruitbasket with apples,pears,oranges, and then ask the question: how many oranges ?
someone else posted the idea of using stock photo : that could be automatically circumvented. it's a matter of cloning the databse.
if the images are 'generated' then this is not possible.
another thing could be cartoon like faces. show 10 faces , with a random male/female ratio. and ask questions like : how many have brown hair...
if you overlay them a bit then pixel mapping tools would have trouble 'counting' colored zones, but for a human the difference is still clear..
or you could still use numbers and letters in the catpcha. deform them and give them a color. overlay the letters partly. then ask question to spell only the letters in a particular color . again matching algorithms would fail. make sure there is overlap between letters of the same color.
for example: the text HELLOWORLD. H and E are partially overlapping and green.
E and L are partially overlapping. both L are yellow. and so on. then RLD are green again .
the answer would be HERLD if the question was green text only..
if you break the 'hello world ' in two lines wo that words overlap vertically it would become a real nightmare. you would have colored 'blurbs' but the human eye can still read this wehre as a computer this would fail. if you then warp the text a bit too the game is really on ...
...doesn't really matter if they're hiring people to figure the things out.
So you want a new free email address?
Best prove you are Mutual then. Here is a list of 20 email addresses which send more than 100 emails per day.
Check through this anonymised list and tell us which addresses are spam. When you are done, we'll cross-check your results with six other people.
Failure to achieve better than 15 out of 20 correct answers will result in no email address -but not to worry, you can keep trying until you get IT right.
Privacy? We were all told in the 90's that email is only as private as a postcard. As I understand IT, this is still the case.
And the voluntary spam checkers? -Think of them as temporary post office staff, sorting postcards.
Re: spam filter on the output?
That wouldn't work -- it could catch legitimate e-mail on the way out. If I start sending friends and family e-mails about great deals I found on eBay (that's germane to their interests) or craigslist or ..., then those mails will almost certainly be flagged as spam, even though they may *not* be spam to them.
And most non-SPF reliant spam filters rely upon word/phrase detection, so if you included a few "bad" phrases into your mails, legitimately, you'd be in the same spot as above.
Wait for a while, then if detected as "bad", bounce back and request confirmation. The confirmation may be as simple as "what is your city" which you answered when you setup the account. Then the mail goes out. Anything that causes interaction on the outbound side will help.
As a note:
Spam exists because it works. People are (somehow) making money doing it. If we, by any means, make it less profitable it WILL go away. Any bit helps in this task.
Making SMTP die and come back with safeguards might do the trick! (we wish!)
Is it a cat or a dog? You decide...
I'm amazed that nobody has mentioned Microsoft's Asirra Project http://research.microsoft.com/asirra/ where you are asked to select all the cats out of a set of ten cats and dogs. The important matter is that the pool of cats and dogs is in the millions, so almost no chance of duplications.
And to Vincent Himpe I have two words to say: colour blindness (and thus his idea could be challenged under the Disability Discrimination Act).
The problem *cannot* be solved with CAPCHAs. By definition they have to be read by humans, so the spammers get human slaves to do it. Whether these slaves live in third world countries and work for $1/day, or they're spotty yoofs running the 'stripper' program, humans are the weakest link.
Can we turn Skynet on now please. Or is that the new name for a botnet?
I always like kittenauth
I've setup kittenauth for an image CAPTCHA system. It is nice because it is easier for a human to get right (I always screw up the text ones ones or twice) and harder for a computer.
Nice thing is you use quite a variety of pictures and customize it for your users. Motorcycle site click all the Harleys, etc.
Nice idea, but it would REALLY screw anyone who's colour blind
- Xmas Round-up Ghosts of Christmas Past: Ten tech treats from yesteryear
- Analysis Microsoft's licence riddles give Linux and pals a free ride to virtual domination
- Review Hey Linux newbie: If you've never had a taste, try perfect Petra ... mmm, smells like Mint 16
- Special Report How Britain could have invented the iPhone: And how the Quangocracy cocked it up
- I KNOW how to SAVE Microsoft. Give Windows 8 away for FREE – analyst