The sooner we nail Sendgrid into its coffin the better.
Cloud-based email marketing service SendGrid has copped to blabbing customer email addresses, chalking it up to some overenthusiastic indexing without explaining why pages were public-facing in the first place. In a breach notice sent out on Tuesday 2 October, SendGrid said that "some email addresses processed through the …
Thursday 4th October 2018 11:18 GMT adam payne
Thursday 4th October 2018 13:15 GMT Pascal
The way it works is that messages they send out have unsubscribe links.
Those links ask for confirmation - displaying the name/address about to unsubscribe, and from what list.
This isn't supposed to be crawlable as nothing refers to those URLs in the first place, and robots.txt files block those paths from indexing.
But then people receive messages and post the full message to various forums or other public sites- for whatever reason (sharing messages they got, etc.) and they don't remove those link.
The forums are indexed, and follow those links -- and Google explicitly ignores robot.txt for content linked from other sites, they only honor explicit headers in pages under that circumstance.
The leak really originates from specific users posting their messages online, added to Sendgrid not understanding how google indexing / robots.txt work.
This specific scenario has happened to most major online services like that - any service that has any sort of links that automate access to any types of profiles is susceptible to that kind of indexing when users don't understand where they're posting those links.
Thursday 4th October 2018 13:31 GMT Anonymous Coward
Fair points, but that doesn't mitigate the fact that sendgrid are a steaming pile of shite that we would be much better off without.
I would dearly love to be able to block them on my mail servers, but have got a few customers on legitimate mailing lists which (however misguided) use sendgrid.
Thursday 4th October 2018 13:43 GMT GnuTzu
"as nothing refers to those URLs"
@Pascal, Really??? Then how are they crawl-able? They'd have to be links in a page that can be found.
And, as others are pointing out, in their own way, a strong security control is not optional and robots.txt is optional, thus robots.txt is not a strong security control.
Thursday 4th October 2018 11:47 GMT iron
Thursday 4th October 2018 15:10 GMT tiggity
I get plenty of access to data that should not be spidered (based on looking at logs)
And not an issue of links followed from elsewhere, as includes data that has been uploaded to the web site "in advance" of other changes i.e. before any publicly available links have been made to it, so could only be grabbed via spiders.
.. though in my case data this is not important, no PII, just new inoffensive niche content on a hobbyist website
Thursday 4th October 2018 17:34 GMT Anonymous Coward
3rd-Party Cloud Email Solutions
This must have seemed like a good idea in the more innocent world of the net a decade ago. But now its just another steady attack vector. I recently tried to unsubscribe from everything. That involved being taken to the likes of Marketo and Masterbase etc... Neither of their opt-out pages worked.
Tried to contact the actual firm who gave out my email address, no luck either! It really helps too when you write to a firm to opt-out, but get this as a reply 'Rejected high-probability of spam'. I just unplug more these days!
Thursday 4th October 2018 21:37 GMT Doctor Syntax
"The Reg asked Sendgrid yesterday why it hadn't focused on making sure nobody could access the pages without proper credentials, instead of just asking crawlers to please not show the information in their search results. We'll update when it responds."
Don't hold your breath. They probably don't understand the question. They're in marketing.
Thursday 4th October 2018 23:46 GMT AustinTX
Mailgun has even bigger problems
Recapping what I wrote here on El Reg a week or so ago is the problem I had with Mailgun SMTP, a freemium email relay service:
Your account is associated with one of Mailgun's SMTP relay servers when you sign up. Many other Mailgun customers share that server with you. Your local SMTP server relays all outgoing email to Mailgun's server, and typically, all of your incoming email comes from the relay too. If your email traffic starts getting blackholed, you can ask Mailgun's staff to switch you to another at random, which may have a better reputation than the one you had.
If you are a spammer looking to avoid being identified and trick others into paying for your deliveries, you just need to find domains which are served by Mailgun SMTP relay servers. Probably, you'll harvest this from header information in other email traffic you're collecting. Another possibility is spamming many domains with "Delivery Status Notification" turned on and looking to see if Mailgun servers convey the response. I'm not really sure. If you sign up for a bunch of Mailgun accounts, request switches, ect., you'll likely manage to acquire accounts with one of each SMTP servers which they offer. Then, all you need to do is send a payload of spam FROM one of your accounts that shares the SMTP relay server with this victim TO their local SMTP server, addressed to various addresses from your spamming list.
Since email servers like Postfix, treat an SMTP relay/gateway as a trusted peer on the local network, it does not consider email which is injected this way to be relay mail. It treats it with the same trust as your workstation or whichever local machine you send your email from. The victim's SMTP server re-sends the spam email back OUT through Mailgun under it's own reputation and quota. It skips local spam filters because since when do you scan "outgoing" email submitted by a trusted peer for spam? And so the spammer uses up your 10K free quota, and then your paid quota if you have one. It doesn't require your victim's login credentials, as Mailgun has given you your own. And, if there's any way to stop this exploit in configuration, I don't know what that is. If you take the SMTP relay server off your Postfix "local networks" list, then while it won't accept mail from there, nor will it send there any more either.
I provided Mailgun staff with every detail I had, log entries, copies of the spammer's incoming emails (which the spammer had stuffed with as many to: cc: and bcc: addresses as possible), but they pigheadedly refused to understand. I was scolded for running an open relay and they said there was no indication one of their other customers was doing anything. Oh, please! The emails I'd captured had all the headers and session data. I get the feeling one of their staff is dirty and exploiting customers who rarely use up their monthly quotas.
My workaround is to block incoming email from Mailgun, at our firewall. Our MX configuration now advertises our cable modem IP address for directly incoming email traffic. Also, Delivery Status Notification has been disabled, though that means legitimate folks won't get address bounce messages.
Saturday 6th October 2018 05:43 GMT Kevin McMurtrie