The last post: Building your own mail server, part 1 Email is one of those internet services that, like it or not, we all have to use. Yet the underlying protocols have been around since before the invention of spam (the electronic sort, of course), and have little in the way of protection. No junk mail. Pic: gajman, Flickr Internet email is far from perfect, but unless you …
I've used greylisting for years. However, lately some of the big providers (Hotmail and their ilk) from whom I do sometimes receive genuine emails, but who use vast server farms, and therefore routinely manage to make contact from a previously unseen server, are not processing the temporary errors properly - they're giving up after the one attempt, just like the spammers do.
I'm watching this fairly carefully, as it may mean I have to abandon the practice (it's no good getting rid of the crap if it costs me real email).
In some cases it may be being re-sent, but is coming through from a different IP address. If you have a facility to ignore the IP address, but can verify that it is from the same email address then Greylisting can still work. Not ideal I agree, because these things could be spoofed, but spammers are after the low-hanging fruit..
I use PostGrey at the moment, and that comes with a file called "postgrey_whitelist_clients" that lists specific domains that should be automatically whitelisted, because they either don't retry at all, have weird patterns, long delays, or big pools of sending addresses that make normal greylisting problematic.
You can, of course, tweak the list yourself, if you find specific problematic senders. Postfix does now also include a tool called PostGrey that can do things like RBL checks before a message even hits the SMTP server, and checks similar to PostGrey. That can, apparently, reduce the load by getting rid of a lot of problems before a message even gets as far as being fed into SpamAssassin. I intend to experiment with that, however since my present experience is with postgrey, that's what I'll be using here.
Nice one,
As it was mentioned by Anonymous generaly is a very bad idea to use mail server on a broadband connection. Google/hotmail/yahoo will reject emails straight away from these ip blocks mo matter what.
Of course the article says what hardware to buy first. Well do not buy anything, you will need a fix public ip from a proper isp, so therefore you need a virtual private server.
Mine is at the moment with OVH which cost me £1.99/month. Fix public ip, 1gb ram, 10gb hdd.
At Hetzner you could get this same server for about 6 quid.
So all in all if you buy any hardware and try to use your broadband for mail/web server, then you will spend 10x more on hardware than you should and also about the same on electric bill to run your small pc.
All small boxes eats up about 20-30w. Also if you try to use any proper MX relay for your broadband, that will also cost about the same amount monthly than a vps server.
Make a calcultation, spend days weeks on the whole lot and you will still end up buying a vps server.
I tried it didnt work, so dont waste your energy on it.
By the way if you have a vps with centos/debian then it takes to fully get configured about 1 day.
On mine i got postfix/dovecot/spamassassin + roundcubemail for webmail.
If you want ssl certificate them get it from ssls.com cost 7 quid for 1 year.
It is true that if you only want a mail server at home then it's not worth it and you are better off with a VPS but is that the only use you'll make of that server?
If you plan to have a home server then use it as a firewall, a media server, a file server, etc... while you use it also as a local relay server which fetches and send emails using your VPS where you are pointing your primary MX record. Like that it makes sense and you get the best of both platforms.
> Google/hotmail/yahoo will reject emails straight away
Sorry, that's bollocks. Unless you are referring to a home dynamic IP broadband connection.
On a fixed-IP broadband link I have no trouble emailing people but on the other hand I do get a lot of spam (attempts) from people on VPS connections and yes, various sub-ranges of 'OVH' feature in my block-list, along with various compute-cloud providers, 'mail relay hosts' etc.
But aside from that, having a personal email server is not primarily a financial decision.
Yes I was referring to a home broadband with Dynamic address.
But even for example Virginmedia's IP does not work properly and they only change the IP 1-1.5 years.
Most providers got these address blocks already, not a big deal, can be checked on ripe.net freely.
Also not to mention all SPAM filter providers, they do know these blocks already.
If you have a domain to play with, try a fix public IP address with a "wrong" not matching reverse dns address without having an SPF record for the domain. All providers Google/Hotmail/Yahoo will reject your email and will end up in the spam folder, marked as spam. Maybe I was wrong about the reject part, it will end up in the SPAM folder, clearly the user wont have it as a legitimate email.
Perhaps my response was a tad blunt...
> try a fix public IP address with a "wrong" not matching reverse dns address without having an SPF record for the domain
No argument with this bit - as I found out the hard way - stuff wasn't even going into spam folders, it was being silently disappeared after being 'accepted for delivery'. This still happens in some places, and for 'reject' you are effectively not entirely wrong if the recipient has a vast overflowing spam folder and doesn't have time to look.
There's too much money in the spam-processing business, it's tied too closely with malware and therefore the money in the virus processing industry, and as long as 99% of people don't know the true scale of it, there's not enough interest in picking up the task of killing something that big.
IMHO the recipient mail server shouldn't do this, it should refuse to accept it in the first place, which is the way I configure systems.
Not sure what the legal/contractual position is on this. My feeling is that "acceptance for delivery" constitutes delivery. Citations welcomed.
"As it was mentioned by Anonymous generaly is a very bad idea to use mail server on a broadband connection"
Rubbish. I have a mail server, very similar to what is being proposed in the article, running at home. It is also an FTP Server, Cloud Server, Media Server, Shared folder server, etc. It runs on Broadband, is behind a firewall, and has a dynamic IP.
It simply works. Has done for years. It got fried by a power spike and took less than a day to recover from the encrypted backups on Amazon S3.
It doesn't get any problems with Google, Amazon, Hotmail etc. It gets tested once a week to ensure it is not an open relay.
And it hosts multiple mail domains easily.
Why do this? Because its my data, in my hands. If any three letter agency wants access, they have to come to me to get it, so I will know.
Just because you couldn't set it up yourself is not a reason to tell others not to try.
Nice one, I'm on the verge of upgrading my mail server (custom qmail variant on OpenBSD) and it's good to see a decent writeup that clarifies one of the possible options because a technical manual is rarely informative about the true nature of the beast. That said, I will admit I might end up being horribly boring and stick with the one I know, partly because it's got "tinker with this" written all over it...
For system specs, noting that my email traffic volume is very low and the now vastly oversized replacement HDDs needed add-on PCI-IDE cards because the motherboards were too old to handle a disk bigger than a 1.44 floppy: current mail server is on a Pentium 233 with 96MB of memory and I think it used some swap when I had a shared printer temporarily attached to it for the faxes (these more useful than people might think) ; the soon-to-be-new mail server is a huge upgrade, being a VIA 400Mhz with 284MB of memory which is seems like overkill.
The current one is reliable but I had to make a custom CD to boot because the BIOS didn't like the setup and the new one seems to have a MAC address randomiser or an imminent nasty hardware fault (probably should RTFM) and I suppose the lesson here is that 'cheapest' (or re-used) is not always the same as 'appropriate minimalism'. Also, a new 20-times-spec box uses a quarter of the power but buying a new one kill half the fun of it...
I have been running my mail server on a Raspberry Pi B2 since my main server AMD64 MoBo died. Using Postfix (built from source) on Slackware it works a treat. And as it is using an SDCard, using dd to copy the image once and a while means I have pretty good back-ups (not worried about the mail, just the config).
Excellant points all. This is only the second time I've saved a comment section to PDF. Here I've got my eMachine Core 2 media center (eMachine) looking for a purpose aside from being effectively a file server from Hell. I look forward to the next installment.
[Aside from the then new PDP-11, BSD is/was my fave. On mi Amiga no less.]
This post has been deleted by its author
...and as the path of least resistance:
1) Buy a domain ($10/yr) [1]
2) Get yourself some cPanel webhosting (£29/yr) [2]
3) Use either an email client or use the built-in 'private' webmail, set up your email addresses and you're all good (if anyone needs a step-by-step let me know)
So why do it like that?
In a cost/risk/benefit sort of fashion, you get a lot of the benefit of 'private' kit; but very little of the brain-damage. You don't have to buy the machine; someone else is largely responsible for security; and you can usually use their certificate (certificate being a major source of brain-damage) for TLS. Plus the total cost is what you would be blowing in electricity anyway. You get a posh-looking email address; server logs; (optional) spam filter; webmail for those wot like it; and you're not going to be constantly blackholed if you pick the right host. If you're on a 'domestic' connection you're going to have to have some sort of bridge over your ISP in any event.
[1] You will often be offered a free domain with the webosting. I find it best to keep hosting and domains separate because you are *far* more likely to encounter problems with hosting than with a domain registrar. If you have to extract your domain from the host, it makes moving a lot more troublesome and time-consuming. If your domains are separate, you just point the nameservers to the new host and you can be back in business within the hour. Webhosting tends to be a bit boom-and-bust...it's all great to start with, so customers are attracted. The company expands faster than they can cope with and it all goes titsup. Seen it happen time and again.
[2] Vidahost. I (and a couple of clients) have been with them for a couple of years now and there's still no sign of impending apocalypse. They know what they're doing and support is good. They get the -extremely conditional, suspicious and grudging- moiety seal of approval. For now...
I ran my own own home server for several years on static IPs, then I got myself a dedicated host and I also ran mail for my family. I used Dovecot, postgrey and various other tools (clamav and spamassassin).
Eventually the dedicated host's HDD died and I spent ages doing a RAID recovery, doing backup recovery, etc. Frankly that tipped me over the edge. The maintenance, dealing with the odd mail that didn't get through and dealing with the hackers attempting to get in was tedious. Okay, dealing with my families requests was the most tedious part, but overall I didn't need the grief overall.
I have since moved my mail and other stuff to Dreamhost on an unlimited hosting deal and at least I don't have to think about it. The performance of Dreamhost mail isn't fantastic and the webmail is basic, but I am happy enough not to have to think about maintenance. I could move my mail to Google but I decided to draw a line somewhere and give myself a little control.
The spooks will be sniffing the wires between the sender and your email server. STARTTLS is a rather poor solution, which a lot of major email providers don't support, and which a MiTM can trivially short circuit after which the connection will still proceed, unencrypted for easy sniffing.
Plus it only helps for sending/receiving emails from others who the spooks can't get to. That's not the case for the vast majority of them, so if you email someone at aol.com, gmail.com, hotmail.com or so forth they'll just get your email on the other end.
You have to encrypt the body of the email if you want to be assured of security, but at that point it doesn't matter whether you run your own email server or not.
While SSL/TLS may help little for mail delivery between SMTP servers, they help a lot to protect your POP/IMAP/SMTP connection/authentication to the mail server.
One important thing to consider when setting up a mail server, is which kind of authentication methods it supports. Some are very unsecure over an unencrypted connection, and anyway using TLS/SSL both encrypts and authenticate (as long as certificates are used properly)
That really is irrelevant at this point. Just getting started with the safe/sane installation and configuration, especially in regards to normal threats, is the topic at hand (which probably explains the down votes). Now if we want to discuss going into competition against nation-state whose multiple intelligence organs have multi-billion dollar budgets. Each. It's an interesting topic but this neither the time nor the place for such discussion. Trust, secure channels, and not giving them a clue. Ahem.
It happened to me once that of the account I setup up on a new phone, luckily the "one to use when you'll know you'll get spammed", wasn't properly set up to use SSL/TLS.
After I used it through an hotel wifi, someone was sniffing data (I guess the hotel network was compromised), cracked the account password, and immediately tried to use it to spam through my server (it looked it didn't access the account through IMAP or POP, anyway, he would have found just mailing lists messages, luckily...).
Thereby if you believe a proper SSL/TLS setup is only useful to guard against state level attackers, I would suggest you to reconsider how many crooks are competent enough to perform relatively sophisticated attacks. Against most three letters agencies, probably SSL/TLS is too weak already.
This post has been deleted by its author
Yep, the config I'm using at the moment (as in the block diagram) does use amavisd-new, which is what summons ClamAV as well.
I did install Postfix with MySQL compiled in, and used that in a previous iteration of the system when I provided a load of mail aliases for a client, and it was easiest to tweak them that way. Now I have far fewer, so they're all in the text file instead. Space permitting, however, I will include notes on how you can use a database to handle one of your domains.
there is no "right" way to do this and any tutorial can be either a guide for those with simple requirements or a starting point for those who want to explore deeper - some great suggestions from commentards but be mindful of your own requirements before you get carried away with an ultra secure house of cards solution
And once you get it working keep a "gold build" backup so you can get your mail working again if it all goes pear shaped in future
My experience was the same as some other commenters - (some) outgoing emails never arrived at the destination.
I suspect that some SMTP servers would silently drop email based on the source IP address (i.e. they knew it was a residential ADSL connection)
It made the whole exercise pointless, and I ended up on gmail.
You need AT LEAST:
1. fixed IP address
2. your OWN Internet domain with a MX record
3. DNS pointing your domain to your fixed IP address
4. PTR record mapping your IP address to your domain (that is on your ISP's DNS server)
5. an ISP allowing direct SMTP traffic
If your mailservers hostname is hermes.yourdomain.uk the PTR must point to hermes.yourdomain.uk
Microsoft hosted domains will not receive mail unless you ask Microsoft to allow your mailserver (or have SPF records), they will however return an error to you.
I recently tried to enhance spam assassin with a free Cloudmark Authority anti-spam client (apparently most of UK ISPs like VirginMedia, Plusnet, TalkTalk use it) and so far it blocks around 25-30% extra spam comparing to spam-assassin alone.
I run a small tech company with a postfix MTA and I had a lot of spam before with spam assassin. I've also tried to short-circuit spam assassin whilst using Cloudmark filter and it delivers good 10-11x times faster performance and processing.
Another handy feature is the fact that Cloudmark Authority lets you to mark spam messages from the webmail client enabling the system to learn dynamically. Postfix integration all seemed to be pretty simple as well.
I've been very impressed with Fastmail for a few years now - seriously committed to reliable service (replicating in real-time between IBM Linux and Sun Solaris hosts in different DCs, to minimise common points of failure: firmware bugs, hardware flaws, OS bugs etc). So far it's been rock-solid for me.
I handle a bit of mail routing myself right now though, on VPSs (I have a few addresses I want special handling for, like blocking particular senders on the SMTP level - Fastmail can only filter post-delivery) - in the next month or two I'll probably shift the balance in that direction a bit further, so everything hits my machines first, then gets copied into Fastmail.
Like the article says, it's not scary or rocket science, just a little bit of effort to get full control. Well worth it for a lot of The Reg's target audience I suspect.
Bah, I really want to mention my employers solution to this (a FLOSS project to make debian + exim = other things easier) but I'm concerned about coming across as a bit shilly.
I'll waffle on about it a bit if someone wants me to, but suffice to say my entire (pitiful) online estate runs on it these days, not because I have to, but because it's really rather good in my humble opinion. We have some fairly hefty servers running dozens of domains using it, too, because it works nicely and tends to Not Break and is easy to configure - a nice mix.
Steven "Google Bytemark Symbiosis, kids" R
As an example, I see gerdesj linked to mail-tester, which was what I was looking for earlier; my simple mail config (with correct DNS, SSL, SPF) gets 9/10; the only thing it lacks is DKIM signing, mainly because I'm not up on that as yet. Just checked the docs, and it doesn't look too tricky. Suppose I'll have a crack at that this week.
Just FYI, natch.
Steven R
That brings me back. I used to use it with the mh mail client and exmh (which I think integrated with fetchmail). Despite exmh being written in tcl/tK, it was as nice to use as any "full fat" mail client I've used since.
The problem I eventually ran into back then was scalability. With the possibility of tens of thousands of emails, each with their own file, the mail directory could get really slow as the dir had to be rescanned for each sub-command. Mind you, that was in the days before the ext? filesystems had optimisations (automatic indexing or something) for huge directories like that. Even with the drawbacks, the maildir format still beat the alternative of a bunch of huge Inbox.bz files that needed to be decompressed twice when you were searching for something (once to find out which inbox file it was in, with no tools apart from zless) followed by a second decompress when you issue the command needed to extract the particular mail you want.
Of course, if I'd foreseen the need to index mailboxes before archiving I could totally have used something like glimpse on them instead of torturing myself with slow searches.
Nowadays, of course, all that seems like an anachronism when Google or Microsoft will happily index everything automatically. That's good, of course, but at what price?
A few suggestions if you want to DiY:
* You *must* have a static IP with A, MX and PTR records
* Exim for the MTA. You are welcome to try others eg Sendmail, Postfix or Qmail
* Greylisting is a great idea but it will get on your nerves after a while and is no more effective than blacklisting (sign ups to new services will be delayed by your greylist)
* Use Spamhaus, Hostkarma and co for blacklists - they are very good for an initial filter
* Spamassassin
* SPF, DKIM, DMARC - they will improve your "reputation" but be careful - they are complex beasts.
Test with this: http://www.mail-tester.com/
To really get to grips with it, from a standing start, allow at least 1 year. I'm not joking. You can get good results in a couple of hours but you will still be learning for years. I am.
Cheers
Jon
You can get good results in a couple of hours but you will still be learning for years. I am.
I don't imagine we'll ever stop learning - but a few hours' study really can give you a useful mailserver. It's not nearly as difficult as some[1] would paint it.
Vic.
[1] Including me, if I'm charging for my time :-)
I figure you need about 500,000 spam messages a month to be able to filter it out properly while minimal false positives. That means you have to be able to throw out about 10 gig a month of data over your home network. It is easy to collect that much if you just put in some random email address in a web page but the spammers will throw away the ones that look randmon like uizctyiutywe@example.com but bob@example.com will get far more spam. Common names all get spam as well so alice, bob and smith will get spam very soon after starting up a new server.
There are antispam services that you point your MX records to and they do the filtering and then deliver to your home server. They can install SSL certs so they only deliver to your dynamic IP address and some can do IPv6 which you might find is static. I have a computer in a data center in LAX and I've about given up on trying to filter spam and letting others try. I'm currently using MXGuardian which seems to work but is getting expensive as I keep finding more and more email address I set up years ago that are still being used. Most of the services are cost per doamin, cost per mailbox or cost per message. With over 100 people using my vanity domain over the last two decades, any of those options get expensive. My habbit of using a new email address everytime I print business cards just adds to the expense.
I have a domain that is over 20 years old
Mine are only about 15 years old.
with plenty of email address published in usenet and on the web and archived mailing lists
Yes, me too.
that is the level I need to process so that my spam level is less than 1%.
Well, I use <1% of the training spam you quote, and my spam level is << 1%, with approximately zero false positives[1].
I don't understand yuour figures.
Vic.
[1] It's silly to quote zero, as a single one ever breaks that promise. But my false positive level is negligible.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
