How to Run Your Own Email Server on Red Hat Enterprise Linux

by Andrew Watters

Update March 14, 2018: I have decided to transition my own email server to a commercial provider. It was interesting running this server for about a year. That was a year full of headaches and uncertainties regarding my configuration, which was occasionally hacked and used to send spam due to my failure to recognize security holes. Protip: by default, the Postfix user authorization table permits all valid users on the server to send mail, even if they cannot connect via SSH. So for example, when I briefly had the postgres user password set to "postgres" for testing, I forgot to change it. Someone guessed the password and used the postgres user to send tens of thousands of spam messages before I caught the issue (which happened when I started getting spam notices from other email servers when sending email). Bottom line: running your own email server is an interesting exercise but it seriously dangerous and difficult. I have enough going on that I no longer want to worry about it. I never completed this tutorial, which I still plan to do at some point.

May 9, 2017

This is a tutorial for absolute beginners.

Introduction

There comes a time in a person's life when he wants to run his own email server. This can be as fleeting as a simple wish to have more control, or as serious as a bucket list item. In my case, it was a combination of both. Thirteen years ago in 2004, almost to the day, I started using DreamHost for my email. It was decent during that thirteen year period, but never great. There were several serious problems with it, including: (1) their SSL certificate never matched their mail server's host name in the entire thirteen years I was with DreamHost, so I would always get a security warning, (2) spam filtering was extremely ineffective, and I ended up with thousands of messages in my Junk mailbox that I had to comb through just to be sure I didn't miss anything important, (3) there was no way to control sender access to my inbox, so literally any spammer in the world could send me email, (4) I had no way of moving messages around on the server manually, and finally (5) it was not possible to back up my email manually. The last straw was recent, and deal-breaking: my email started getting flagged as spam by major providers such as GMail and Yahoo. As a lawyer, I have to be sure that my email gets through to clients, so this was unacceptable. In any case, all of these problems were solved when I started hosting my own email. I should have done this years ago, no joke. The purpose of this tutorial is to help you get your own email server up and running. Here is what you need to replicate this setup:

  1. A server running RHEL with an active subscription
  2. A business-class internet connection such as Comcast Business
  3. A static IP address that is clean, i.e., not on any spam blacklists
  4. Access to your MX record
  5. A SSL certificate for your correct hostname
  6. Postfix
  7. Dovecot
  8. Fail2Ban
Please keep in mind: this tutorial shows you the minimum to get your server up and running. It does not include SPF or DKIM-- two really desirable additional features that will make your email more likely to be marked as genuine, non-spam. I will update this tutorial once I add SPF and DKIM.

Connection Requirements

You need a reliable, always-on connection so your mail server is always accessible to someone who wants to send you email. You also need a static IP address so that your IP address can be evaluated against spam blacklists and so that you can have a real, fully qualified host name. Before starting this project, make sure your new IP address is not on any spam blacklists by using Spamhaus's lookup tool. If your IP is on there, you need to have it removed by following the instructions on the Spamhaus website before coming back to this tutorial.

DNS Configuration

The MX record tells external mail servers where to find your server on the internet. This part is really easy. All you do is enter "10 [your host name]" which sets that host name as the highest priority mail server.

Keep in mind that once you click that button, you are responsible for your own email from that point forward. Also, your old email will stop working almost immediately, so you had better back up or transfer your old emails using a method of your choice. You can set up backup mail servers if you like, but I wouldn't bother because mail servers will re-try sending if your server is down, giving you ample time to get it back up and running.

SSL Certificate

It's critical that you have a SSL certificate so that you can have a secure, TLS-encrypted connection between your email program and the server. Otherwise, your password will be transmitted in plain text and anyone between your email program and your server will have complete access to your email (and potentially your server). Also, you want a SSL certificate so you can receive TLS connections from other mail servers. You don't need anything fancy like an Extended Validation certificate. Just get something from a recognized certificate authority such as SSL.com. Once you are approved, you should have three things: (1) your certificate, (2) your private key, and (3) a certificate authority bundle. You will need each of these items when setting up Postfix below. For now, just make sure you have the three items on your server. I put them in my main user's home directory under a private directory called "cert" but it's up to you.

External Firewall Configuration

I have a security appliance between the internet and my server. It runs the pfSense operating system and is very effective. To get email running, you need to forward ports 25, 143, 587, and 993 from the internet to your mail server. Port 25 is for SMTP, port 143 is for IMAP, port 587 is for secure SMTP ("Submission"), and port 993 is for secure IMAP. Here is what my external firewall configuration looks like:

I have also forwarded port 465 in case I feel like using Outlook Express as my email program.

Server Firewall Configuration

In addition to forwarding the above four ports to your mail server, you must ensure that these ports are open on your mail server. The easiest way to do this is to use the RHEL graphical firewall utility and add the ports one by one.

Postfix

Postfix is a great mail server. It has numerous features and performs very well. It's also free, well-maintained, and well-supported. This make Postfix the natural choice for someone running his own email server. Postfix handles sending and receiving mail between your server and the internet. In other words, it does not handle delivery of email to local users. That's what Dovecot is for. Postfix is included in RHEL, but you should make sure you have the latest RHEL-approved version:

sudo yum update postfix

Create a new user who will be the mail delivery user. This can be anything from "vmail" to "justicewarrior" or anything in between. You can make the user a non-shell user for added security. I personally run my server with my main user as the mail delivery user. This isn't a good idea, but it is what it is. I did it that way through trial and error as I was trying to get the system working, and I just left it that way because it works. With fail2ban installed, I feel like I am pretty well-secured and don't need to worry about it excessively. Next time I will create a dedicated mail delivery user.

The Postfix configuration directory is found at /etc/postfix/

The most important configuration file is /etc/postfix/main.cf. Here is what the custom portion of mine looks like:

I'll discuss each option in turn.

The Dovecot interface is found in /etc/postfix/master.cf

The virtual address mappings are found in /etc/postfix/virtual and the PAM authentication aliases are in /etc/aliases

You can restrict any sender you like in sender_access using regular expressions.

Dovecot

Dovecot is a great IMAP and POP server. It's like Postfix in that there is a vibrant community maintaining and supporting it. It's also free. So it just makes sense. Fortunately, Dovecot is included in the RHEL repository, so there is no compilation or source configuration needed. Dovecot is not installed by default, so you need to install it:

sudo yum install dovecot













Setuid root for dovecot-lda by issuing the following command: "chmod 04750 /usr/local/libexec/dovecot/dovecot-lda". This permits Dovecot to enter any user's Maildir and deliver messages. Although insecure because it would allow local users to potentially escalate privileges, it's really the only way to deliver mail to multiple users using one mail delivery user. See here for more information.

Fail2Ban

Fail2Ban is a great tool that automatically bans IP addresses when someone has a number of authentication failures to your server. It does this by examining your server log files to find authentication failures, and then adding those IP addresses to your server's firewall. Fail2Ban is not included in RHEL or in the RHEL repository, so your best bet is to install it from the official site. There is an install script that you run as root to get everything installed. After that, you must configure jail.conf and then start fail2ban from the command line.





Conclusion

If you followed the above exactly right, you should have a functioning email server. If it didn't work, feel free to email me at andrew@andrewwatters.com and I'll see what I can do. Or if you want me to set up your server for you, I charge $200 per hour and it takes 10-20 hours ($2,000 to $4,000). Good luck, and happy emailing.

top