Setting up a mail server on a Linux system can seem daunting, but with the right tools and configurations, you can create a robust and secure email server. This guide will walk you through the process of setting up a mail server using Postfix, Dovecot, and SpamAssassin.

Prerequisites

Before starting, ensure you have the following:

• A Linux server (Ubuntu 20.04 is used in this example)
• Root or sudo access to the server
• A domain name pointing to your server's IP address

Step 1: Update Your System

First, make sure your system is up to date.

sudo apt update && sudo apt upgrade -y

Step 2: Install Postfix

Postfix is a popular mail transfer agent (MTA) that routes and delivers email.

sudo apt install postfix -y

During the installation, you will be prompted to configure Postfix. Select "Internet Site" and enter your domain name when asked.

Step 3: Configure Postfix

Edit the main Postfix configuration file to set up your domain and other settings.

sudo nano /etc/postfix/main.cf

Add or modify the following lines:

myhostname = mail.yourdomain.com
mydomain = yourdomain.com
myorigin = /etc/mailname
inet_interfaces = all
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
relayhost =
mynetworks = 127.0.0.0/8
home_mailbox = Maildir/

Save and close the file, then restart Postfix to apply the changes.

sudo systemctl restart postfix

Step 4: Install Dovecot

Dovecot is an IMAP and POP3 server that will allow users to retrieve their email.

sudo apt install dovecot-core dovecot-imapd dovecot-pop3d -y

Step 5: Configure Dovecot

Edit the Dovecot configuration file.

sudo nano /etc/dovecot/dovecot.conf

Ensure the following lines are present:

protocols = imap pop3

Next, configure Dovecot to use Maildir format.

sudo nano /etc/dovecot/conf.d/10-mail.conf

Modify the following line:

mail_location = maildir:~/Maildir

Save and close the file, then restart Dovecot.

sudo systemctl restart dovecot

Step 6: Install SpamAssassin

SpamAssassin is a tool to filter out spam.

sudo apt install spamassassin spamc -y

Enable and start the SpamAssassin service.

sudo systemctl enable spamassassin
sudo systemctl start spamassassin

Step 7: Configure Postfix to Use SpamAssassin

Edit the Postfix master configuration file.

sudo nano /etc/postfix/master.cf

Add the following lines at the end of the file:

smtp      inet  n       -       y       -       -       smtpd
  -o content_filter=spamassassin
spamassassin unix -     n       n       -       -       pipe
  user=spamd argv=/usr/bin/spamc -f -e
  /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Save and close the file, then restart Postfix.

sudo systemctl restart postfix

Step 8: Testing Your Mail Server

Create a new user to test the mail server.

sudo adduser testuser

Send a test email using the mail command.

echo "This is a test email." | mail -s "Test Email" testuser@yourdomain.com

Check the mail directory for the new email.

sudo -i -u testuser
cd ~/Maildir/new

You should see a new file representing the received email.

Conclusion

You've successfully set up a basic mail server on your Linux system using Postfix, Dovecot, and SpamAssassin. This setup can be expanded with additional features like SSL/TLS encryption, user authentication, and more.