This article is a pre-release chapter in the upcoming “Take Control of OS X Server,” by Charles Edge, scheduled for public release later in 2014. Apart from, and , these chapters are available only to ; see  for details.
Despite the constant claims about how email is dead (to be replaced by instant messages, Facebook, Twitter, iMessage, or whatever), there is no communication medium more important than email to most organizations. Even people who claim they don’t use email much get testy when they miss an important message or can’t log in for some amount of time. Email is important.
It’s also simple—the SMTP, IMAP, and POP protocols on which email relies have been around for decades, and there’s not much to maintaining the database of user and message information. In OS X Server, enabling all the Unix apps that provide mail services under the hood is merely a matter of clicking an ON button and twiddling a few checkboxes.
But there’s a huge catch. Email is an ecosystem—no mail server can stand on its own, because it must be willing to accept messages from anywhere on the Internet and be capable of sending to anywhere. And due to spammers and other nogoodniks, every email server on the Internet is constantly being bombarded with spam, viruses, and malware, often sent by zombie computers marching in massive international botnets. And don’t get me started about dealing with distributed denial of service (DDoS) attacks, having your server used to relay spam, or the massive blowback that happens when one of your users’ email addresses is used as the return address for spam. As I said in an earlier chapter, email is a toxic hellstew, and I strongly recommend you avoid running your own mail server. I don’t, not any more.
Here are my recommendations:
That said, as much as I’m trying to warn you off enabling mail services in OS X Server, I’d be remiss if I didn’t walk you through the basics either so you can turn on mail services or so you can reflect on what would be involved.
And if you turn on mail services, I recommend keeping these configuration options and policies in mind as you go, because they will help secure your server:
Before you start configuring in the Server app, you have some ducks to line up. These tasks vary (widely) depending on your specific situation, but the general instructions below should get you started.
So far, we’ve been configuring your server with the assumption that it would be used only inside your LAN, and not be accessible to the outside Internet. For a mail server, that obviously won’t work. (Technically, you could set up the mail server to provide email for just staff in an office without any connections to the outside world, but in my experience, that’s a non-starter because users always want to exchange email with other Internet users.) So, some of these tasks relate to putting your server on the Internet.
Back in, in Chapter 3, you ensured that your server would have a static IP address on the LAN. It’s also essential that it have a static IP address for the outside Internet (the WAN, or wide area network, in networking jargon). You’ll need to set that up with your DNS provider.
In, in Chapter 3, I talked about how you might be able to use dynamic DNS instead, but for a mail server, dynamic DNS is a bad idea. Any interruptions in service will likely cause mail delivery failures, which will make your users crazy.
Along with that external static IP address, you’ll need to have a real domain under which all this mail will be handled, and an external hostname for the server. So, where we’ve been using
mavserver.pretendco.lan as an example for the internal domain name, for the purposes of this discussion, we’ll assume you want to receive email at
pretendco.com. And we’ll assume that your server’s hostname (and thus machine, or A record) will be
The tasks you’ll need to complete with your external DNS provider and ISP include:
osx.pretendco.com, pointing it at your external static IP address.
mail.pretendco.com, pointing it at
pretendco.comshould be handled by
It’s also essential that you open particular ports on your router so email can flow in and out. This is easily done in AirPort Utility, assuming you’re using an AirPort base station as your router; other routers should be similar. The ports you’ll need to forward may include the following (but if you’re not planning to let clients use POP, for instance, don’t forward those ports):
Happily, AirPort Utility lets you choose these from a pop-up menu (Figure 1), making configuration even easier, since it fills in the Public TCP Ports, Private IP Address (that of your server), and the Private TCP Ports fields for you. (To view the menu, edit your base station in AU, click the Network button, and then click the plusbutton beneath the Port Settings list.)
One of the main ways that server administrators configure their mail servers to block spam is via real-time blackhole lists, or RBLs. The basic idea is that as soon as a mail server is identified as sending spam, its IP address is added to an RBL. When receiving messages, other mail servers consult RBLs to see if the sending server is known to relay spam, and if so, the receiving server rejects the message.
In general, RBLs are good, since they reduce the impact of spam on the Internet immeasurably. However, if your IP address is on an RBL when you set up your mail server, the mail your server sends may be rejected or even dropped silently without being delivered. So, it’s important to check the RBLs before you start to make sure your IP address isn’t on such a list.
(Why would this happen? Depending on how your ISP sets things up, other clients may share the same IP address as you, or if you’ve gotten the IP address recently, it’s possible the previous user’s mail server had been taken over and used to send spam, landing the IP address on an RBL.)
Luckily, there are multiple services that will check a long list of different RBLs for your IP address. A popular checking site is (Figure 2); you might also try the  of The Anti-Abuse Project.
Mail is critical for most businesses, and even many home users live and die by their email (which will be stored on the server for those using IMAP), so it’s not only essential that your mail server be up and running 24/7, it’s imperative that you back it up constantly and be able to restore it quickly in the event of a disk failure or other catastrophe.
Chapter 12, Backup, is devoted to the topic of backups, so I won’t say much about it here, but before you bring your mail server online, jump forward to that chapter and make sure you’re backing up appropriately. I realize that’s not possible for those reading this chapter as a TidBITS article, but the practical upshot is that I recommend using Time Machine at a minimum; enhancing your backup strategy with a bootable duplicate and an offsite backup would also be good.
Once all the tasks listed in the previous topic are taken care of (and I mean all), it’s time to start working in the Server app.
The first step is to make sure you have an SSL certificate associated with various mail services, since it will be used to encrypt your logins. Luckily, since you configured push notifications back in, in Chapter 3, this is just a quick confirmation step.
In the Server app, click Certificates in the Server section of the sidebar. Under Settings, the Secure Services Using pop-up menu should be set to your server’s host name, indicating that the self-signed certificate you created when enabling push notifications is being used to secure all services.
To verify that it’s being used for mail in particular, click that menu and choose Custom. In the dialog that slides down, you should see two Mail entries (Figure 3). Click OK.
Now let’s configure the mail service. Click Mail in the Services section of the sidebar (Figure 4). Although there are only six controls, plus the main ON button, some of them require a fair amount of explanation.
The first setting is the domain for which you’re going to provide mail services. This is not the fake domain we’ve been working with all along, but the real domain that you configured at your DNS provider earlier in this chapter. Since OS X Server defaults to the domain in the server’s host name, you’ll need to change it.
Click the Edit button to the right of Provide Mail For, enter the desired domain in place of the fake one, and if you want to provide mail for additional virtual domains (which would require additional DNS setup at your DNS host), click the plusbutton and enter them as well. Click OK.
Each user account that you’ve set up on your server has a short name, and you’ll end up with an email address for each short name at each domain. For example, if you have accounts for
emerald, those accounts will be able to receive email at
firstname.lastname@example.org. And if you’d set up a virtual domain of
toxichellstew.com, those users could also receive mail at
Your users must authenticate whenever they try to exchange mail with your server, both incoming and outgoing. OS X Server supports a wide variety of authentication protocols, including Kerberos, Digest (CRAM-MD5), Cleartext, APOP (for POP only), and Digest-MD5, and a number of sources for that authentication information, including Open Directory, Active Directory, and the set of local users.
The easiest approach here is simply to stick with Automatic, which will query your Open Directory master for authentication information and allow any of the authentication protocols.
If you want to set particular authentication sources or protocols, click the Edit button next to Authentication. Make your choices using the Authenticate Using pop-up menu, and if you choose Custom, the protocol checkboxes (Figure 6). Click OK.
There’s nothing to do here, since you turned on push notifications long ago, in, in Chapter 3. What you actually did was to enable push notification of new email messages. What’s that mean?
Imagine that a new message arrives at the mail server for
email@example.com. With push notifications on, the mail server notifies that user’s email client that it should retrieve the message via IMAP—push notifications are just an alert to the client that it should check in.
The utility of push notifications, particularly for a mobile email client, is that it doesn’t have to check for new email unless there is some. This reduces network traffic, CPU usage on the device, and battery drain.
To reduce the damage done by badly set up (and thus easily compromised) mail servers, many ISPs block all SMTP traffic on their networks that doesn’t go through one of their servers. In other words, depending on your ISP, your mail server may not be able to send outgoing mail to any arbitrary server on the Internet at all.
The workaround is to relay all outgoing mail through a particular SMTP server that will accept your outgoing messages and send them on to their eventual destinations.
Even if you can send outgoing mail directly, you may not wish to, because relaying your mail through another server reduces the chance that your server will be blacklisted for some bad behavior.
There are three main approaches for relaying through another server:
To configure your mail relay, click the checkbox for Relay Outgoing Mail through ISP and in the dialog that appears, enter the host name of the other mail server, adding your username and password if SMTP authentication is necessary (Figure 7).
Many users don’t delete incoming email, which has both pros and cons. On the pro side, thinking about which messages to delete and which to keep takes time that could be spent more productively, and it can be useful to have a complete archive of old email. On the con side, that email can take up a lot of disk space, especially if your users tend to receive a lot of large attachments. Plus, for some businesses, archived mail can prove a legal liability if subpoenaed in a lawsuit.
OS X Server doesn’t have any way of enforcing email retention policies, but you can set the total amount of mail each user can keep. Select the Limit Mail To checkbox, and enter the mail store size, in megabytes. It defaults to 200 MB, but you can enter any number of megabytes you like.
Unless you’re using an email filtering service like McAfee’s, you’ll certainly want to turn on Server’s email filtering capabilities. To view these controls (Figure 8), in the Mail pane, click the Edit Filtering Settings button.
Apple packs a lot into a single dialog, so let’s go through each of these in turn:
Message containing viruses aren’t delivered, but are stored in
/var/virusmails, and notification is sent to the address configured for email alerts.
If the number of points accumulated by a message is greater than the SpamAssassin setting, SpamAssassin adds
***JUNK MAIL*** to the Subject of the message and delivers it. Users can then configure their email clients to filter such messages to a Junk Mail mailbox.
Once you’ve configured all the settings for the Mail service, click the ON button to enable the service. It may take a few minutes while Server starts all the various related services and downloads new virus definitions.
You can set up a mail client to check your work, but for the moment, it’s easier to launch Terminal on another computer and use telnet to see if your mail server is working. In Terminal, enter this command, changing the host name to that of your mail server:
telnet mail.pretendco.com 25
If it’s working, you should see a response like the following:
Zeus:~ adam$ telnet `mail.pretendco.com` 25
Connected to `mail.pretendco.com`.
Escape character is '^]'.
220-`mail.pretendco.com` ESMTP Postfix
To exit the conversation with your server, type
quit and press Return.
Apple has exposed relatively few settings for Postfix, Dovecot, ClamAV, and SpamAssassin via Server’s graphical interface. As you might expect from mature Unix apps, there are many more settings you might want to configure, which must be done from the command line.
First off, there’s a useful command line invocation that will give you details of the status of each service. In Terminal on the server itself, enter this command:
sudo serveradmin fullstatus mail
Next, to see a full list of mail-related options, enter:
sudo serveradmin settings mail
To be clear, many of the default settings are fine and shouldn’t be messed with unnecessarily. If you’re going to change something, make a note of what the setting was before you change it, in case you break something and need to put it back. Here are a few settings that people commonly want to change.
If you don’t like SpamAssassin’s default
***JUNK MAIL*** Subject tag, you can change it with this command:
sudo serveradmin settings mail:postfix:spam_subject_tag = "***DIE SPAMMER DIE***"
By default, the administrator doesn’t receive an email notification when ClamAV catches an email containing a virus-infected attachment. To enable this option and configure the address that will receive notification, use these commands, changing the address as desired:
sudo serveradmin settings mail:postfix:virus_notify_admin = yes
sudo serveradmin settings mail:postfix:virus_notify_admin_email = "firstname.lastname@example.org"
I find a lot of Mac environments want to accept email messages of pretty much any size rather than reject messages with too-large attachments. By default, message size limits are enabled. To disable them, use this command:
sudo serveradmin settings mail:postfix:message_size_limit_enabled = no
Even better, set a new limit (in bytes)—the default is 10 MB and this command sets it to 50 MB:
sudo serveradmin settings mail:postfix:message_size_limit = 52428800
Look through the rest of the settings that are listed when you enter
sudo serveradmin settings mail and see if any pique your curiosity, but again, don’t go changing things willy-nilly, or you’ll likely break your mail server. If you’re not certain what a particular setting does, run a Web search on it and see what you can find. Apple has documented many of them in various spots, and other users may have discussed the setting as well.
Now that you have mail services running in OS X Server, it’s time to set up the client side, which is straightforward. Although the specifics vary a little by app, I’m going to look at setting up the Mail client that comes with OS X. The same types of settings—IMAP and SMTP server names, and authentication credentials—apply to all mail clients, whether on Windows or in iOS.
To set up Mail for yourself (the steps will be the same for other users), follow these steps:
Read More: Chapter 12 |  |  |  |  |  |  |  |  |  |  |  |  |  |