Postfix, Sender Address Verification, and Spammers

matthew's picture

EDIT: This blog entry is quite old, in the world of Postfix. It's easy to find RPMs for 2.0 experimental releases now, and the word "experimental" (as of August 2003) is starting to sound a little funny for what is becoming the mainstream release people are using day-to-day. If something doesn't work for you, I suggest you hit the Postfix home page and see if you can figure out what's up from the mailing list archives.


Yesterday evening (Saturday night. There are a whole lot more fun things to do on a Saturday night than this!) I upgraded the Postfix install on my firewall at work. The primary purpose of my upgrade was to enable "Sender Address Verification". This stuff is pretty cool. Although the sample-smtpd.cf has wonderful examples of how to use this, I found other information on the 'net totally lacking.

Now, some important information before you get excited about Sender Address Verification:

My environment

  1. postfix-2.0.9-20030424. This is a "pre-release" or "experimenal" version of Postfix. That means you're not going to currently get this functionality using some pre-packaged binary rpm or something.
  2. db2/3/4-devel. This build of Postfix requires db2,3, or 4. So if you don't have it installed, you're not going to be able to build and install Postfix.
  3. pcre.  This is the "Perl Compatible Regular Expressions" library.  They tell you which version of pcre is the minimum for building Postfix on the postfix download page.  I would have a lot of trouble handling all the strange addresses I use without the pcre package; quickly, pcre is becoming a requirement for using Postfix. Postfix has some standard regular expression functionality by itself, and I think that you can build it without pcre, but I didn't try.
  4. gcc and other building tools. If you've never compiled anything before, you're probably out of your depth at this point.
  5. A TEST SYSTEM. Yes, please, try this out on a test system before you get down and dirty with your production firewall. I did the install on two test systems two days before rolling it out on a production server, to make sure it works.

The basic process itself is pretty darn easy. After downloading postfix-2.0.9-20030424 (or whatever the current experimental release is), rather than the usual "./configure; make; make install" three-step you'll find on most packages, Postfix follows its own standard.

The Upgrading Postfix Four-step

  1. tar xzvf postfix-2.0.9-20030424.tar.gz
  2. cd postfix-2.0.9-20030424
  3. make
  4. sudo make upgrade

Note that I use, and heartily recommend that everybody else does, too, "sudo" for all my root-privilege stuff. If you're doing all the above as root, leave off the word "sudo" from the last step.

It's unfortunate there's no RPM for the experimental releases, but Postfix is pretty good about figuring out how your previous version is installed (by looking at your old .cf files) and putting its stuff there. One thing the "make upgrade" doesn't do is copy over all the sample-foo.cf files. I heartily recommend figuring out where those sample files live on your system. On my FreeBSD test system, they live in /usr/local/etc/postfix/. On the RedHat 8 firewall, they live in /usr/share/doc/postfix-1.1.11/samples/.

On the FreeBSD test system, the install was flawless. Postfix worked exactly the same after "make upgrade" as it did before. Of course, that system runs barnson.org and is a slightly less complex install than the Red Hat 8 firewall at work. On my OpenBSD firewall at home, the results were similarly flawless.

However, when I reached my RedHat 8 install on the firewall, I immediately noticed some errors.

Problems with Postfix's "make upgrade" on a complex Red Hat 8 firewall

  • Postfix has no idea if you are running one instance of postfix, or more than one. We have two copies running at all times for historical reasons: one whos configuration files live in /etc/postfix, and the other lives in /etc/postfix-out. The postfix-out main.cf and master.cf were not updated with the change. I heartily recommend that you redirect output (stdout, put ">somefile.txt" at the end of the make upgrade line) from make upgrade to a file so that you can see what changes it has made (it's quite verbose) and make the same changes to your secondary install.
  • Postfix has changed the way it handles real-time blackhole lists. Instead of defining a "maps_rbl_domains" parameter and then calling "reject_maps_rbl" from within smtpd_recipient_restrictions, you define the rbls within smtpd_recipient_restrictions (or wherever you use them) itself. This, I think, can provide a great deal of flexibility with which real-time blackhole lists you use at different parts of your config file.

The reason we were running two Postfix instances on one box was because we were running Spam Assassin on mail, and didn't want mail leaving our network with spamassassin headers attached. We've since reduced that to just Anomy, and rely on Bayesian filters for those that want them within the firewall. It works better, and the firewall isn't pegged at 100% CPU usage 70% of the time anymore. So I turned off the secondary instance in my /etc/init.d/postfix script, and also added the localhost and internal interfaces to postfix's main.cf. I'll still have to see how that works.

OK, so you've read all this way, and you're probably wondering about all this Sender Address Verification stuff. It's really quite simple. In your "smtpd_recipient_restrictions" section, just add this one line:

reject_unverified_sender,

somewhere before your "permit" at the end of the section. Reload postfix, and try this sample mail session. Any line that begins with a number is a response from the remote server. Any line that begins with an all-caps command is what you type. Start by telnetting to port 25 on the remote mail server.

[matthew@localhost matthew]$ telnet some.internet.host 25
Trying 123.45.67.89...
Connected to some.internet.host.
Escape character is '^]'.
220 220 some.internet.host ESMTP Postfix
HELO barnson.org
250 bubba.aib.com
MAIL FROM: 

250 Ok
RCPT TO: 
550 : Sender address rejected:
undeliverable address: host mail.barnson.org[209.237.255.54]
said: 550 : User unknown in
local recipient table (in reply to RCPT TO command)
RSET
250 Ok
QUIT
221 Bye
Connection closed by foreign host.

In the following list, I use the terms "client", "server", and "verification host". The client is the machine attempting to send mail to your Postfix server. The server is your Postfix server. The Verification Host is the first available MX (Mail exchanger) host based on the MAIL FROM: sent by the client. The basic process that just happened was this:

How Sender Address Validation works:

  1. Client sends HELO, MAIL FROM, and RCPT TO commands.
  2. On RCPT TO, Postfix server makes a connection to the MX host(s) for the domain listed in MAIL FROM by the client.
  3. If Postfix is able to connect to that MX (if you're talking about spammers, that's always a dubious possibility at best), it then sends its own MAIL FROM: declaring that it is postmaster@some.internet.host, and puts a RCPT TO: of the MAIL FROM: address of the client attempting to mail to itself.
  4. If the server responds with anything but an OK, Postfix sends a RSET and QUIT to the verification host, and then sends a rejection to the client saying "undeliverable address" and the response from the verification host.
  5. If the verification host replies with OK, Postfix sends a RSET and QUIT and then sends an OK to the client. Alternatively, if you have additional rules after reject_unverified_sender, you can configure Postfix to default to DUNNO (meaning "passed this rule, but go on to the next") rather than sending an OK or a REJECT.

So far, reject_unverified_sender is really mostly catching the "corner cases" for me, rather than the majority of spam. However, it has almost eliminated a class of messages that occasionally kill our mail server: dictionary attacks. Often a spammer will send thousands of messages to random addresses in our domain in hopes that someone will read their pitch. Well, our Postfix gateway is configured to blindly relay any address in our domain on back to our Groupwise server behind the firewall. Yeah, I know, it's not the best thing in the world, but at the moment it's easier to do that than to enter several thousand email addresses by hand into the "virtual" table for Postfix. Nasty getting those out of Groupwise. Anyway, if a spammer dictionary-attacks us using an invalid return address, when Groupwise bounces the message, those bounces bounce because the return address won't work. Basically, we're just checking to see that there's a real, live email address listed in the return. This prevents those thousands of "double bounces" from ever getting onto our network in the first place, and ending up in my inbox as postmaster of the domain. And that's really what this is all about: lowering my frustration level with double-bounces.

We're in the process of moving our organization from an ancient install of Groupwise 5.0 to Cyrus Mail. Yeah, we're going to lose a little functionality here and there, but my hope is that it's totally work-around-able for our users. Once that migration is complete, we can actually easily streamline this process by using reject_unverified_recipient, which makes a connection to the ultimate destination SMTP server, validates that a mail address exists and is deliverable, and does pretty much the same thing otherwise. This would still provide us with protection for our internal mail server by using an inbound mail gateway, yet dramatically reduce our double-bounces. Alternatively, we can just maintain a list of users in two places (one on the firewall), but I'm leery of storing any non-essential user names on a firewall, you know?

Eventually, I think the spammers will get wise to thise protection and begin always using valid email addresses as the return address of their promotions. Unfortunately, I think it's most likely that they will use known-good mailing addresses as the forged source of their unsolicited commercial email. However, this will catch those who are slow to catch on, and at least force fraud to be more quickly exposed. If I were your average mail administrator, I'd certainly want to keep an eye on RCPT TO: requests to my mail server, and if I see too many to a certain individual, temporarily disable their mail account until the attack is over.

OK, now the last thing. I'd planned on getting this implemented by the time I wrote this article, but I just haven't had the time yet. The final phase in my plan for anti-spam is to create a temporary rejection table called "five-fifties". This Postfix table would store the IP addresses of clients who do not honor 55x rejection error codes and make multiple mail delivery attempts after receiving a 550/554/whatever. I realize that it's not really going to be much of a help defending against spam (since I'm already rejecting them with a 55x anyway), but it would be a big help in tracking connection attempts from abusive IP addresses and setting up another daemon to automatically add repeat offenders to an iptables blacklist. Looks like you'll have to wait for another article to read more about that though, after I have implemented it!

Hope this helps!

***NOTE: Please be aware that I'm still getting used to my chosen blogging software, Drupal, at this point. So there may be strange typographical issues here and there. Please let me know via comment if you see something that needs fixing.***

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
matthew's picture

There now are better ways

There now are better ways to accomplish this purpose, without doing things like ending up with nasty mailing list bounces. The solution I'm working on (gotta migrate all my DNS to my own machines first) is SPF, or Sender Policy Framework. It's a simple opt-in procedure. If a sending domain publishes SPF DNS records, your mail server checks them to see if the machine from which the mail is being sent is listed in their SPF record. If it's not, you reject it.

It's kind of like an automated sorting machine that checks postmarks on envelopes. If you get a message claiming to be from me, with a return address of Tooele, Utah, and the postmark is from Hong Kong, you can tell something's amiss and that you probably shouldn't believe the message is actually from me. Of course, this does pose a problem for "road warriors" that may be sending mail from anywhere on the globe with a return address of their home office. But there are technical solutions to "opt out" of the SPF system for your domain(s), and if you choose not to participate, it's the same as if you'd opted out.

It's an idea who's time has come. The first step in stopping the flood of spam that's been drowning us for years is to force spammers to be accountable. If they choose to send spam, that's their right, but they must do so from a legitimate mailing address. As the number of SPF-using domains grows, spammers who wish to fraudulently abuse sending domains will have to restrict themselves to a smaller and smaller subset of domains that don't publish SPF records. Eventually, all "useful" mail for an organization will only come from SPF-listed members, and they'll choose to reject mail that isn't from other SPF-using organizations.

At that point, the spammers must either choose to peddle their wares using legitimate lists and a legitimate return address, or face the consequences of their fraud not getting them the results they want.

That'll hit them in the pocketbook, which is exactly where the pain needs to be felt. I don't object to advertisers sending bulk email, but since I run my own post office, it's my choice whether to accept it, mark all such soliciations "return to sender", or simply refuse to accept the mail at all and force the sending domain to deal with it rather than me. That's a good thing.

--
Matthew P. Barnson

--
Matthew P. Barnson