Postfix in a FreeBSD jail

matthew's picture

I recently ran into problems with postfix running inside of a FreeBSD jail. It would constantly report "file too large" errors, and other errors in that vein. Since there appears to be no documentation for FreeBSD "newbies" on fixing this problem, it seems like most people default back to sendmail or exim, rather than using the excellent Postfix MTA on their virtual file systems.

I love postfix, so that is a situation I'd like to change! Here are the directions on how to get postfix working inside a FreeBSD "jail" system.

Well, the basic problem with Postfix in a FreeBSD 4.8 jail is the disconnect in the setrlimit() call in Postfix versus the kernel. You end up with spurious "file too large" errors when sending and receiving messages, which just sucks.

There's a patch at http://www.kozubik.com/published/misc/postfix_patch_fbsd45jail

Here's how I do this without using the above patch:

IMPORTANT NOTE: A generous reader left a clarification in a comment below this entry that may be more useful for you. My procedure works fine because you're running "make clean" inside the postfix work directory -- using Postfix's built-in Makefile -- and thus not destroying the source file you're going to edit. However, running "make clean" from the /usr/ports/mail/postfix-current/ directory would blow away your work/ directory, nullifying the effort you're putting into patching :)

1. cd usr/ports/mail/postfix-current

2. Run "make" -- I normally just leave it alone and let Postfix go ahead and build itself. Not a big deal.

3. cd work/postfix-(version)/src/util

4. using your favorite text editor, open "file_limit.c", and go to the SECOND instance of this line:

struct rlimit rlim;

(This is the one followed by "rlim.rlim_cur = rlim.rlim_max = limit;" -- if you don't see that, you're at the wrong one)

5. Paste this line immediately after the "struct rlimit rlim" declaration, then save and exit:

limit = RLIM_INFINITY;

6. Now cd /usr/ports/mail/postfix-current/work/postfix-(version) again.

7. run "make clean"

8. run "make"

9. cd ../.. (or /usr/ports/mail/postfix-current)

10. If you've already installed postfix, then run "postfix stop" and "make deinstall". Otherwise, just run "make" and then "make install" and YOU ARE DONE!

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Harumph!

Postfix sucks.

Why not use Exim instead? It works perfectly in a FreeBSD jail.

true: exim is good, but false: postfix does not suck

we acquired a lot of knowledge of tuning postfix and it served us very well in the last years. several hundreds of thousand mails a day are no problem. so why change? exim has certain drawbacks i do not like, as it does not generate RFC compliant DSNs when bouncing mails... but that could have changed since then...

Um... Houston, we have a problem.

Make clean will kill the work folder.

I think you'll want to do something like this:

    1. cd to /usr/ports/mail/postfix


    2. make extract


    3. <instructions for applying patch here>


    4. make install (or make install clean).

matthew's picture

Very cool, thanks for the clarification!

I appreciate your clarification; this document is about due for a rewrite, anyway, so I'l update the main doc with the revised instructions.

I wonder why it worked for me this way?

--
Matthew P. Barnson

--
Matthew P. Barnson

matthew's picture

I figured out why it worked for me...

I figured out why it works for me. I'm not running "make clean" from /usr/ports/mail/postfix-current/ -- I'm running it from /usr/ports/mail/postfix-current/work/postfix-(version)/ . It's postfix's own "make clean" that I'm running, which won't alter source files like running "make clean" using the port's Makefile.

Either way works satisfactorily, as far as I can tell after rebuilding once again :)

--
Matthew P. Barnson

--
Matthew P. Barnson

Thanks

This is just what I needed. But, what are the potential risks of overriding the limit? And why doesn't the setrlimit function work right in the jail. I have not been able to find any docs about that.

matthew's picture

Jailed stuff that doesn't work

There are several resource allocation calls that don't work in a FreeBSD jail. One day, I'd like to make a complete list of them, but I'm only aware of the ones that have hit me:

  • Shared memory is unavailable in secure jail implementations. Apparently, you can turn it on but then your jails can end up overwriting one another's memory or something :) Kernel developers appear to be working on this for the FreeBSD 5.X series, but in 4.X, it's not working.
  • ICMP (Ping & Traceroute) doesn't work from a jail.
  • You can't make a kvm_open call, which fooses "top". I think there's an alternative mode you can compile certain versions of "top" to that read the process table differently. Curiously, a "ps aux" works just fine, but only shows children of the jailed process, as it is supposed to.
  • Each jail can only handle one IP address.

With those limitations in mind, jails can still be really useful. This site runs in such a jail on a virtual hosting provider, and I can still do a lot of useful stuff, and it doesn't feel crippled at all 99.99% of the time.

As far as ramifications for Postfix, I think the only problem with lack of setrlimit() is that you could possibly get DOS'd on your Postfix process by someone sending an arbitrarily long message. My solution is ulimit the process to not take more than 50MB of RAM ("ulimit -m 51200"). Then the process handling incoming messages (postfix's "smtpd") would run up hard against that ulimit and bomb out on incoming messages exceeding that threshold. Since SMTPD only starts when postfix's "master" process detects a connection on port 25, it dying off running up against the memory ulimit will just show up as a blip on your syslog with something about "exhausted memory".

The default ulimit for most processes on freebsd is unlimited, so it's probably wise to ulimit any of your network-available daemons you think may grow too large. However, it's rather tedious editing all the init scripts. In order to make Postfix do it, I had to "rm /usr/local/etc/rc.d/postfix.sh", because that was just a symlink to /usr/local/sbin/postfix (an ELF binary), and I then created a shell script that did the ulimit for me and then I named that "postfix.sh". It does the job nicely, and I'm not concerned about the remote possibility of someone crafting a large DOS-oriented message to try to kill my virtual private server.

It works well. It would be nice if postfix's configure script detected setrlimit()'s broken-ness in a FreeBSD jail and automatically ulimit-ted smtpd processes, though.

--
Matthew P. Barnson

--
Matthew P. Barnson

Does this still work?

I tried this patch and i'm still getting tons of "file too large" errors in my maillog.

matthew's picture

Check this page..

You may want to check out this page at jvds.com about the same problem. It's solved it for most people; the biggest problem seems to be people making these changes in postfix-current when their currently installed postfix is just /usr/ports/mail/postfix.

--
Matthew P. Barnson

--
Matthew P. Barnson

works

thanks for this write up.

This patch is *severly* br0ken!

There is one seriously flawed description of the problem which says:

http://lists.freebsd.org/pipermail/freebsd-questions/2004-August/056599.html

In normal operation, Postfix makes a system call to check to see if it can create a file of a certain size. Inside a jail, this call will not succeed as per the very design of jails.
[...]
This patch probably hasn't made it into the port because it completely bypasses a moderately important check.

This description is wrong. The code in question SETS the message file size limit instead of asking for the max available size.

The patch is no less than catastrophic. It allows everyone to completely use up all free queue space in the queue with a single email message.

If people prefer to use some inferior MTA then so be it.

EDIT by matthew: Linked.

matthew's picture

True... but ulimit?

I think I figured out a solution which bypasses the need for patching code in a FreeBSD jail. It's absurdly simple; I'll update my main blog posting once I've tested it.

--
Matthew P. Barnson

--
Matthew P. Barnson

The problem does not reproduc

The problem does not reproduce with FreeBSD 5.3.

I set up a jail on FreeBSD 5.3, installed Postfix 2.2.3, and it
accepted local mail submission without making any changes to
Postfix configuration files.

No errors at all about files to large, and no need for patches
that turn off the message size limit.

Wietse

May 22 23:15:27 tail postfix/master[22821]: daemon started -- version 2.2.3, configuration /etc/postfix
May 22 23:15:34 tail postfix/pickup[22822]: AEB3D16B0B11: uid=0 from=<root>
May 22 23:15:34 tail postfix/cleanup[22828]: AEB3D16B0B11: message-id=<20050522231534.AEB3D16B0B11@wzv.porcupine.org>
May 22 23:15:34 tail postfix/qmgr[22823]: AEB3D16B0B11: from=<root@wzv.porcupine.org>, size=328, nrcpt=1 (queue active)
May 22 23:15:34 tail postfix/local[22830]: AEB3D16B0B11: to=<wietse@wzv.porcupine.org>, orig_to=<wietse>, relay=local, delay=0, status=sent (delivered to mailbox)
May 22 23:15:34 tail postfix/qmgr[22823]: AEB3D16B0B11: removed