2014-08-01 Jørgen Thomsen, jth@jth.net
Valid XHTML 1.0!

Virtual domains using GNU pop3d and Postfix 

Belarusian translation  Deutsche Übersetzung  Tlumaczenie polskie Traduction francais 

GNU pop3d 0.10.2

The GNU pop3d 0.10.2 is ready. You may download it here GNU pop3d 0.10.2


CHANGES / HISTORY

22/Mar/2014 0.10.2 Jørgen Thomsen bugs@jth.net, http://jth.net/virtual.html
	New features
	------------
	IPv6 support with configuration --disable--ipv6 and at runtime --ipv6

	POP3S support with --enable-pop3s
		Will install working configuration of xinetd using stunnel incl. sample certificate and key
		Prerequisite: xinetd and stunnel installed
        Support for encrypted (by C crypt) passwords in virtual domains (no APOP then)

	Daemon start support extended --enable-service=SERVICE
		now supporting configuration for sysvinit, init, event.d, systemd and single user inetd, xinetd
                (probably not perfect :) Check your installation )

	Installation root can be specified during configuration --enable-rootdir=DIR as well as
	by make install ROOT=DIR

	uninstall feature added

	pop3client.pl program for simple testing and stress testing a pop3 server
	mkpwd.pl for making and/or encrypting passwords for virtual domains compatible with glibc crypt()

	kill -SIGUSR2  will display a few status variables in the syslog
	Daemon: display count and pids of active children, value of flag determining termination, as well as number of sessions till now.
	Child: display time used and commands for user connected from IP-address

	Changes
	-------
	extra.c/pop3_syslog may now log to STDERR (debug=0x1yy).
	Individual syslog priorities kept in pop3_syslog.
	More logging via pop3_syslog.
	Reduced use of library routines in pop3_syslog for improving stability.
	Obsolete gethostbyname and gethostbyaddr replaced by getaddrinfo and getnameinfo
	inet_ntoa replaced by inet_ntop
	A 1 second delay for returning BAD LOGIN to deter robots checking for weak passwords.

	Help texts cleaned up and improved.

	Improved installation script
	- handling additional daemon start systems
	- configuration of pop3s using stunnel + xinetd
	- specification of installation root directory
	- IPv6 support
	- firewall firewalld supported
	- uninstall feature
	Please, observe that testing of installation has only been done on Linux Fedora 12, 15, 18, 20, so it may not be perfect in all cases.
	As GNU-pop3d is only one program without any required configuration files it can manually be placed in /usr/local/sbin/gnu-pop3d
	and entered into the startup script of your system services

	MD5 code updated (Gray Watson) adding support for big/little endian
	Most recent GNU config.guess and config.sub installed.

	Bugs fixed
	----------
	More robust child management (busy server, DoS attacks).
	Recent changes in the crypt or strcmp functions apparently caused an infinite loop during login in certain cases.
	Login using stolen encrypted password in virtual domains prevented.


07/Feb/2011 0.9.13 Jørgen Thomsen jth@jth.net, http://jth.net
        Syslog logging from the pop3_signal signal handler routine caused hanging processes especially during DoS attacks or
        by many simultaneous clients. Partly fixed (may still occur when using this bit (08) in the debug mask)

        Bugfix: certain login methods did not work (David B. Cortarello)

        The global child counter child_procs changed from int to 'volatile sig_atomic_t' to decrease the possibility of
        it not being updated properly.
        The counter was not always decremented so the program used only one instance, when max child
        processes were incorrectly reached.

        Most recent GNU config.guess and config.sub installed.

        Makefile.in and configure.in improved and support for detection of Atom processors added and more support for
        service installation (/etc/init)

        The RFC directory deleted as the RFCs are readily available on the Internet

26/Apr/2010 0.9.12 Jørgen Thomsen jth@jth.net, http://jth.net
	Possible double free of the mailbox name variable in special error situations e.g at two simultaneous logins to
	the same mailbox.

	config.guess and config.sub missing from the distribution

	Added a start message to the syslog

18/Apr/2010 0.9.11 Jørgen Thomsen jth@jth.net, http://jth.net
        The program is maintained on a Linux Fedora platform with no testing possible on other platforms, but
        with an attempt to avoid Linux specific features.

        Improvements to the configure script including 64 bit CPU support.

        The Nagle (TCP No Delay) algorithm has been disabled causing a large improvement in speed.

        SO_REUSEADDR added to daemon socket for immediate reuse of socket after a restart.

        Signal handling improved to be more portable (sigaction) and unsafe calls removed from the signal handler. Termination
        after a signal should now be much better not leaving the mailbox.lock files around.

        *bug* the sock variable in the child was always -1 and not the actual socket (sock2) so pop3_signal in a child did not close the socket.
        Removed sync() calls as e.g. copying large files to an USB disk would cause a hang until all data were written to the disk (several minutes)

        Conditional compilation of debugging statements removed as the overhead is so small.
        The debugging parameter has been changed to a 2 hex digit bit mask (children, daemon) and additional
        debugging statements have been inserted.

        MAXHOSTNAMELEN increased to 128 due to IDNA domain names

        gnu-pop3d file for /etc/event.d included

        man page updated

18/Jul/2007 0.9.10 Jørgen Thomsen jth@jth.net, http://jth.net
        fcntl lock on mailbox changed to write lock
        sync() always called at unlock
        read data from client when timeout enabled (select) now checks for broken pipe too causing immediate exit,
        if the client disappears suddenly.
        When mailbox was copied to new mailbox, some MTAs (e,g, postfix) might continue writing to the old mailbox.
        Now the empty old mailbox is kept as mailbox.old and merged with the new one if not empty at next run.

24/Aug/2003 0.9.9-5 Jørgen Thomsen jth@jth.net, http://jth.net
        The test for header lines wasn't strict enough Status: and X-Status: considered equal (Jem Berkes)
        Command lines were not assembled to one line if received as separate TCP-packets (Nicolas Gregoire)

18/Jan/2003 0.9.9-4 Jørgen Thomsen jth@jth.net, http://jth.net
        Support for Postfix X-Original-To: header line for Message Disposition Notification.
        UIDL computation will no longer include X- headerlines, as they may be added by some mailreaders.
        '%' is allowed as delimiter instead of @ in a virtual username (email address)
        Minor internal changes.
        Additional support for the virtdomain.pl by the author (how to find the mailbox)
18/Oct/2002 0.9.9-3 Jørgen Thomsen jth@jth.net, http://jth.net/
	Corrected an error when a bad mailbox did not contain 'From ' as first line.
05/Sep/2002 0.9.9-2 Jørgen Thomsen jth@jth.net, http://jth.net/
        The fix for disk full prevented the changed mailbox to be written to the .new file. It was retained unchanged
04/Sep/2002 0.9.9-1 Jørgen Thomsen jth@jth.net, http://jth.net/
        If the mailbox did not terminate by a line feed (if crash while writing a message into it)
        an infinite loop was occurring. If the mailbox is modified by GNU pop3d, it will be fixed.
        When the disk became full when writing the new mailbox it was not detected and the new mailbox was truncated
19/Aug/2002 0.9.9 Jørgen Thomsen jth@jth.net, http://jth.net/
        Compatibility with vm-pop3d passwd files introduced (not fully tested)
        Added graceful stop by catching the signals (SIGHUP and SIGTERM).
        The final touches and version number changed to 0.9.9
19/Oct/2001 Jørgen Thomsen jth@jth.net, http://jth.net/
        Added Message Disposition Notification (return receipt, RFC 2298)
        Added configure options to avoid editing config.h in most cases
        Changed the update of the mailbox from copying inside the file to creating a new file (mailbox.new)
        if needed and improved the copying inside the file. This improves the speed greatly especially for webmail programs.
        Introduced support for the classical Status: header line RFC 2076 (supported by e.q. Null Webmail
        http://nullwebmail.sourceforge.net/)

17/Oct/2001 Jørgen Thomsen jth@jth.net, http://jth.net/
        Added an old patch for better handling of children processes creation and death.
        Now child processes are created as needed and only the parent process is permanent.

   

VirtDomain 1.05

The web GUI interface is functional for both standard Postfix virtual domains and my flavour: jth virtual domains. It comes as a single Perl program, whcih is called from the commandline to create a virtual domain and from the webserver to administer it. Requires a recent release of Postfix VirtDomain 1.05

#!/usr/bin/perl -w
#**********************************************************************
#
# Utility to administer virtual domains in postfix (both the standard
# ones and those defined in http://jth.net/virtual.html)
#
# This program should reside in /var/www/cgi-bin/virtdomain.pl with global
# read and execute rights (chmod o+rx /var/www/cgi-bin/virtdomain.pl).
#
# Remember to define $main::postfixadmin below !
#
# The first argument to the program determines its function:

# CREATE:  call from commandline as root: create virtual domain by updating
#          /etc/postfix/virtual and virtual_regexp
#          and creating /etc/virtual/domain/passwd and /var/spool/virtual/domain
#          CREATE <domain name> <real user administering the domain>
#                               <passwd> [postfix]

# DELETE:  call from commandline as root: delete virtual domain by updating
#          /etc/postfix/virtual and virtual_regexp
#          and deleting /etc/virtual/domain/passwd and /var/spool/virtual/domain as
#          well as .forward files in the administrators homedir
#          DELETE <domain name> <real user administering the domain> <passwd>

# <blank>
# MAILBOUNCE
#          if called from postfix as mail to user@domain the email on STDIN will be processed
#          according to the setup of the user in /etc/virtual/domain/passwd.
#          MAILBOUNCE: bounce email to unknown users otherwise store in postmaster mailbox

# <blank>  if called from postfix as mail to postmaster@domain or <postfixadmin+postmaster>
#          with the subject "*** Virtual domain update ***" and line 1 is trans=UPDATE....
#          a user will be created/updated/deleted
#          otherwise process email normally as above

# <blank>: if called from a web server displays webpages to update the user
#          base in a virtual domain e,g, http://jth.net/cgi-bin/virtdomain.pl

#
# As a basis the user postmaster must be defined for the domain.
# in the /etc/virtual/domain/passwd file. The user and group ids as
# well as homedir from this user is used for the virtual domain.
# For each virtual user the file homedir/.forward+user--domain is created.
# The web server should have read access to these files as well as
# the /etc/virtual/domain/passwd file.
# The GNU pop3d deamon as maintained by me is supporting this
# virtual domain implementation.
#
# Using this program to deliver the mail for jth style domains is not suitable for
# a high volume system, as loading the Perl interpreter
# has a fairly high initial CPU usage compared to this program (90% of total run time).
# However, using the provision for hooking in another program to deliver the mail
# may be useful. An awk utility may be used instead virtual.awk
#
# ©2002 Jørgen Thomsen jth@jth.net, http://jth.net/virtual.html
# 1.01  2002-08-22 Official release
# 1.02  2003-01-24 Support for new virtual_alias_map parameter and relay_domains behaviour
# 1.03  2003-04-26 Better language support and fix webmail URL in info
# 1.04  2003-05-30 Chars | and " were removed from forwards in input. Now user shall not enter |, but start commands with "
# 1.05  2004-08-06 Modifications specific to jth.net: /var/www/perl, body background changed, no CGI used
#
#**********************************************************************
   

Old but basically still valid

Changes to GNU pop3d 0.9.8 
(including some changes not related to virtual domains e.g. POP-before-SMTP)

Based on my own needs I have made a patch for GNU-pop3d vers. 0.9.8
I also listened to some of the requests on the GNU pop3d mailing list.
The patch may be applied also if you don't need the virtual domain stuff. 

Basicallly I took the virtual-patch by Jeremy Reed, made it more general and added my own GNU pop3d changes:

The search for a users mailbox is performed as

  1. search MAILSPOOLHOME (e.g. /home/user/Mail/mailbox)
  2. search _MAILDIR (e.g. /var/mail/user)
  3. search VIRTUAL_MAILDIR (e.g. /var/spool/virtual/domain/user)
  4. give up and use /dev/null

Patching GNU pop3d

In stead of everybody applying the patches I have done it.

  1. Download the patched source gnu-pop3d-0.9.11.tar.gz
  2. Create the config.h file by
    ./configure
    or
    ./configure --enable-ip-based-virtual
    (se also another call below)
  3. If needed/wanted define MAILSPOOLHOME (e.g. "/Mail/mailbox") and/or DEFAULT_DOMAIN in config.h
  4. Compile by executing make

The technical changes 

Implementing virtual domains in Postfix and GNU pop3d

The reason for not using the standard Postfix virtual domains are twofold:
1) this implementation focuses on virtual domains with each its own administrator
2) the following limitations (quote from the Postfix documentation (my emphasis)):
This agent was originally based on the Postfix local delivery agent. Modifications mainly consisted of removing code that either was not applicable or that was not safe in this context: aliases, ~user/.forward files, delivery to "|command" or to /file/name

The following shows one way of implementing a virtual domain, kvt.dk, in Postfix and GNU pop3d.
A real user kvt_dk has been defined on the system. The home directory of this user contains some data for the virtual domain kvt.dk.
Two users are defined at kvt.dk: abc@kvt.dk (virtual user only) and kvt@kvt.dk (real user kvt_dk at the mail server)
The implementation is based upon Redhat Linux 7.0
 

GNU pop3d

Jeremy Reeds original description of the virtual patch

The virtual patch assumes the following file structures:
/etc/virtual/<domain>/passwd
 for the definition of users, passwords and mailbox directory

In this example create the directory /etc/virtual/kvt.dk

The default mailbox for a virtual user is
/var/spool/virtual/<domain>/<user>

In this example create the directory /var/spool/virtual/kvt.dk

At my server the mail is stored in the user's homedir /home/<usr>/Mail/mailbox requiring the definition of MAILSPOOLHOME in config.h.
I also have only one IP-address and want my main domain "jth.net" to be automatically stripped from mail requests to GNU pop3d.

I generated the config.h file by the command

./configure --enable-64bit --enable-maildir -enable-default-domain=jth.net --enable-language=DA

Before compiling GNU pop3d I made sure the following constants were defined in config.h:

#define MAILSPOOLHOME "/Mail/mailbox"
#define USE_VIRTUAL 1
#define DEFAULT_DOMAIN "jth.net"

/etc/virtual/kvt.dk/passwd

Create this file with the following contents
kvt:kvtpasswd:601:100:kvtname:/home/kvt_dk/:
abc:abcpasswd:601:100:abcname:.:

601 is the userid of the kvt_dk user. 100 is the groupid (group mail at my server). The '.' homedir for abc is just to make pwck happy.
Mail for kvt@kvt.dk is delivered to /home/kvt_dk/Mail/mailbox (see below).
Mail for abc@kvt.dk is delivered to /var/spool/virtual/kvt.dk/abc

If you trust the kvt_dk user, then create the passwd file in the home directory of the user kvt_dk. and link it into the /etc/virtual/kvt.dk directory
ln -s /home/kvt_dk/passwd /etc/virtual/kvt.dk/passwd

Note, that there might be a security risk here, because GNU pop3d will execute with any userid specified in the passwd file. Whether this is a real threat remains to be analyzed.

The owner and permissions on the files and directories should be kvt_dk.mail and 700 e.g
chmod 700 /etc/virtual/kvt.dk /etc/virtual/kvt.dk/passwd /var/spool/virtual/kvt.dk
chown kvt_dk.100 /etc/virtual/kvt.dk /etc/virtual/kvt.dk/passwd /var/spool/virtual/kvt.dk

Postfix

This implementation may be more complex than other ones, but it allows different administrators of each virtual domain keeping everything in the home directory of the domain administrator.

/etc/postfix/main.cf

recipient_delimiter = +
home_mailbox = Mail/mailbox
allow_mail_to_commands = forward
allow_mail_to_files = forward

/home/kvt_dk

-rw-r--r--    1 kvt_dk   mail    12 Dec 10 02:52 .forward
-rw-r--r--    1 kvt_dk   mail    25 Dec 10 01:55 .forward+abc--kvt.dk
-rw-r--r--    1 kvt_dk   mail    20 Dec 10 02:53 .forward+kvt--kvt.dk

/home/kvt_dk/.forward

|"exit 67"

Bounce any message not for abc@kvt.dk or kvt@kvt.dk

/home/kvt_dk/.forward+abc--kvt.dk

|/usr/local/bin/virtual

Deliver all mail for abc@kvt.dk to the virtual domain mailbox for abc changing the Delivered-To: header line to the correct one.

/home/kvt_dk/.forward+kvt--kvt.dk

/home/kvt_dk/Mail/mailbox

Deliver all mail for kvt@kvt.dk to the mailbox of kvt_dk

/etc/postfix/virtual

kvt.dk                    anything
postmaster@kvt.dk         postmaster
webmaster@kvt.dk          jth

The real users: postmaster and jth must receive all mail for the postmaster and webmaster of the virtual domain

/etc/postfix/virtual_regexp

/^(.+)@(.+\.)?(kvt\.dk)$/ kvt_dk+$1--$2$3

This expression will cause postfix to deliver any mail with adresses x@kvt.dk and y@z.kvt.dk to the real user kvt_dk keeping the original envelope recipient as an extension to the new recipient address.

/usr/local/bin/virtual

awk  -F: -f /usr/local/bin/virtual.awk

/usr/local/bin/virtual.awk

# J. Thomsen 2002-04-11 Postfix utility
# /usr/local/bin/virtual.awk
#
# Used by Postfix for delivering mail to a GNU-pop3d virtual domain
# mailbox.
#
# Postfix will use this utility for delivering a message to the local
# mailbox rewriting the Delivered-To header line to the original recipient
# address.
# 2002-04-11 Fix 'From ' lines in msg body and add empty line to msgs to ensure the separation of messages
#

Deliver  to virtual domain mailbox e.g. abc@def.kvt.dk to /var/spool/virtual/kvt.dk/abc.
The Delivered-To: header is modified to contain the original envelope recipient address (after possible modification by aliases).