smtp + sql = more than it seems so (part 8)

last time, i showed how to add automatical filtering of mails. and i told you that i will show another way.

i happen to be long-time linux user. as such i really do like using it in console mode. i enjoy mutt as my mail reader. and procmail as my tool of choice for mail filtering.

i dont really think that there are many people like me, but since it's my howto i will show how to add procmail support to our sql-based-exim.

additionally, while making it work i found a really interesting bug in procmail, so describing it will have some educational purpose 🙂

first, we will of course need procmail:

=> apt-get install procmail

that was easy.

now, we have to find procmail router in exim4.conf.template.

it looks like this:

procmail:
  debug_print = "R: procmail for $local_part@$domain"
  driver = accept
  domains = +local_domains
  check_local_user
  transport = procmail_pipe
  # emulate OR with "if exists"-expansion
  require_files = ${local_part}:\
                  ${if exists{/etc/procmailrc}\
                    {/etc/procmailrc}{${home}/.procmailrc}}:\
                  +/usr/bin/procmail
  no_verify
  no_expn

cool. now, let's modify it a bit to make it work with sql-based virtual accounts.

in home directory procmailrc is saved as hidden file (.procmailrc). since our users are virtual there is no need to make it hidden. so let's just drop the dot from beginning.

fixed version of procmail router looks like this:

procmail:
    debug_print = "R: procmail for $local_part@$domain"
    driver = accept
    domains = +local_domains
    transport = procmail_pipe
    # emulate OR WITH "if exists"-expansion
    require_files = ${lookup pgsql {SELECT get_account_homedir('${local_part}', '${domain}')}}/procmailrc:+/usr/bin/procmail
    no_verify
    no_expn

as you can see i reused get_account_homedir function, and made exim use procmailrc file stored directly in this directory.

now, i need to modify “procmail_pipe" transport.

our new, improved version will look like this:

procmail_pipe:
  debug_print = "T: procmail_pipe for $local_part@$domain"
  driver = pipe
  path = "/bin:/usr/bin:/usr/local/bin"
  USER      = ${lookup pgsql{SELECT get_account_uid('${local_part}', '${domain}')}}
  GROUP     = ${lookup pgsql{SELECT get_account_gid('${local_part}', '${domain}')}}
  command = procmail "${lookup pgsql{SELECT get_account_homedir('${local_part}', '${domain}')}}/procmailrc"
  return_path_add
  delivery_date_add
  envelope_to_add

that would be it. so let's save it, restart exim, and add some basic procmailrc to our test user:

=> cat /mails/exim.depesz/depesz/procmailrc
SHELL=/bin/bash
MAILDIR=/mails/exim.depesz/depesz/maildir
LOGFILE=/mails/exim.depesz/depesz/procmail.log
UMASK=027
INBOX=$MAILDIR
 
:0
* ^From:.*@exim
.internal/
 
:0
$INBOX/

how does it work?

let's check with 2 simple mails. first one from not-filtered domain:

=> perl -MMail::Sender -e '$m=new Mail::Sender {smtp=>"192.168.0.101",from=>"root\@localhost"};$m->MailMsg({to=>"depesz\@exim.depesz",msg=>"test"})'

logs:

2008-03-01 21:44:57 1JVZVm-0004Ch-7J H=(localhost) [192.168.0.101] Warning: SPAM-score: 37
2008-03-01 21:44:57 1JVZVm-0004Ch-7J <= root@localhost H=(localhost) [192.168.0.101] P=esmtp S=516 id=20080301_214454_090903.root@localhost
2008-03-01 21:44:57 1JVZVm-0004Ch-7J => depesz <depesz@exim.depesz> R=procmail T=procmail_pipe
2008-03-01 21:44:57 1JVZVm-0004Ch-7J Completed

and procmail.log show:

From root@localhost Sat Mar 01 21:44:57 2008
 Subject: <No subject>
  Folder: /mails/exim.depesz/depesz/maildir/new/1204407897.16172_2.xxx      626

needless to say, the file is really there:

=> ls -la /mails/exim.depesz/depesz/maildir/new/1204407897.16172_2.xxx
-rw-r----- 1 10004 10002 626 2008-03-01 21:44 /mails/exim.depesz/depesz/maildir/new/1204407897.16172_2.xxx

and what if i'll send mail from some @exim domain?

=> perl -MMail::Sender -e '$m=new Mail::Sender {smtp=>"192.168.0.101",from=>"test\@exim.depesz"};$m->MailMsg({to=>"depesz\@exim.depesz",msg=>"test"})'

exim logs show correctly procmail action:

2008-03-01 21:47:09 1JVZXu-0004Dz-Kg H=(localhost) [192.168.0.101] Warning: SPAM-score: 29
2008-03-01 21:47:09 1JVZXu-0004Dz-Kg <= test@exim.depesz H=(localhost) [192.168.0.101] P=esmtp S=522 id=20080301_214706_028083.test@exim.depesz
2008-03-01 21:47:09 1JVZXu-0004Dz-Kg => depesz <depesz@exim.depesz> R=procmail T=procmail_pipe
2008-03-01 21:47:09 1JVZXu-0004Dz-Kg Completed

and procmail.log show us:

From test@exim.depesz Sat Mar 01 21:47:09 2008
 Subject: <No subject>
  Folder: .internal/new/1204408029.16252_0.xxx                              634

please note that the folder name is shown in short way.

is the file where it should be?

=> find /mails/exim.depesz/depesz/maildir/ -type f
/mails/exim.depesz/depesz/maildir/.internal/new/1204408029.16252_0.xxx
/mails/exim.depesz/depesz/maildir/new/1204407897.16172_2.xxx

looks fine.

but is it?

let's copy the procmailrc file to another account (and modify it accordingly):

=> cat /mails/badtest/depesz/procmailrc
SHELL=/bin/bash
MAILDIR=/mails/badtest/depesz/maildir
LOGFILE= /mails/badtest/depesz/procmail.log
UMASK=027
INBOX=$MAILDIR
 
:0
* ^From:.*@exim
.internal/
 
:0
$INBOX/

what will happen now if i'll try to mail to depesz@badtest?

=> perl -MMail::Sender -e '$m=new Mail::Sender {smtp=>"192.168.0.101",from=>"test\@exim.depesz"};$m->MailMsg({to=>"depesz\@badtest",msg=>"test"})'

and what does exim show?

2008-03-01 21:51:08 1JVZbl-0004Fq-PN H=(localhost) [192.168.0.101] Warning: SPAM-score: 29
2008-03-01 21:51:08 1JVZbl-0004Fq-PN <= test@exim.depesz H=(localhost) [192.168.0.101] P=esmtp S=514 id=20080301_215105_026087.test@exim.depesz
2008-03-01 21:51:25 1JVZbl-0004Fq-PN ** depesz@badtest R=procmail T=procmail_pipe: Child process OF procmail_pipe transport (running command "procmail "${lookup pgsql{SELECT get_account_homedir('${local_part}', '${domain}')}}/procmailrc"") was TERMINATED BY signal 11 (Segmentation fault)
2008-03-01 21:51:25 1JVZc5-0004G7-3r <= <> R=1JVZbl-0004Fq-PN U=Debian-exim P=LOCAL S=1285
2008-03-01 21:51:25 1JVZbl-0004Fq-PN Completed
2008-03-01 21:51:25 1JVZc5-0004G7-3r => test <test@exim.depesz> R=pg_user T=pg_delivery
2008-03-01 21:51:25 1JVZc5-0004G7-3r Completed

sig11?!

there is no procmail.log file:

=> ls -l /mails/badtest/depesz/procmail.log
ls: /mails/badtest/depesz/procmail.log: No such file or directory

so we basically dont know what happened.

to be honest – it took me a lot of time to figure it out. but i got the solution.

in case your procmailrc file is faulty procmail delivers mail to $HOME/dead.letter. but in our case $HOME is set to /tmp.

which means that it uses /tmp/dead.letter. and since it is mailbox, it sets mode of the file to 600. now. – mail to another domain will be passed as another “uid", so procmail can't open the dead.letter file.

and it tries to open it even if procmailrc is perfectly fine.

if opening of the file fails, procmail dies with “segmentation fault". for me it looks like a genuine bug, but since last version of procmail was released in 2001, i will nto even bother to raport it anywhere.

instead, i will simply modify our procmail_pipe transport. it's command line should look:

  command = procmail -m "ORGMAIL=${lookup pgsql{SELECT get_account_maildir('${local_part}', '${domain}')}}/dead.letter" "${lookup pgsql{SELECT get_account_homedir('${local_part}', '${domain}')}}/procmailrc"

now, let's restart exim, and deliver some mails:

=> perl -MMail::Sender -e '$m=new Mail::Sender {smtp=>"192.168.0.101",from=>"test\@exim.depesz"};$m->MailMsg({to=>"depesz\@exim.depesz",msg=>"test"})'
=> perl -MMail::Sender -e '$m=new Mail::Sender {smtp=>"192.168.0.101",from=>"test\@exim.depesz"};$m->MailMsg({to=>"depesz\@badtest",msg=>"test"})'

exim logs:

2008-03-01 21:57:55 1JVZiL-0004N4-5f H=(localhost) [192.168.0.101] Warning: SPAM-score: 29
2008-03-01 21:57:55 1JVZiL-0004N4-5f <= test@exim.depesz H=(localhost) [192.168.0.101] P=esmtp S=522 id=20080301_215753_029738.test@exim.depesz
2008-03-01 21:57:56 1JVZiL-0004N4-5f => depesz <depesz@exim.depesz> R=procmail T=procmail_pipe
2008-03-01 21:57:56 1JVZiL-0004N4-5f Completed
2008-03-01 21:58:03 1JVZiT-0004NG-72 H=(localhost) [192.168.0.101] Warning: SPAM-score: 29
2008-03-01 21:58:03 1JVZiT-0004NG-72 <= test@exim.depesz H=(localhost) [192.168.0.101] P=esmtp S=514 id=20080301_215801_064012.test@exim.depesz
2008-03-01 21:58:04 1JVZiT-0004NG-72 => depesz <depesz@badtest> R=procmail T=procmail_pipe
2008-03-01 21:58:04 1JVZiT-0004NG-72 Completed

great. works.

as always, you can get current version of exim4.conf.template and config.autogenerated files.

in next part of our ongoing tutorial i will show how to add “+" aliases that gmail introduced.

One thought on “smtp + sql = more than it seems so (part 8)”

Comments are closed.