/etc/cron.daily/exim4-base exited with return code 123

The problem has been reported here, but I'm not sure the solution was clearly stated.

I got this under Ubuntu 9.04, exim4 4.69-9ubuntu1.

In short, the problem is caused by the following lines in /etc/cron.daily/exim4-base:

 + find /var/spool/exim4/db -maxdepth 1 -name *.lockfile \ 
    -or -type f -printf %f\0
 + su - --shell /bin/bash --command xargs -0r -n 1 \ 
    /usr/sbin/exim_tidydb /var/spool/exim4 > /dev/null Debian-exim

These two lines find files in /var/spool/exim4/db, where live various databases used by exim, and perform an exim_tidydb on each file. The files matching *.lockfile are filtered out.

exim_tidydb thinks there are live databases, and thus try to acquire the lock in file xxx.lock for a file named xxx. This is where the problems occurs.

In my case, there is another file in the directory, left by some upgrade script:

 # cd /var/spool/exim4/db/ 
 # ls
 retry  retry.lockfile  wait-remote_smtp
 wait-remote_smtp.lockfile  wait-remote_smtp.old

The faulty file is wait-remote_smtp.old. When operating on it, exim_tidydb tries to acquire non-existent lock filewait-remote_smtp.old.lockfile and fails.

The solution: just delete that spurious wait-remote_smtp.old or move it elsewhere.

How did I come to that ?

By manually stepping through the script lines:

First I replaced the first line of /etc/cron.daily/exim4-base

 #!/bin/sh

by

 #!/bin/sh -x

This way, the cron output contains every steps done by the script. This is how we spot the 2 faulty lines. Then in order to understand what's going on, we just play the lines:

 # cd /var/spool/exim4/db
 # find /var/spool/exim4/db -maxdepth 1 -name *.lockfile -or -type f -print
 /var/spool/exim4/db/retry
 /var/spool/exim4/db/wait-remote_smtp.old
 /var/spool/exim4/db/wait-remote_smtp

Now a line a bit tricky. The scripts runs as user Debian-exim , so we want to do that too. However this user has no login shell, as a security measure. We thus have to specify a working shell:

 # su - -s /bin/bash Debian-exim

Now we can do our tests:

 # /usr/sbin/exim_tidydb /var/spool/exim4 retry
 Tidying Exim hints database /var/spool/exim4/db/retry
 Tidying complete

Doesn't look bad. Another one:

 # /usr/sbin/exim_tidydb /var/spool/exim4 wait-remote_smtp
 Tidying Exim hints database /var/spool/exim4/db/wait-remote_smtp
 Tidying complete

So what ? Let's try the last one:

 # /usr/sbin/exim_tidydb /var/spool/exim4 wait-remote_smtp.old
 Tidying Exim hints database /var/spool/exim4/db/wait-remote_smtp.old
 ** Failed to open database lock 
 file /var/spool/exim4/db/wait-remote_smtp.old.lockfile:
 No such file or directory

Gotcha.