Bear's nest

Installing Mailman

In a previous blog post, I lamented Gmail not accepting posts from a mailing list I administer. I ended up setting up an SPF record for the domain, which didn't seem to work at first, but after a while I didn't see bounces anymore.

This became current again, as today I noticed that my mail.log has a lot of entries like the following:

Sep 14 07:17:11 ks3353770 postfix/smtp[20101]: 98D4E60C58: host
gmail-smtp-in.l.google.com[173.194.66.26] said: 421-4.7.0
[37.187.19.113 15] Our system has detected an unusual rate of
421-4.7.0 unsolicited mail originating from your IP address. To
protect our 421-4.7.0 users from spam, mail sent from your IP address
has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0
http://www.google.com/mail/help/bulk_mail.html to review our Bulk 421
4.7.0 Email Senders Guidelines. lj9si9839389wic.105 - gsmtp (in reply
to end of DATA command)

It seems that Gmail has just relaxed the controls, only throttling the messages, not rejecting them anymore. This is a big step in the right direction, although I'd like to see more precise documentation why the messages are tagged as they are. I understand why Google would want to keep the exact algorithms and processes shrouded in mystery; to keep the spammers ignorant and make it more difficult to develop counter-measures, and to also keep competitors out of the loop.

I installed GNU Mailman to hopefully make Gmail accept my mailing list emails without the hassle of throttling, so that my users would not see unnecessary delay.

I am running a Debian-based installation on my server, so I was hoping I could just get it running with the packages in the repository. Maintaining source installations is a pain.

% sudo apt-get install mailman

Although Nginx and Mailman page in Nginx wiki talks about using thttpd, I have good expeciences with fastcgi and Nginx, albeit using supervisord, so I am going to try fcgiwrap out, as explained in Mailman and Fast CGI.

I needed to tweak the configuration a bit, and probably this still needs changes (for archive access etc.):

root /usr/local/mailman/cgi-bin;

location / {
  rewrite ^ /mailman/listinfo permanent;
}


location /mailman {
  fastcgi_split_path_info ^/mailman/([^/]*)(.*)$;
  fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
  fastcgi_param PATH_INFO $fastcgi_path_info;
  #fastcgi_param PATH_TRANSLATED   $document_root$fastcgi_path_info;
  include /etc/nginx/fastcgi_params;
  fastcgi_pass  unix:/var/run/fcgiwrap.socket;
}

location /icons {
  alias /usr/local/mailman/icons;
}

Oh, and I like OVH's DNS configuration. Adding a new CNAME field for my the new mailing list administration interface was a breeze.

The only difficulties I had were with conflicting documentation: /etc/mailman/postfix-to-mailman.py in the installation and official Mailman documentation http://www.list.org/mailman-install/postfix-integration.html seemed to be in conflict. From Mark Sapiro's comment in MailMan wiki article

I do not recommend people follow this article. I do not recommend postfix_to_mailman.py which is a 3rd party module neither distributed nor officially supported by the GNU Mailman project.

I got the hint and started reading the Mailman documentation. I was a little bit confused, should I add lists.liskot.org to virtual domains or not? After a while I decided that I should.

I tried sending the email to my new shining list, but I got this bounce and log message:

Sep 14 12:19:27 ks3353770 Mailman mail-wrapper: Group mismatch error.
Mailman expected the mail wrapper script to be executed as group
"daemon", but the system's mail server executed the mail script
as group "mail-nobody".  Try tweaking the mail server to run
the script as group "daemon", or re-run configure, providing the
command line option `--with-mail-gid=mail-nobody'.

I tried googling this for some time, but could not find an answer. I gave up, downloaded the source code, created the suggested user and group mailman and compiled. After fixing some lingering configuration problems, I got the following log message:

Sep 14 15:02:38 ks3353770 Mailman mail-wrapper: Group mismatch error.
Mailman expected the mail wrapper script to be executed as group
"mail-nobody", but the system's mail server executed the mail script
as group "mailman".  Try tweaking the mail server to run
the script as group "mail-nobody", or re-run configure, providing
the command line option `--with-mail-gid=mailman'.

Ah ha hah hah. For some reason postfix runs the Mailman programs now as mailman, instead of mail-nobody. I reconfigure with ./configure --with-mail-gid=mailman --with-cgi-gid=www-data, retried and did not get a bounce. I did not get anything in fact, but this was resolved soon.

After following the rest of the installation guide and restarting Mailman qrunner (cp scripts/mailman /etc/init.d/mailman; service mailman start) I got my first mails from Mailman. Uninstalling the Debian stock Mailman left some files behind, namely /etc/mailman and /etc/init.d/mailman, which were confusing after the installation from source, so remember to cp /usr/local/mailman/scripts/mailman /etc/init.d/mailman and rm -rf /etc/mailman.

Visiting the Mailman web interface, I noticed that the link to the archives was wrong, as were URLs for form handling. After tweaking /usr/local/mailman/Mailman/mm_cfg.py with

DEFAULT_EMAIL_HOST = 'lists.liskot.org'
DEFAULT_URL_HOST = 'lists.liskot.org'
DEFAULT_URL_PATTERN = 'https://%s/mailman/'
VIRTUAL_HOSTS.clear()
add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)

and still not getting it to work (I restarted Nginx and fcgiwrap a few times here), I re-read the documentation, and noticed that the configuration will only take effect on new lists. Old lists need to be updated by running sudo -u mailman bin/withlist -l -a -r fix_url, as described here after which the Archive link was correct.

Now I need to test the interface with real users and migrate the old lists to the new interface.

links

social