In a previous blog post, I lamented Gmail not accepting posts from a mailing list I administer. I ended up setting up an SPF record for the domain, which didn't seem to work at first, but after a while I didn't see bounces anymore.
This became current again, as today I noticed that my mail.log has a lot of entries like the following:
Sep 14 07:17:11 ks3353770 postfix/smtp[20101]: 98D4E60C58: host gmail-smtp-in.l.google.com[173.194.66.26] said: 421-4.7.0 [37.187.19.113 15] Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0 http://www.google.com/mail/help/bulk_mail.html to review our Bulk 421 4.7.0 Email Senders Guidelines. lj9si9839389wic.105 - gsmtp (in reply to end of DATA command)
It seems that Gmail has just relaxed the controls, only throttling the messages, not rejecting them anymore. This is a big step in the right direction, although I'd like to see more precise documentation why the messages are tagged as they are. I understand why Google would want to keep the exact algorithms and processes shrouded in mystery; to keep the spammers ignorant and make it more difficult to develop counter-measures, and to also keep competitors out of the loop.
I installed GNU Mailman to hopefully make Gmail accept my mailing list emails without the hassle of throttling, so that my users would not see unnecessary delay.
I am running a Debian-based installation on my server, so I was hoping I could just get it running with the packages in the repository. Maintaining source installations is a pain.
% sudo apt-get install mailman
Although Nginx and Mailman page in Nginx wiki talks about using thttpd, I have good expeciences with fastcgi and Nginx, albeit using supervisord, so I am going to try fcgiwrap out, as explained in Mailman and Fast CGI.
I needed to tweak the configuration a bit, and probably this still needs changes (for archive access etc.):
root /usr/local/mailman/cgi-bin;
location / {
rewrite ^ /mailman/listinfo permanent;
}
location /mailman {
fastcgi_split_path_info ^/mailman/([^/]*)(.*)$;
fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
#fastcgi_param PATH_TRANSLATED $document_root$fastcgi_path_info;
include /etc/nginx/fastcgi_params;
fastcgi_pass unix:/var/run/fcgiwrap.socket;
}
location /icons {
alias /usr/local/mailman/icons;
}
Oh, and I like OVH's DNS configuration. Adding a new CNAME field for my the new mailing list administration interface was a breeze.
The only difficulties I had were with conflicting documentation: /etc/mailman/postfix-to-mailman.py in the installation and official Mailman documentation http://www.list.org/mailman-install/postfix-integration.html seemed to be in conflict. From Mark Sapiro's comment in MailMan wiki article
I do not recommend people follow this article. I do not recommend postfix_to_mailman.py which is a 3rd party module neither distributed nor officially supported by the GNU Mailman project.
I got the hint and started reading the Mailman documentation. I was a little bit confused, should I add lists.liskot.org to virtual domains or not? After a while I decided that I should.
I tried sending the email to my new shining list, but I got this bounce and log message:
Sep 14 12:19:27 ks3353770 Mailman mail-wrapper: Group mismatch error. Mailman expected the mail wrapper script to be executed as group "daemon", but the system's mail server executed the mail script as group "mail-nobody". Try tweaking the mail server to run the script as group "daemon", or re-run configure, providing the command line option `--with-mail-gid=mail-nobody'.
I tried googling this for some time, but could not find an answer. I gave up, downloaded the source code, created the suggested user and group mailman and compiled. After fixing some lingering configuration problems, I got the following log message:
Sep 14 15:02:38 ks3353770 Mailman mail-wrapper: Group mismatch error. Mailman expected the mail wrapper script to be executed as group "mail-nobody", but the system's mail server executed the mail script as group "mailman". Try tweaking the mail server to run the script as group "mail-nobody", or re-run configure, providing the command line option `--with-mail-gid=mailman'.
Ah ha hah hah. For some reason postfix runs the Mailman programs now as mailman, instead of mail-nobody. I reconfigure with ./configure --with-mail-gid=mailman --with-cgi-gid=www-data, retried and did not get a bounce. I did not get anything in fact, but this was resolved soon.
After following the rest of the installation guide and restarting Mailman qrunner (cp scripts/mailman /etc/init.d/mailman; service mailman start) I got my first mails from Mailman. Uninstalling the Debian stock Mailman left some files behind, namely /etc/mailman and /etc/init.d/mailman, which were confusing after the installation from source, so remember to cp /usr/local/mailman/scripts/mailman /etc/init.d/mailman and rm -rf /etc/mailman.
Visiting the Mailman web interface, I noticed that the link to the archives was wrong, as were URLs for form handling. After tweaking /usr/local/mailman/Mailman/mm_cfg.py with
DEFAULT_EMAIL_HOST = 'lists.liskot.org' DEFAULT_URL_HOST = 'lists.liskot.org' DEFAULT_URL_PATTERN = 'https://%s/mailman/' VIRTUAL_HOSTS.clear() add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
and still not getting it to work (I restarted Nginx and fcgiwrap a few times here), I re-read the documentation, and noticed that the configuration will only take effect on new lists. Old lists need to be updated by running sudo -u mailman bin/withlist -l -a -r fix_url, as described here after which the Archive link was correct.
Now I need to test the interface with real users and migrate the old lists to the new interface.