Tag: spamassassin

Visualizing Exim logs with Graylog

Ryanexim, graylog, spam, spamassassin

I spent some time the last few days tweaking my mail server settings since there has been an annoying rise in spam lately. Nothing special, mostly spring-cleaning of the blocklists and Spam Assassin settings. But as I was going over my config, I realized I didn’t have any way to measure “success”. I don’t really know which Blocklists work well for me and which don’t.

I use Graylog to collect logs from my systems and applications. But as far as my Exim logs are concerned my setup was pretty barebones (i.e. not parsing any fields, just dumping them as they were into Graylog). So I spent some time setting up proper extractors for my Exim logs to store everything useful in fields. A lot of the Exim logs use a straightforward key=value structure, making them easy to parse.

(spoiler: I bundled up everything here as a Graylog Content pack in case anyone wants to use it. Link at the bottom of the post)

Once the logs were properly parsed, I moved on to Dashboards to visualize the data. I started out with a visualization of the score Spam Assassin assigns to incoming emails (negative is good, positiv is bad, it’s been years since I’ve seen anything above a score of 5.0 that wasn’t spam). This gives me an indication of the quality of the mail making it through filters to my mailbox.
Then a little overview of incoming and outgoing mail, and how much is discarded by SPF and DNSBL.

This dashboard s the most interesting one when it comes to deciding which DNSBL lists are useful and which aren’t. It shows which lists are finding spambot globally as well as over time.
All my dashboards also have a widget with the relevant logs from the dashboard underneath to have easy access to the raw logs.

Since I had the data anyway, I also create a dashboard to show transport encryption information. About 60% of mail servers seem to support transport encryption, which is a lot lower than I would have expected (since it is easy to configure). I didn’t dig deeper into this, but I wouldn’t be surprised if the 40% sending email using plain unencrypted methods are mostly spammers that have very simple bots running to send their email.

 

This dashboard is technically also not related to spam, it’s bots trying to brute force user accounts on my mail server to abuse them to send more spam. Fairly aggressive fail2ban settings take care of that though.
It’s interesting to see, that the botnets aren’t used solely for sending spam, they are also used to try and compromise mail server accounts to increase the volume of mails they can send.

Link: Graylog content pack for Exim

Playing around with Exim, Spamassassin, DKIM and SPF

RyanDKIM, exim, intenet, SMTP, spamassassin, SPF

I was playing around with my mail server today and decided to have a look at DKIM and SPF. Jump to the bottom for some useful links. You will have to be able to change TXT entries for your domain/subdomains to do any of this. Sending a mail to check-auth@verifier.port25.com will generate an auto-reply with the result of your mail servers settings. I also noticed that Exim wasn’t listening to the IPv6 interface and promptly fixed that, and have received my first mail via IPv6.

Some spamassassin scores for your local/user settings:

SPF:
Syntax: http://www.openspf.org/SPF_Record_Syntax
Tools & Information: http://www.openspf.org/
Wizard: http://www.spfwizard.net/

DKIM:
Exim walkthrough: http://mikepultz.com/2010/02/using-dkim-in-exim/
Domain specific settings: http://www.debian-administration.org/users/lee/weblog/41
Tools: http://dkimcore.org/tools/
Wizard: https://www.port25.com/dkim-wizard/

Moving Exim/Spamassassin/Cyrus -> Debian config

Ryancyrus, Debian, exim, imap, linux, mail, spam, spamassassin

I’ve been putting off moving my mail system to the new server for a few weeks now since the old system was configured from scratch using the original config files and not the debian style config files. The differences in the Exim config are extreme. Debian splits the one large config file into lots of smaller files. This is great if: you never worked with exim before, you aren’t trying to migrate an existing configuration that is in one large file, and you don’t have all kinds of custom stuff like imap, spamassassin, greylisting mixed in. Yeah, not me. Even though I find the “one large file” a whole lot faster and easier to read, the Debian way has one big advantage: it is way easier for external scripts and packages to drop their custom config into exim. They just add a file to the right directory and thats it.

So I decided to go for it and merge my custom stuff into the Debain config. Greylisting worked out-of-the-box, spamassassin needed some minor tweaks, exim was (more or less) easy. The p.i.t.a. with exim is when you know exactly what is missing and where it would be configured, but because of “smart debian scripts” you have to find some config in an unrelated script and put the value there so it gets put in the right placeholder.

Here is a little summary in case I ever do this again and need to see if I forgot something:
– cyrus: copy /var/spool/cyrus/mail/ , /var/lib/cyrus/user/ , use cyradm to add the user.blargh account and /usr/sbin/cyrreconstruct -rf user (don’t forget the sieve filters)
– getmail: nothing special here, just copy config and add cronjob
– spamassassin: alter exim acl to set noscan for auth’d connections and have spamassassin scan everything not “noscan” (because per default local mail isn’t scanned, that includes everything we pick up via getmail)
– exim: check update-exim4.conf.conf for stupid entries, remember to turn on TLS (imap can use the same certificates), since we are using sasl for imap, have smtp auth use the same database (plain_saslauthd_server), turn on TLS by creating a file conf.d/main/00_exim4-config_localmacros with “MAIN_TLS_ENABLE = true” in it

Now that I’m done I found a pretty detailed German website with steps to set up such a system E-Mail-Server mit Debian, Exim und Cyrus. I did the exim router/transports a bit differently to have a bit more control over what goes where when. Still, defiantly worth reading if you are thinking about building such a system.

Email Server

Ryanexim, greylisting, spamassassin

I’m not a big fan of using greylisting to cut down on email spam, since it tends to big a big pain in the ass when I run into it running on other email servers. On the other hand the amount of incomming spam has risen to a level where I decided it’s time to really do something about it (not that Spamassassin was doing a bad job, but it shouldn’t have to go through hundreds of mails each day to sort out the 15-20 real mails).

So I did a quick google what packages were recommended for greylisting with exim, and ended up using “greylistd” which came with a nice example config for exim. I went through the config and loosened up the settings a bit. And after letting it run for 2 days, the stats say that a bit over 96% of incoming mail was never retried (e.g. either spam, or a crappy MTA sending me mail).

Next thing I’m having a look at is SPF. I’ve got it up and running, but it only throws warnings to the log right now, doesn’t block anything (same goes for messed up DNS entries of servers sending me mail).

Something positive I noted was that the number of MTA servers supporting TLS connections seems to be rising. Server <-> Server encryption of data is a good thing. Having users view and send mail via secure HTTPS/SSL/TLS cconnections is pretty useless if the mails gets forwarded to the destination in plaintext.