Tag: spam

Visualizing Exim logs with Graylog

Ryanexim, graylog, spam, spamassassin

I spent some time the last few days tweaking my mail server settings since there has been an annoying rise in spam lately. Nothing special, mostly spring-cleaning of the blocklists and Spam Assassin settings. But as I was going over my config, I realized I didn’t have any way to measure “success”. I don’t really know which Blocklists work well for me and which don’t.

I use Graylog to collect logs from my systems and applications. But as far as my Exim logs are concerned my setup was pretty barebones (i.e. not parsing any fields, just dumping them as they were into Graylog). So I spent some time setting up proper extractors for my Exim logs to store everything useful in fields. A lot of the Exim logs use a straightforward key=value structure, making them easy to parse.

(spoiler: I bundled up everything here as a Graylog Content pack in case anyone wants to use it. Link at the bottom of the post)

Once the logs were properly parsed, I moved on to Dashboards to visualize the data. I started out with a visualization of the score Spam Assassin assigns to incoming emails (negative is good, positiv is bad, it’s been years since I’ve seen anything above a score of 5.0 that wasn’t spam). This gives me an indication of the quality of the mail making it through filters to my mailbox.
Then a little overview of incoming and outgoing mail, and how much is discarded by SPF and DNSBL.

This dashboard s the most interesting one when it comes to deciding which DNSBL lists are useful and which aren’t. It shows which lists are finding spambot globally as well as over time.
All my dashboards also have a widget with the relevant logs from the dashboard underneath to have easy access to the raw logs.

Since I had the data anyway, I also create a dashboard to show transport encryption information. About 60% of mail servers seem to support transport encryption, which is a lot lower than I would have expected (since it is easy to configure). I didn’t dig deeper into this, but I wouldn’t be surprised if the 40% sending email using plain unencrypted methods are mostly spammers that have very simple bots running to send their email.

 

This dashboard is technically also not related to spam, it’s bots trying to brute force user accounts on my mail server to abuse them to send more spam. Fairly aggressive fail2ban settings take care of that though.
It’s interesting to see, that the botnets aren’t used solely for sending spam, they are also used to try and compromise mail server accounts to increase the volume of mails they can send.

Link: Graylog content pack for Exim

Moving Exim/Spamassassin/Cyrus -> Debian config

Ryancyrus, Debian, exim, imap, linux, mail, spam, spamassassin

I’ve been putting off moving my mail system to the new server for a few weeks now since the old system was configured from scratch using the original config files and not the debian style config files. The differences in the Exim config are extreme. Debian splits the one large config file into lots of smaller files. This is great if: you never worked with exim before, you aren’t trying to migrate an existing configuration that is in one large file, and you don’t have all kinds of custom stuff like imap, spamassassin, greylisting mixed in. Yeah, not me. Even though I find the “one large file” a whole lot faster and easier to read, the Debian way has one big advantage: it is way easier for external scripts and packages to drop their custom config into exim. They just add a file to the right directory and thats it.

So I decided to go for it and merge my custom stuff into the Debain config. Greylisting worked out-of-the-box, spamassassin needed some minor tweaks, exim was (more or less) easy. The p.i.t.a. with exim is when you know exactly what is missing and where it would be configured, but because of “smart debian scripts” you have to find some config in an unrelated script and put the value there so it gets put in the right placeholder.

Here is a little summary in case I ever do this again and need to see if I forgot something:
– cyrus: copy /var/spool/cyrus/mail/ , /var/lib/cyrus/user/ , use cyradm to add the user.blargh account and /usr/sbin/cyrreconstruct -rf user (don’t forget the sieve filters)
– getmail: nothing special here, just copy config and add cronjob
– spamassassin: alter exim acl to set noscan for auth’d connections and have spamassassin scan everything not “noscan” (because per default local mail isn’t scanned, that includes everything we pick up via getmail)
– exim: check update-exim4.conf.conf for stupid entries, remember to turn on TLS (imap can use the same certificates), since we are using sasl for imap, have smtp auth use the same database (plain_saslauthd_server), turn on TLS by creating a file conf.d/main/00_exim4-config_localmacros with “MAIN_TLS_ENABLE = true” in it

Now that I’m done I found a pretty detailed German website with steps to set up such a system E-Mail-Server mit Debian, Exim und Cyrus. I did the exim router/transports a bit differently to have a bit more control over what goes where when. Still, defiantly worth reading if you are thinking about building such a system.