Visualizing Exim logs with Graylog

I spent some time the last few days tweaking my mail server settings since there has been an annoying rise in spam lately. Nothing special, mostly spring-cleaning of the blocklists and Spam Assassin settings. But as I was going over my config, I realized I didn’t have any way to measure “success”. I don’t really know which Blocklists work well for me and which don’t.

I use Graylog to collect logs from my systems and applications. But as far as my Exim logs are concerned my setup was pretty barebones (i.e. not parsing any fields, just dumping them as they were into Graylog). So I spent some time setting up proper extractors for my Exim logs to store everything useful in fields. A lot of the Exim logs use a straightforward key=value structure, making them easy to parse.

(spoiler: I bundled up everything here as a Graylog Content pack in case anyone wants to use it. Link at the bottom of the post)

Once the logs were properly parsed, I moved on to Dashboards to visualize the data. I started out with a visualization of the score Spam Assassin assigns to incoming emails (negative is good, positiv is bad, it’s been years since I’ve seen anything above a score of 5.0 that wasn’t spam). This gives me an indication of the quality of the mail making it through filters to my mailbox.
Then a little overview of incoming and outgoing mail, and how much is discarded by SPF and DNSBL.

This dashboard s the most interesting one when it comes to deciding which DNSBL lists are useful and which aren’t. It shows which lists are finding spambot globally as well as over time.
All my dashboards also have a widget with the relevant logs from the dashboard underneath to have easy access to the raw logs.

Since I had the data anyway, I also create a dashboard to show transport encryption information. About 60% of mail servers seem to support transport encryption, which is a lot lower than I would have expected (since it is easy to configure). I didn’t dig deeper into this, but I wouldn’t be surprised if the 40% sending email using plain unencrypted methods are mostly spammers that have very simple bots running to send their email.

 

This dashboard is technically also not related to spam, it’s bots trying to brute force user accounts on my mail server to abuse them to send more spam. Fairly aggressive fail2ban settings take care of that though.
It’s interesting to see, that the botnets aren’t used solely for sending spam, they are also used to try and compromise mail server accounts to increase the volume of mails they can send.

Link: Graylog content pack for Exim

How to fix Mono crashing on Odroid XU4

Recently I’ve been noticing my Sonarr and Radarr applications behaving erratic (sometimes not responsive, sometimes not performing tasks, not searching or adding content, but at other times behaving totally fine). A quick look at the logs told me the applications were crashing and being restarted by systemctl, after a few crashes they seemed to stabilize.
Today I had some time to dig deeper into the issue. I had already searched the Internet for general issues, but it didn’t seem to be a widespread problem. I assumed it might have to do with the ARM architecture, but Raspberry Pi users didn’t seem to be having these issues.

In the past I had difficulty reproducing the issue, but today I was in luck, every time I tried to kick off the “Process Monitored Downloads” task in Radarr, it would start working on the task and then crash and restart. The core issue turned out to be oddly specific to the Odroid XU4 hardware.

The XU4 has eight CPU cores, four A7 cores running at 1.4GHz, and four A15 cores running at 2GHz.

 

Whenever the mono process moved from an A7 to an A15 core (or vice versa) the process crashed.
Since both Sonarr and Radarr are Mono applications, they were both affected. Pinning the applications to either the A15 or the A7 CPUs resolved the problem.
taskset --cpu-list  can be used to change the CPU affinity of a process.

First look up the systemctl  service files for Radarr and Sonarr (e.g. via systemctl status radarr ). Edit the service files by prefixing the ExecStart  command with /usr/bin/taskset -c 0-3  (for the A7 cores) or /usr/bin/taskset -c 4-7  (for the A15 cores).
Then reload the systemctl files ( systemctl daemon-reload ) and restart the service.

Example:

If the service files are managed via the package manager, you may want to create an systemctl override instead of editing the service file, so that the package manager doesn’t overwrite your changes:

 

Now you probably want to see if it worked and want to know how to check which core a process is running on? There a re a few options:

htop: launch htop . Press <F2> , go to Columns , and add PROCESSOR  from Available Columns. The CPUs are numbered 1-8 in htop (as opposed to 0-7 by the system).

ps: the PSR column can display the core a process is on. The CPUs are numbered 0-7 in ps.

taskset: can display the current affinity of a process.

 

Printing wood filament

I wanted to try printing a wood filament and picked up a roll of Easywood Pine from FormFutura. Wood filaments are a mixture of PLA and wood fibers or particles. I tried printing a few test objects, and from a printing perspective it prints very similar to normal PLA.

Due to the fibers in the filament I’d suggest a 0.6mm nozzle (or larger) to reduce the chance of clogging. And I also went with a steel nozzle from micro-swiss since the wood in the filament can be a bit abrasive and wanted a more durable nozzle.

Realistically, we are just printing plastic with wood fibers in it, nothing we print will ever be confused with real wood. That being said, it does have wood like characteristics, the surface and smell do have a slightly wood like quality to them. If I had to compare it to something, I’d say it reminds me most of thick rough cardboard.

Filament card Top

As far as print accuracy goes, It faired pretty well. some details were lost (most obvious in the last photo of this post), but that was to be expected both printing with a rough material and printing with a larger nozzle.

Viewed from the back we can see the underside of the bridging on the bottom right. The card was printed on a flat metal surface, the filament adheres very well to the build plate. I also tried out different build surfaces, even a rough one in the hopes it looked more wood-like. But the rougher surfaces all turned out shiny and looking very fake due to the plasticky rough surface.

Filament card Bottom

I printed the same card with ironing enabled on the top layer. The results were not good at all. The ironed parts are very uneven and the wood feeling of the surface is pretty much gone, the surface is non-uniformly rough, but not in a good way. It feels as if too much heat is transferred into the printed object and the plastic becomes very dominant. Having the filament heated up but flowing slowly through the nozzle can also “burn” the wood fibers and/or increase the chance of clogging.
There are probably situations where (localized) ironing my have beneficial results, but It’s not something I suggest having on by default.

Ironing

A final photo with a direct comparison between a 0.4mm nozzle printed PLA+ filament and the 0.6mm nozzle printed wood filament.
It’s not a filament I’d use every day, but I’m happy to have it here, and have a few ideas I’d like to try out with this filament.

“Wood” filament compared to PLA+ filament

Statically served wordpress content

I’m currently still evaluating hugo and jekyll, themes and plugins, as an alternative to the current WordPress site. Until I decide what route to eventually go with, I had a look at WordPress plugins to generate static versions of a site.

Simply Static looked fine and I gave it a spin, it can easily crawl through the site and you can provide additional file/urls/directories to add to the static version (as well as exemptions).

The static version of the website is created regularly and stored locally, so I added a few ansible tasks to set up a periodic rsync of the files to my webserver that serves static content.

I have a HAProxy load balancer in front of my webservers that I have configured to serve the static version of the website first, and fall back to the wordpress server as a backup (that also gives me a nice redundancy, so I can update and reboot servers without causing a downtime).
HAProxy is also configured to always send certain requests (admin interface, search) to the WordPress server since they require PHP. This all happens transparently for the user.

I’m not going to bore with the details since it was all pretty standard stuff. It’s nothing fancy, but it looks reliable and does what it should.

I have this blog entry scheduled to go live in a few days, so we’ll see if all the automatisms work and the static version of the page generated and synced to the webserver.

Why filament cards are useful.

One thing I learned while printing 3D is that a lot depends on which printer you use, what and how you print, and what filamants you use. This blog entry is a “this is how I do it and what works for me, and why it works for me”, other people may have different opinions based on how/what they print and which filaments they use, and that is totally fine. If filament cards are something new to you or something you are interested in, I’m happy to share my experiences and knowledge here with you. Links to models in the pictures here are at the bottom of the blog post.

What is a filament card

A filament card (or color swatches, or whatever you want to call them) is basically just a small card. It’s design can range from a plain rectangle for color reference, or have features built into it like different layer depths, bridging and wall strengths. It can any shape, but rectangles are often used, but I’d suggest using the same shape/model for all cards.

 

Why are filament cards useful?

Printing filament cards when you get a new roll of filament is a habit I’d suggest getting into early. They allow you to easily reference or compare colors and printing results to chose the right filament for the job. They are also useful if you get samples from a product.
If you were ever trying to decide “which of these blues would look best for what I need to print” or “which of my filaments had the best bridging results”, then this is a system you will enjoy.

filament cards

 

Labeling

Some people creates cards for each specific filament, printing the name and settings (e.g. temperature) directly into or onto the card. While I find this neat, I wanted to avoid having to create a new card for each filament (since newer slicer versions may create different gcode and I’d be comparing not only filament A to filament B, but also different slicer results, and the name can be quite long depending on how much information I want to present). The upside to only having one STL/gcode for all cards is that is is easy to manage and compare results. The downside is that a) I need to label the cards in a different manner, and b) adjust the temperature settings for the filament on the printer.
For labeling I use a traditional label printer (as seen on the pictures), and document manufacturer and color of the filament (I only add the temperature if it is unusual). On the pictures above I put the label on the front, but on the latest cards I’ve printed I’ve started placing the label to the back since there is more space there (back surface quality is mostly dictated by the printer bed and less interesting), the front does a better job of showing the surface quality of the filament, and in the back the label doesnt interfere with the tray slots.

Card features

Comparing colors and the “feel” of the filament is what I use the cards mostly for, but I also chose a cards with a few extra “features” that come in handy. The cards I use are a little bit higher than what you normally see to give me more room for the label and more material to compare. On the top there are a few small tests (round ball, wall with hole, bridging, cylinder only supported on one end, slanted surface), the card isn’t uniformly square on the outside (some edges are rounded). On the top we have square corners, on the bottom round corners. The bottom part of the card starts off with an unsupported bridge, then slowly increasing card thickness (if you hold the card up against a light, you can see how translucent the card is at which thickness). Along the bottom edge ther is an overhang test, another thickness test with more gradients and a wall thickness test.

There are a variety of simpler and more complex cards out there, pick whatever is useful for you and what you print (or make your own).

 

filament card front
Front
filament card back
Back

Storage

Some people put their cards on zip ties or rings, some just collect them loosely. I like to put they cards in trays and sort by filament type and color. Trays come in all sorts and sizes, but should be high enough to keep the cards from falling out, and low enough to easily remove individual cards.

filament card tray

Where to get

Thingiverse has plenty of options if you search for “filament card”. I personally prefer customizable objects that I can tweak with OpenSCAD to my liking. I’d suggest saving the STL file and generating the gcode files once and using the same files for all filaments, since slicer software updates and changes over time, and you want to compare filaments and not slicer results. I have two files, for 0.4mm, and 0.6mm nozzles (since some filaments require a larger nozzle).

I can highly recommend the following card and card tray (these are the ones used in the pictures above):

Customizable filament cards: thingiverse 3346069
Customizable tray thingiverse 3595869