Paperless is a document management system that helps you manage digital scans of your documents. I’ve been using it for a while, but as with many projects the developer(s) lost motivation/time to keep the project up-to-date, a fork was made (paperless-ng) which after time also died off. Now a few developers got together, worked through the backlog of issues and forked the next generation as “paperless-ngx”.
Short version: I have my scanner set up to be able to store scans on a network share as PDFs, and paperless monitors the same share for new documents. If it finds a new file, it performs OCR on it (saves the PDF as well as the text) and runs user defined rules against it (e.g. detecting the date of the document, or correspondents, or what kind of document it is, …). It actually makes it very easy to manage physical documents. I just pop them in the scanner, select the destination share, and that’s it, everything else happens automatically. So if I need to search for documents needed to do my taxes. I can do it digitally and not have to go through folders with physical paper.
It can also ingest images or office documents, just toss them in the directory the application monitors for ingesting documents (or use the “Upload” button in the application).
I’m not a big fan of the default dark theme, but a normal light theme is still available.
All in all, it’s pretty nifty, so have a look if this is something that you might find useful.
I spent some time the last few days tweaking my mail server settings since there has been an annoying rise in spam lately. Nothing special, mostly spring-cleaning of the blocklists and Spam Assassin settings. But as I was going over my config, I realized I didn’t have any way to measure “success”. I don’t really know which Blocklists work well for me and which don’t.
I use Graylog to collect logs from my systems and applications. But as far as my Exim logs are concerned my setup was pretty barebones (i.e. not parsing any fields, just dumping them as they were into Graylog). So I spent some time setting up proper extractors for my Exim logs to store everything useful in fields. A lot of the Exim logs use a straightforward key=value structure, making them easy to parse.
(spoiler: I bundled up everything here as a Graylog Content pack in case anyone wants to use it. Link at the bottom of the post)
Once the logs were properly parsed, I moved on to Dashboards to visualize the data. I started out with a visualization of the score Spam Assassin assigns to incoming emails (negative is good, positiv is bad, it’s been years since I’ve seen anything above a score of 5.0 that wasn’t spam). This gives me an indication of the quality of the mail making it through filters to my mailbox. Then a little overview of incoming and outgoing mail, and how much is discarded by SPF and DNSBL.
This dashboard s the most interesting one when it comes to deciding which DNSBL lists are useful and which aren’t. It shows which lists are finding spambot globally as well as over time. All my dashboards also have a widget with the relevant logs from the dashboard underneath to have easy access to the raw logs.
Since I had the data anyway, I also create a dashboard to show transport encryption information. About 60% of mail servers seem to support transport encryption, which is a lot lower than I would have expected (since it is easy to configure). I didn’t dig deeper into this, but I wouldn’t be surprised if the 40% sending email using plain unencrypted methods are mostly spammers that have very simple bots running to send their email.
This dashboard is technically also not related to spam, it’s bots trying to brute force user accounts on my mail server to abuse them to send more spam. Fairly aggressive fail2ban settings take care of that though. It’s interesting to see, that the botnets aren’t used solely for sending spam, they are also used to try and compromise mail server accounts to increase the volume of mails they can send.
Recently I’ve been noticing my Sonarr and Radarr applications behaving erratic (sometimes not responsive, sometimes not performing tasks, not searching or adding content, but at other times behaving totally fine). A quick look at the logs told me the applications were crashing and being restarted by systemctl, after a few crashes they seemed to stabilize. Today I had some time to dig deeper into the issue. I had already searched the Internet for general issues, but it didn’t seem to be a widespread problem. I assumed it might have to do with the ARM architecture, but Raspberry Pi users didn’t seem to be having these issues.
In the past I had difficulty reproducing the issue, but today I was in luck, every time I tried to kick off the “Process Monitored Downloads” task in Radarr, it would start working on the task and then crash and restart. The core issue turned out to be oddly specific to the Odroid XU4 hardware.
The XU4 has eight CPU cores, four A7 cores running at 1.4GHz, and four A15 cores running at 2GHz.
Whenever the mono process moved from an A7 to an A15 core (or vice versa) the process crashed. Since both Sonarr and Radarr are Mono applications, they were both affected. Pinning the applications to either the A15 or the A7 CPUs resolved the problem. taskset--cpu-list can be used to change the CPU affinity of a process.
First look up the systemctl service files for Radarr and Sonarr (e.g. via systemctl status radarr ). Edit the service files by prefixing the ExecStart command with /usr/bin/taskset-c0-3 (for the A7 cores) or /usr/bin/taskset-c4-7 (for the A15 cores). Then reload the systemctl files ( systemctl daemon-reload ) and restart the service.
If the service files are managed via the package manager, you may want to create an systemctl override instead of editing the service file, so that the package manager doesn’t overwrite your changes:
Now you probably want to see if it worked and want to know how to check which core a process is running on? There a re a few options:
htop: launch htop . Press <F2> , go to Columns , and add PROCESSOR from Available Columns. The CPUs are numbered 1-8 in htop (as opposed to 0-7 by the system).
ps: the PSR column can display the core a process is on. The CPUs are numbered 0-7 in ps.
1
2
3
# ps -o pid,psr,comm -p 21130
PID PSR COMMAND
211300mono
taskset: can display the current affinity of a process.
I’m currently still evaluating hugo and jekyll, themes and plugins, as an alternative to the current WordPress site. Until I decide what route to eventually go with, I had a look at WordPress plugins to generate static versions of a site.
Simply Static looked fine and I gave it a spin, it can easily crawl through the site and you can provide additional file/urls/directories to add to the static version (as well as exemptions).
The static version of the website is created regularly and stored locally, so I added a few ansible tasks to set up a periodic rsync of the files to my webserver that serves static content.
I have a HAProxy load balancer in front of my webservers that I have configured to serve the static version of the website first, and fall back to the wordpress server as a backup (that also gives me a nice redundancy, so I can update and reboot servers without causing a downtime). HAProxy is also configured to always send certain requests (admin interface, search) to the WordPress server since they require PHP. This all happens transparently for the user.
I’m not going to bore with the details since it was all pretty standard stuff. It’s nothing fancy, but it looks reliable and does what it should.
I have this blog entry scheduled to go live in a few days, so we’ll see if all the automatisms work and the static version of the page generated and synced to the webserver.
Taking a different direction here and trying out a more HowTo style post. I thought I’d do a quick write-up covering WiFi SSIDs on multiple VLANs, something that might be useful in a home setup, separating SSIDs better from one another. While I usually prefer the CLI, a lot of the devices mentioned here only (or mainly) use a web GUI for configuration management, so this posting will include a lot more screenshots than my usual postings. This posting won’t contain anything exciting for people who run and support networks professionally (mostly just a “so what options are available on SOHO hardware?”), and may even be overly simple and assume settings you wouldn’t do in a professional setup. But it is useful for homelab enthusiasts looking to build a more complex setup than “everything on the same VLAN/network” or trying to figure out “how do I use VLANs, give me an example”.
Ubiquiti and Netgear have some reasonable priced network gear that I’ve written about in the past that behave decent in home networks. For out little setup today we will have a WiFi access point with multiple SSIDs, two switches, and a firewall/router. We will be focusing on the configuration of the WiFi Access Point (UniFi AP) , the two Netgear switches (GS105Ev2 / GS108Ev3) and the Ubiquiti Router (Edgerouter X).
Our goal is to have two SSIDs on the access point, one for the “normal” devices (VLAN 1), and one for separate devices (VLAN 178). Since the router to VLAN 178 is on a different switch than the access point we also need to trunk both VLANs across the network. Another way of looking at it, is that we are expanding Subnet/VLAN 178 into our network (but limiting it to the WiFi Access points). 178 doesn’t hold any special meaning, you can use any VLAN number you want, just chose 178 because the subnet in this test setup was 10.0.178.0/24.
Here is a diagram of our test setup:
VLAN 1: The management and main client VLAN VLAN 178: Subnet 178, 2nd WiFi network
Netgear 1
Port 1: Connection to Netgear 2 Port 5: Access Point
Netgear 2
Port 1: Connection to Netgear 1 Port 8: Edgerouter
Edgerouter
eth0: Connection to Subnet 178 eth1: Connection to Netgear 2
Wireless Configuration
In the UniFi controller go to Settings->Wireless Networks either create a new network or edit an existing one. If you go into the advanced options area, you will find the VLAN setting for that SSID. Activating the option and setting a value means the access point will tag all packets from that SSID to the specified VLAN number.
In the network overview below we see that packets from one network are tagged with VLAN 178, and packets from the other network are untagged (the switch configuration will take care of deciding which VLAN untagged packets are part of).
Netgear Configuration
The Netgear devices only have a VLAN 1 defined by default, so the first step will be to add VLAN 178 to both of them. Enable Advanced 802.1q (VLAN->802.1Q->Advanced->VLAN Configuration), enter the VLAN ID 178 in the box and click on “Add” at the top.
Now let’s move on to the VLAN Membership configuration. To switch settings, just click on the port and it will switch from tagged->untagged->none.
Netgear 1 device
For VLAN 1 we want port 1 to be (T)agged since it is the trunk to the next switch, and all other ports (U)ntagged since they will be dealing with untagged VLAN 1 traffic. The PVID will also be set to 1 for all ports.
For VLAN 178 we want port 1 to be (T)agged since it is the trunk to the next switch, port 5 (T)agged since it is the connection to the access point, and all other ports “none” since none of those ports are part of VLAN 178.
VLAN overview after changing all the settings:
For this scenario it is sane to leave the Port PVID configuration to PVID 1 on all ports. You could change a port to 178 if you are expecting untagged traffic on that port that needs to be in VLAN 178.
Netgear 2 device
Similar configuration as the first switch so I’ll skip the screenshots. Port 1 is the connection to the Netgear 1 device, port 8 is connected to the Edgerouter.
all ports PVID 1
VLAN 1 Tagged port 1 and 8, all other ports untagged
VLAN 178 Tagged port 1 and 8, all other ports none
Edgerouter Configuration:
Make sure the interface switch0 spans all ports you want to use VLANs on. Add the two VLANs 1 and 178. They should show up in the list as switch0.1 and switch0.178, and you can apply firewall rules to these interfaces to restrict traffic if you are doing any routing between the two networks. We will want to tag everything from Subnet 178 as VLAN 178 on the interface connected to that network (eth0 in our setup), and allow VLAN 178 on the outgoing interface (eth1 in our setup). The Edgerouter doesn’t need us to specifically allow VLANs already set by the PVID, only additional VLANs need to be specified in the vid field.
Where to go from here?
You could put servers or virtual machines in their own subnet/VLAN and then use a firewall to restrict access between the different networks.
Adding more Access Points is straightforward, UniFi automatically applies the wireless configuration to all Access Points it manages, so all you need to do is take care of the switch port configuration.
We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.OkPrivacy policy
