Category: Programming

Using regex comparision in bash and BASH_REMATCH

Ryanbash

Bash supports regular expressions in comparisons via the =~ operator.  But what is rarely used or documented is that you can use the ${BASH_REMATCH[n]}  array to access successful matches (back-references to capture groups). So if you use parentheses for grouping ()  in your regex, you can access the content of that group.

Here is an example where I am parsing date placeholders in a text with an optional offset (e.g. |YYYY.MM.DD|+2 ). Storing the format and offset in separate groups:

 

 

 

Multiply floats by 10,100, … in bash

Ryanbash

A short one today. Bash can only handle integer numbers and not floats, so when someone searches the internet on how to use math on floats in bash the solution they find is usually “use bc” and looks something like this:

Or if they want the result to be an integer:

It’s a fine solution, and readable (which can mean a lot for people maintaining scripts). But if all you want to do is multiply by 10,100,1000, … you can achieve this faster with a bit of string manipulation:

It just splits the number into two strings, and assembles it again with the decimal shifted. Have a look at substring_removal and substring_expansion for more examples on how to modify strings in bash. I’d highly suggest either sticking this in a separate function, or commenting the code since it isn’t necessarily obvious what is going on

Since it is all pure bash and doesn’t need to spawn external commands, it quicker (not that bc  is slow, but if you are doing a lot of calculations, it can add up). I know what you are thinking “if your goal is speed, you shouldn’t be using bash”, that doesn’t mean we can’t write efficient code.

How to fetch IP ranges/entries from SPF records in bash

Ryanbash, SPF

Recently I needed to fetch IP ranges from SPF records. After looking at different python/ruby/perl modules I came to the conclusion that a fancy module (sometimes with wonky dependencies) was overkill just to parse a simple SPF record. So I threw together a simple bash script that is mainly just fetching the SPF record with dig and grep:

It iterates through the options (it currently recognizes a, mx, ip4, ip6, include, and redirect), and then sorts the output by ipv4, then ipv6.

Download URL: fetch_spf.sh

How to compare package version strings in bash

Ryanbash

This is a little function I use to compare package version strings. Sometimes they can get complex with multiple different delimiters or strings in them. I cheated a bit by using sort –version-sort for the actual comparison. If you are looking for a pure bash version to compare simpler strings (e.g. compare 1.2.4 with 1.10.2), I’d suggest this stackoverflow posting.

The function takes three parameters (the version strings and the comparison you want to apply) and uses the return code to signal if the result was valid or not. This gives the function a somewhat natural feel, for example compare_version 3.2.0-113.155 “<” 3.2.0-130.145 would return true. Aside from < and > you can also use a few words like bigger/smaller, older/newer or higher/lower for comparing the strings.

List of return codes and meanings:

 

 

Convert configuration files to ansible templates

Ryanansible, bash, linux, scripting

I’ve been playing around with ansible a lot lately, and I noticed that while changing stuff from “installed and configured manually” to “installed and configured by ansible” I was running into quite a few configuration files that needed to be manually turned into templates. It can be quite tedious to replace values in a configuration file with placeholders and put all those placeholders in a .yml file with default values.
Automating this is something I would have typically done in perl, but since I wanted to learn more about using regex in bash I decided to have a go at it in bash using regex and ${BASH_REMATCH}

The script takes a configuration file and spits out an ansible template, as well as the variable definitions you will need to add to your defaults/main.yml or vars/main.yml

The whole script is a bit to long to post here, but the interesting part is:

(You can download the full script here ansible_template.sh).

You can use regular expressions in a [[ ]] with =~ (e.g. if [[ “boot” =~ ^b ]]), and you can access the result of the regular expression by using ( ) to mark what parts of the result to store and access them via $BASH_REMATCH (comparable to how you would do it for other languages). Here I am parsing out anything that looks like a key=value from the configfile (with multiple possible separators) and storing the results in BASH_REMATCH[1] and BASH_REMATCH[2]

Usage of the script is pretty straightforward. you give it a prefix for the variable names (so you don’t end up with multiple roles all using a common variable name like “port”), and either a local or remote file to work with, and it spits out something like this:

There a tons of different configuration file formats out there so this script won’t work perfectly 100% of the time, but it does do quite well and reduces the manually copy&pasting to a minimum.