Internet Stuff – Page 3

Updated OSSEC Web UI 0.3 files for OSSEC 2.6

June 24, 2012May 13, 2013RyanOSSEC, OSWUI, php, security

OSSEC is an open source HIDS (Host-based Intrusion Detection System), and a pretty darn good one too. It also has a simple web front-end to view what’s going on, search through alerts and stuff like that (called OSSEC Web UI, I’ll just call it “WUI” here). Unfortunately the code is a bit outdated (the last official update was from 2008 as far as I can tell) and it doesn’t support newer features of OSSEC like polling data from a database. Something I’d like to tackle if I find the time 😉

The latest version of OSSEC is 2.6, and due to some small changes to the format of the logs WUI no longer works out-of-the-box. I had a look at the code this weekend and am providing patches and downloads of the files needed to change to get everything running again with OSSEC 2.6.

List of changes:

Works with the OSSEC 2.6 alert log file format
Changed Rule ID Link to better work with the new documentation wiki
Added “user” field to alert output
Widened the layout by a few pixels (to 1000px) and changed the CSS / alert layout to make the individual alerts better readable
Moved some of the hardcoded formatting to CSS

Download changed files
Patch download

Playing around with Exim, Spamassassin, DKIM and SPF

March 30, 2012July 29, 2016RyanDKIM, exim, intenet, SMTP, spamassassin, SPF

I was playing around with my mail server today and decided to have a look at DKIM and SPF. Jump to the bottom for some useful links. You will have to be able to change TXT entries for your domain/subdomains to do any of this. Sending a mail to check-auth@verifier.port25.com will generate an auto-reply with the result of your mail servers settings. I also noticed that Exim wasn’t listening to the IPv6 interface and promptly fixed that, and have received my first mail via IPv6.

Some spamassassin scores for your local/user settings:

score SPF_PASS -1.0
score SPF_FAIL 3.5
score SPF_SOFTFAIL 1.0
score SPF_HELO_PASS -1.1
score SPF_HELO_FAIL 3.5
score SPF_HELO_SOFTFAIL 1.0

score SPF_PASS -1.0

score SPF_FAIL 3.5

score SPF_SOFTFAIL 1.0

score SPF_HELO_PASS -1.1

score SPF_HELO_FAIL 3.5

score SPF_HELO_SOFTFAIL 1.0

SPF:
Syntax: http://www.openspf.org/SPF_Record_Syntax
Tools & Information: http://www.openspf.org/
Wizard: http://www.spfwizard.net/

DKIM:
Exim walkthrough: http://mikepultz.com/2010/02/using-dkim-in-exim/
Domain specific settings: http://www.debian-administration.org/users/lee/weblog/41
Tools: http://dkimcore.org/tools/
Wizard: https://www.port25.com/dkim-wizard/

New Server (with IPv6)

February 18, 2012May 13, 2013Ryanhetzner, ipv6

I got a new server and I finished moving everything to the new hardware this week. If you notice anything behaving strange just drop me a note. I also set up IPv6 on all my VMs so will be playing around with that a bit. It’s a shame that hardly any providers give customers IPv6 addresses, migrating from IPv4 to IPv6 will take forever if none of the users can use them (and in the meantime IPv4 addresses are getting more and more expensive for me).

The new server is from hetzner since I have been very satisfied with them in the past. Impressive hardware features for low prices and (what really makes me keep choosing them) fantastic technical support. I’ve been with a few companies in the past, but none have had as good support as hetzner. Sooner or later hardware will die, and having a hosting provider that responds promptly and knowledgeably to request is priceless.

How to build an efficient GeoIP SQL table

August 22, 2011April 10, 2012Ryan1 Commentbash, GeoIP, MySQL

This here is a very handy little script I threw together to generate a geoip.sql table for quickly determining which country a IP is from. I already hear you saying “Just convert the IP to an INT and use BETWEEN, how hard can it be”. And you are right, that works. And it may even be your easiest solution, but it just isn’t fast. And if you are planning on hammering the table with thousands of queries you are going to end up looking for something fast.

A while back I found a very interesting posting at www.jcole.us that described how to use Spacial Indexes together with MySQL’s GIS to speed up the queries. The posting has been online for a while and both it and the replies are worth reading.

All I did was make a small bash script to download the current “lite” version of GeoIP CSV file from maxmind.com, use the information from the posting to throw/transform it into a local database table and dump out a .sql file that can be easily imported into any other database. The script isn’t failproof though, it expects your user to be able to use mysql and have permission to create databases/tables and “load data local infile”.

generate_geoip_sql.sh

#!/bin/bash
#===============================================================================
#
#          FILE: generate_geoip_sql.sh
#
#         USAGE: ./generate_geoip_sql.sh
#
#   DESCRIPTION: Generates a optimized sql dump with geoipdata from maxmind.com
#
#         USAGE: import sql into a database
#
#  REQUIREMENTS: mysql database rights to create databases and LOAD DATA LOCAL INFILE
#
#       EXAMPLE:
#                 SELECT country_code,country_name
#                 FROM   geo_ip
#                 WHERE  MBRCONTAINS(ip_poly, POINTFROMWKB(POINT(INET_ATON('1.2.3.4'), 0)))
#
#                 SELECT country_code,country_name
#                 FROM   geo_ip
#                 WHERE  INET_ATON('1.2.3.4') 
#                 BETWEEN ip_from AND ip_to;
#
#        AUTHOR:  Ryan Schulze (rs), ryan@dopefish.de
#
#===============================================================================

set -o nounset                              # Treat unset variables as an error

CSVFile="GeoIPCountryWhois.csv"

if [[ -e ${CSVFile} ]]
  then
  echo -en "\n${CSVFile} already exists, download a newer version? [Y/n]: "
  read answer
  if [[ "$(echo $answer|tr "N" "n")" != "n" ]]
  then
    rm ${CSVFile}
  fi
fi

if [[ ! -e ${CSVFile} ]]
then
  tmpfile=$(mktemp)
  wget -O ${tmpfile} http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
  unzip ${tmpfile} &amp;&amp; rm ${tmpfile}
fi

echo -e "\ngenerating geoip.sql"

mysql -e "create database if not exists temp"
mysql -D temp -e "drop table if exists geo_ip"

echo '
CREATE TABLE geo_ip
(
  id           INT UNSIGNED  NOT NULL auto_increment,
  ip_poly      POLYGON       NOT NULL,
  ip_from      INT UNSIGNED  NOT NULL,
  ip_to        INT UNSIGNED  NOT NULL,
  country_code CHAR(2)       NOT NULL,
  country_name CHAR(50)      NOT NULL,
  PRIMARY KEY (id),
  SPATIAL INDEX (ip_poly)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
LOAD DATA LOCAL INFILE "GeoIPCountryWhois.csv"
INTO TABLE geo_ip
FIELDS
  TERMINATED BY ","
  ENCLOSED BY "\""
LINES
  TERMINATED BY "\n"
(
  @ip_from_string, @ip_to_string,
  @ip_from, @ip_to,
  @country_code, @country_string
)
SET
  id      := NULL,
  ip_from := @ip_from,
  ip_to   := @ip_to,
  ip_poly := GEOMFROMWKB(POLYGON(LINESTRING(
    /* clockwise, 4 points and back to 0 */
    POINT(@ip_from, -1), /* 0, top left */
    POINT(@ip_to,   -1), /* 1, top right */
    POINT(@ip_to,    1), /* 2, bottom right */
    POINT(@ip_from,  1), /* 3, bottom left */
    POINT(@ip_from, -1)  /* 0, back to start */
  ))),
  country_code := @country_code,
  country_name := @country_string
;
' | mysql --local-infile=1 -D temp

mysqldump --opt temp geo_ip &gt; geoip.sql

echo -e "\ndone generating geoip.sql"

#!/bin/bash

#===============================================================================

# FILE: generate_geoip_sql.sh

# USAGE: ./generate_geoip_sql.sh

# DESCRIPTION: Generates a optimized sql dump with geoipdata from maxmind.com

# USAGE: import sql into a database

# REQUIREMENTS: mysql database rights to create databases and LOAD DATA LOCAL INFILE

# EXAMPLE:

# SELECT country_code,country_name

# FROM geo_ip

# WHERE MBRCONTAINS(ip_poly, POINTFROMWKB(POINT(INET_ATON('1.2.3.4'), 0)))

# SELECT country_code,country_name

# FROM geo_ip

# WHERE INET_ATON('1.2.3.4')

# BETWEEN ip_from AND ip_to;

# AUTHOR: Ryan Schulze (rs), ryan@dopefish.de

#===============================================================================

set -o nounset # Treat unset variables as an error

CSVFile="GeoIPCountryWhois.csv"

if [[ -e ${CSVFile} ]]

then

echo -en "\n${CSVFile} already exists, download a newer version? [Y/n]: "

read answer

if [[ "$(echo $answer|tr "N" "n")" != "n" ]]

then

rm ${CSVFile}

if [[ ! -e ${CSVFile} ]]

then

tmpfile=$(mktemp)

wget -O ${tmpfile} http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip

unzip ${tmpfile} && rm ${tmpfile}

echo -e "\ngenerating geoip.sql"

mysql -e "create database if not exists temp"

mysql -D temp -e "drop table if exists geo_ip"

echo '

CREATE TABLE geo_ip

(

id INT UNSIGNED NOT NULL auto_increment,

ip_poly POLYGON NOT NULL,

ip_from INT UNSIGNED NOT NULL,

ip_to INT UNSIGNED NOT NULL,

country_code CHAR(2) NOT NULL,

country_name CHAR(50) NOT NULL,

PRIMARY KEY (id),

SPATIAL INDEX (ip_poly)

) ENGINE=MyISAM DEFAULT CHARSET=latin1;

LOAD DATA LOCAL INFILE "GeoIPCountryWhois.csv"

INTO TABLE geo_ip

FIELDS

TERMINATED BY ","

ENCLOSED BY "\""

LINES

TERMINATED BY "\n"

(

@ip_from_string, @ip_to_string,

@ip_from, @ip_to,

@country_code, @country_string

)

SET

id := NULL,

ip_from := @ip_from,

ip_to := @ip_to,

ip_poly := GEOMFROMWKB(POLYGON(LINESTRING(

/* clockwise, 4 points and back to 0 */

POINT(@ip_from, -1), /* 0, top left */

POINT(@ip_to, -1), /* 1, top right */

POINT(@ip_to, 1), /* 2, bottom right */

POINT(@ip_from, 1), /* 3, bottom left */

POINT(@ip_from, -1) /* 0, back to start */

))),

country_code := @country_code,

country_name := @country_string

;

' | mysql --local-infile=1 -D temp

mysqldump --opt temp geo_ip > geoip.sql

echo -e "\ndone generating geoip.sql"

How to add file versions/history to iFolder

July 21, 2011May 13, 2013Ryanfiles, history, iFolder, sync

I’ve been using iFolder for about a month now and am pretty satisfied with the performance and features, but there is one feature I dearly miss: having access to older versions of a file. To work around this problem I simply created a SVN repository for every iFolder user. This allows the users to “check in” files whenever they want to save a specific version of a file, they can tell SVN to ignore directories that they don’t need or want in the SVN system. And you can easily browse through the history of a file, pulling up versions from specific dates, or seeing what changed when in the file (obviously not for binary data).

I am fully aware that SVN may be a bit overkill and that I am only using a very small subset of the SVN features, but on the other hand it is very easy to set up, and there are clients for all the major OS out there that integrate seamlessly into the file managers. So even if I don’t need all the bells and whistles it does what I need it to do without any worries.

As with all workarounds this of course brings some limitations and implications with it.

First of all you will only have the history of a file on systems with a svn client, so no history if you are accessing the files via webinterface.
Situations where multiple people have access to the same files must be set up in the same consellation in SVN to ensure all users have access to file histories
Due to the usage of two systems (iFolder and SVN) the files will be stored in both systems

Depending on your setup and needs the implications may turn out to be a dealbreaker. But if not it may be a viable “addon” to add some features you otherwise missed. I’m not going to go into “how to setup SVN”, since anything I write would be tuned to a specific installation. Just use google, there are plenty of Howtos out there, just find the one that meets your system.