Programming – Page 2 – Ryan Schulze

How to install the latest Nmap for Debian/Ubuntu

August 21, 2013August 27, 2013Ryanbash, nmap, security

A quick & dirty script to download the latest version of nmap (sourcecode) and generate a deb and install it (so that it’s correctly in the package management). Yes, I know this is not much more than a glorified configure && make && checkinstall

#!/bin/bash

# some packages we will need. I put nmap in here to satisfy any dependencies like LUA
apt-get -qqy install checkinstall nmap build-essential

# get latest version
URL='http://nmap.org/dist/'
FILE="$(wget -qO - ${URL}|cut -d\" -f8|grep "tar.bz2"|sort -rn|head -n1)"

INSTALLED="$(dpkg -p nmap|grep Version:|awk '{print $2}')"

echo "Installed version: ${INSTALLED}, Downloaded version: ${FILE/.tar.bz2/}"
read -p "Compile and install new version? [Y/n] " answer
if [[ ${answer,,} = "n" ]] ; then
  exit 0
fi

echo "Downloading ${FILE}"
[[ -e  ${FILE} ]] || wget -qO ${FILE} ${URL}${FILE}
tar -xjf ${FILE}

cd ${FILE/.tar.bz2/}
./configure
if [[ ${FILE} =~ ^nmap-6.40. ]] ; then 
  # fixing an error in the 6.40 ncat Makefile
  sed -i 's/^LUA_LIBS.*/LUA_LIBS = $(top_srcdir)\/..\/liblua\/liblua.a -ldl -lm/1' ncat/Makefile
fi
make &amp;&amp; \
checkinstall -y

echo -e "\n Installed nmap version:"
$(which nmap) --version

#!/bin/bash

# some packages we will need. I put nmap in here to satisfy any dependencies like LUA

apt-get -qqy install checkinstall nmap build-essential

# get latest version

URL='http://nmap.org/dist/'

FILE="$(wget -qO - ${URL}|cut -d\" -f8|grep "tar.bz2"|sort -rn|head -n1)"

INSTALLED="$(dpkg -p nmap|grep Version:|awk '{print $2}')"

echo "Installed version: ${INSTALLED}, Downloaded version: ${FILE/.tar.bz2/}"

read -p "Compile and install new version? [Y/n] " answer

if [[ ${answer,,} = "n" ]] ; then

exit 0

echo "Downloading ${FILE}"

[[ -e ${FILE} ]] || wget -qO ${FILE} ${URL}${FILE}

tar -xjf ${FILE}

cd ${FILE/.tar.bz2/}

./configure

if [[ ${FILE} =~ ^nmap-6.40. ]] ; then

# fixing an error in the 6.40 ncat Makefile

sed -i 's/^LUA_LIBS.*/LUA_LIBS = $(top_srcdir)\/..\/liblua\/liblua.a -ldl -lm/1' ncat/Makefile

make && \

checkinstall -y

echo -e "\n Installed nmap version:"

$(which nmap) --version

Bash snippet, verify ctrl+c

July 10, 2013August 27, 2013Ryanbash, ctrl+c, trap

Lately I’ve been working on a pair of more elaborate scripts using ncat and openssl to transfer data between hosts. I’ll get around to posting it eventually, but until then a few small snippets that people may find useful.

Today we will catch ctrl+c and ask the user if he really want’s to terminate the script.

#!/bin/bash

initialize() { #{{{
  # Treat unset variables as an error
  set -o nounset
  trap verify_quit INT # verify ending script
  trap cleanup TERM EXIT # clean up if script exits
} #}}} 

verify_quit() { #{{{
  local timeout=10
  echo "Press ctrl+c again within ${timeout} seconds to end script"
  sleep ${timeout}
  [[ $? -gt 0 ]] &amp;&amp; cleanup
} #}}}

cleanup() { #{{{
  exit 0
} #}}} 

initialize
while [ true ]
do 
  # do stuff
  sleep 10
done

#!/bin/bash

initialize() { #{{{

# Treat unset variables as an error

set -o nounset

trap verify_quit INT # verify ending script

trap cleanup TERM EXIT # clean up if script exits

} #}}}

verify_quit() { #{{{

local timeout=10

echo "Press ctrl+c again within ${timeout} seconds to end script"

sleep ${timeout}

[[ $? -gt 0 ]] && cleanup

} #}}}

cleanup() { #{{{

exit 0

} #}}}

initialize

while [ true ]

# do stuff

sleep 10

done

The initialize() and cleanup() are my usual function names I have in every script, making sure general settings and variables are defined and that on exit any tempfiles get deleted.
What has been added was a trap for the INT signal (ctrl+c) which calls the verify_quit() function, giving the user 10 seconds to press ctrl+c again to exit (via cleanup()) or return back to wherever we were in the code. There is one unavoidable caveat, the first ctrl+c will kill whatever the script was doing before it jumps into the verify_quit() function.

Simple “try” function for bash

May 13, 2013August 27, 2013Ryanbash

Made a nice little try() function today for simplifying checking/dealing with return codes from commands. It uses the function text() I posted earlier to colorfy output: How to easily add colored text output in bash scripts. The function accepts 2 parameters, how it should behave if a problem occurs and the command to be executed: try <silent|warn|fatal> command

silent: save the return status in the global variable command_status
warn: if the command has a return code > 0, print a warning and save the return status in the global variable command_status
fatal: if the command has a return code > 0, print an error and exit

Obviously not as versatile as a python try/except, bu streamlines verifying the command return codes.
Example Usage:

try warn   false
try fatal  ls -al doesnotexist

1 2	try warn false try fatal ls -al doesnotexist

Output
Warning: ‘false‘ failed with return code –1–
ls: cannot access doesnotexist: No such file or directory
Error: ‘ls -al doesnotexist‘ failed with return code –2–

File: error_handling.sh

command_status=
function try() { #{{{
	if [ "$#" = "0" ]
	then # not enough parameters were provided
		echo "ERROR: sytnax is 'try &lt;silent|warn|fatal&gt; command'"
		exit 1
	fi
	local returncode=0
	local severity=${1}
	shift
	local cmd="${@}"

	$cmd
	returncode=$?

	if [[ ${returncode} -gt 0 ]]
	then
		case ${severity} in
			silent  ) 
				command_status=${returncode}
				;;
			warn  ) 
				printf "%s: '%s' failed with return code -%s-\n" $(text yellow "Warning") "$(text blue "${cmd}")" $(text yellow "${returncode}") 
				command_status=${returncode}
				;;
			fatal ) 
				printf "%s: '%s' failed with return code -%s-\n" $(text red "Error") "$(text blue "${cmd}")" $(text yellow "${returncode}")
				exit ${returncode} 
				;;
			* )
				printf "%s: Wrong usage of the try() function\n" $(text red "Error")
				exit 1
				;;
		esac
	fi
} #}}}

command_status=

function try() { #{{{

if [ "$#" = "0" ]

then # not enough parameters were provided

echo "ERROR: sytnax is 'try <silent|warn|fatal> command'"

exit 1

local returncode=0

local severity=${1}

shift

local cmd="${@}"

$cmd

returncode=$?

if [[ ${returncode} -gt 0 ]]

then

case ${severity} in

silent )

command_status=${returncode}

;;

warn )

printf "%s: '%s' failed with return code -%s-\n" $(text yellow "Warning") "$(text blue "${cmd}")" $(text yellow "${returncode}")

command_status=${returncode}

;;

fatal )

printf "%s: '%s' failed with return code -%s-\n" $(text red "Error") "$(text blue "${cmd}")" $(text yellow "${returncode}")

exit ${returncode}

;;

* )

printf "%s: Wrong usage of the try() function\n" $(text red "Error")

exit 1

;;

esac

} #}}}

How to break down CIDR subnets in Bash

December 11, 2012September 27, 2014Ryanbash, cidr, ip, math, subnet

I was playing around with subnets in bash recently and needed an elegant/easy way to split up a subnet into smaller subnets. First I used 2 functions I found on stackoverflow.com to convert an IP addresse to and from an integer. After that it was just a bit of math in bash to split up any networks too big.
Any network larger than $maxSubnet gets split up.
Here the useful code:

#!/bin/bash
. functions_ip2long.sh

input='10.1.4.0/22 10.2.2.0/24 10.3.3.128/25'
output=
maxSubnet=24

for pair in ${input}
do
  # split up the network and netmask
  network=${pair%/*}
  netmask=${pair#*/}
  if [[ ${netmask} -ge ${maxSubnet} ]]
  then # network is smaller than max. size. add it to the array
    output="${output} ${network}/${netmask}"
  else # network is too big. split it up into smaller chunks and add them to the array
    for i in $(seq 0 $(( $(( 2 ** (${maxSubnet} - ${netmask}) )) - 1 )) )
    do # convert network to long integer, add n * ${maxSubnet} and then convert back to string
      output="${output} $( INET_NTOA $(( $( INET_ATON ${network} ) + $(( 2 ** ( 32 - ${maxSubnet} ) * ${i} )) )) )/${maxSubnet}"
    done
  fi
done

echo ${output}|tr ' ' '\n'

#!/bin/bash

. functions_ip2long.sh

input='10.1.4.0/22 10.2.2.0/24 10.3.3.128/25'

output=

maxSubnet=24

for pair in ${input}

# split up the network and netmask

network=${pair%/*}

netmask=${pair#*/}

if [[ ${netmask} -ge ${maxSubnet} ]]

then # network is smaller than max. size. add it to the array

output="${output} ${network}/${netmask}"

else # network is too big. split it up into smaller chunks and add them to the array

for i in $(seq 0 $(( $(( 2 ** (${maxSubnet} - ${netmask}) )) - 1 )) )

do # convert network to long integer, add n * ${maxSubnet} and then convert back to string

output="${output} $( INET_NTOA $(( $( INET_ATON ${network} ) + $(( 2 ** ( 32 - ${maxSubnet} ) * ${i} )) )) )/${maxSubnet}"

done

echo ${output}|tr ' ' '\n'

Output of script:

10.1.4.0/24
10.1.5.0/24
10.1.6.0/24
10.1.7.0/24
10.2.2.0/24
10.3.3.128/25

10.1.4.0/24

10.1.5.0/24

10.1.6.0/24

10.1.7.0/24

10.2.2.0/24

10.3.3.128/25

How to get the intersecting area of two polygons in MySQL

December 3, 2012September 27, 2014RyanGeometric Objects, Intersection, MySQL, polygon, spatial, stored procedure

I was playing around with spatial features of MySQL this weekend and stumbled into a problem where I was looking for the area of two rectangles that overlap. MySQL provides a function to check if they overlap, but no function to extract the region that overlaps.

I’ve never written a stored routine in MySQL before, so I decided it would be a good exercise to try making one. As you can see the function is pretty straightforward and it assumes you are working with rectangles, but other than that it does what it is supposed to.
You pass the function 2 polygons (e.g. Intersection(a.poly,b.poly)), and it returns the intersecting area as a new polygon.

DROP FUNCTION IF EXISTS Intersection;
DELIMITER //
CREATE FUNCTION Intersection(g1 POLYGON, g2 POLYGON) RETURNS POLYGON DETERMINISTIC
BEGIN
  DECLARE minx1, minx2, maxx1, maxx2 DOUBLE;
  DECLARE miny1, miny2, maxy1, maxy2 DOUBLE;
  DECLARE miny,  maxy,  minx,  maxx DOUBLE;
  SET minx1 =    LEAST(X(PointN(ExteriorRing(g1),1)),X(PointN(ExteriorRing(g1),2)),X(PointN(ExteriorRing(g1),3)),X(PointN(ExteriorRing(g1),4)));
  SET maxx1 = GREATEST(X(PointN(ExteriorRing(g1),1)),X(PointN(ExteriorRing(g1),2)),X(PointN(ExteriorRing(g1),3)),X(PointN(ExteriorRing(g1),4)));
  SET miny1 =    LEAST(Y(PointN(ExteriorRing(g1),1)),Y(PointN(ExteriorRing(g1),2)),Y(PointN(ExteriorRing(g1),3)),Y(PointN(ExteriorRing(g1),4)));
  SET maxy1 = GREATEST(Y(PointN(ExteriorRing(g1),1)),Y(PointN(ExteriorRing(g1),2)),Y(PointN(ExteriorRing(g1),3)),Y(PointN(ExteriorRing(g1),4)));
  SET minx2 =    LEAST(X(PointN(ExteriorRing(g2),1)),X(PointN(ExteriorRing(g2),2)),X(PointN(ExteriorRing(g2),3)),X(PointN(ExteriorRing(g2),4)));
  SET maxx2 = GREATEST(X(PointN(ExteriorRing(g2),1)),X(PointN(ExteriorRing(g2),2)),X(PointN(ExteriorRing(g2),3)),X(PointN(ExteriorRing(g2),4)));
  SET miny2 =    LEAST(Y(PointN(ExteriorRing(g2),1)),Y(PointN(ExteriorRing(g2),2)),Y(PointN(ExteriorRing(g2),3)),Y(PointN(ExteriorRing(g2),4)));
  SET maxy2 = GREATEST(Y(PointN(ExteriorRing(g2),1)),Y(PointN(ExteriorRing(g2),2)),Y(PointN(ExteriorRing(g2),3)),Y(PointN(ExteriorRing(g2),4)));
  SET minx = IF ( minx2 > minx1 , minx2 , minx1 );
  SET maxx = IF ( maxx2 > maxx1 , maxx1 , maxx2 );
  SET miny = IF ( miny2 > miny1 , miny2 , miny1 );
  SET maxy = IF ( maxy2 > maxy1 , maxy1 , maxy2 );
  RETURN POLYGON(LINESTRING(POINT(minx,miny),POINT(maxx,miny), POINT(maxx,maxy), POINT(minx,maxy),POINT(minx,miny)));
END //
DELIMITER ;

DROP FUNCTION IF EXISTS Intersection;

DELIMITER //

CREATE FUNCTION Intersection(g1 POLYGON, g2 POLYGON) RETURNS POLYGON DETERMINISTIC

BEGIN

DECLARE minx1, minx2, maxx1, maxx2 DOUBLE;

DECLARE miny1, miny2, maxy1, maxy2 DOUBLE;

DECLARE miny, maxy, minx, maxx DOUBLE;

SET minx1 = LEAST(X(PointN(ExteriorRing(g1),1)),X(PointN(ExteriorRing(g1),2)),X(PointN(ExteriorRing(g1),3)),X(PointN(ExteriorRing(g1),4)));

SET maxx1 = GREATEST(X(PointN(ExteriorRing(g1),1)),X(PointN(ExteriorRing(g1),2)),X(PointN(ExteriorRing(g1),3)),X(PointN(ExteriorRing(g1),4)));

SET miny1 = LEAST(Y(PointN(ExteriorRing(g1),1)),Y(PointN(ExteriorRing(g1),2)),Y(PointN(ExteriorRing(g1),3)),Y(PointN(ExteriorRing(g1),4)));

SET maxy1 = GREATEST(Y(PointN(ExteriorRing(g1),1)),Y(PointN(ExteriorRing(g1),2)),Y(PointN(ExteriorRing(g1),3)),Y(PointN(ExteriorRing(g1),4)));

SET minx2 = LEAST(X(PointN(ExteriorRing(g2),1)),X(PointN(ExteriorRing(g2),2)),X(PointN(ExteriorRing(g2),3)),X(PointN(ExteriorRing(g2),4)));

SET maxx2 = GREATEST(X(PointN(ExteriorRing(g2),1)),X(PointN(ExteriorRing(g2),2)),X(PointN(ExteriorRing(g2),3)),X(PointN(ExteriorRing(g2),4)));

SET miny2 = LEAST(Y(PointN(ExteriorRing(g2),1)),Y(PointN(ExteriorRing(g2),2)),Y(PointN(ExteriorRing(g2),3)),Y(PointN(ExteriorRing(g2),4)));

SET maxy2 = GREATEST(Y(PointN(ExteriorRing(g2),1)),Y(PointN(ExteriorRing(g2),2)),Y(PointN(ExteriorRing(g2),3)),Y(PointN(ExteriorRing(g2),4)));

SET minx = IF ( minx2 > minx1 , minx2 , minx1 );

SET maxx = IF ( maxx2 > maxx1 , maxx1 , maxx2 );

SET miny = IF ( miny2 > miny1 , miny2 , miny1 );

SET maxy = IF ( maxy2 > maxy1 , maxy1 , maxy2 );

RETURN POLYGON(LINESTRING(POINT(minx,miny),POINT(maxx,miny), POINT(maxx,maxy), POINT(minx,maxy),POINT(minx,miny)));

END //

DELIMITER ;

Example comparing some rectangles in 2 tables using the function:

CREATE TABLE `poly1` (
  `id` int(10) unsigned NOT NULL,
  `poly` polygon NOT NULL,
  PRIMARY KEY (`id`),
  SPATIAL KEY `poly` (`poly`)
) ENGINE=MyISAM;

CREATE TABLE `poly2` (
  `id` int(10) unsigned NOT NULL,
  `poly` polygon NOT NULL,
  PRIMARY KEY (`id`),
  SPATIAL KEY `poly` (`poly`)
) ENGINE=MyISAM;

-- Insert some test data into the tables
insert into poly1 (id,poly) VALUES (1, GEOMFROMWKB(POLYGON(LINESTRING(POINT(1,0),POINT(4,0), POINT(4,2), POINT(1,2),POINT(1,0)))));
insert into poly1 (id,poly) VALUES (2, GEOMFROMWKB(POLYGON(LINESTRING(POINT(6,0),POINT(10,0),POINT(10,2),POINT(6,2),POINT(6,0)))));
insert into poly1 (id,poly) VALUES (3, GEOMFROMWKB(POLYGON(LINESTRING(POINT(10,10),POINT(20,10),POINT(20,5),POINT(10,5),POINT(10,10)))));

insert into poly2 (id,poly) VALUES (1, GEOMFROMWKB(POLYGON(LINESTRING(POINT(1,0),POINT(4,0), POINT(4,2), POINT(1,2),POINT(1,0)))));
insert into poly2 (id,poly) VALUES (2, GEOMFROMWKB(POLYGON(LINESTRING(POINT(3,0),POINT(7,0), POINT(7,2), POINT(3,2),POINT(3,0)))));
insert into poly2 (id,poly) VALUES (3, GEOMFROMWKB(POLYGON(LINESTRING(POINT(5,0),POINT(7,0), POINT(7,2), POINT(5,2),POINT(5,0)))));
insert into poly2 (id,poly) VALUES (4, GEOMFROMWKB(POLYGON(LINESTRING(POINT(8,0),POINT(10,0),POINT(10,2),POINT(8,2),POINT(8,0)))));
insert into poly2 (id,poly) VALUES (5, GEOMFROMWKB(POLYGON(LINESTRING(POINT(3,0),POINT(6,0), POINT(6,2), POINT(3,2),POINT(3,0)))));
insert into poly2 (id,poly) VALUES (6, GEOMFROMWKB(POLYGON(LINESTRING(POINT(5,0),POINT(11,0),POINT(11,2),POINT(5,2),POINT(5,0)))));
insert into poly2 (id,poly) VALUES (7, GEOMFROMWKB(POLYGON(LINESTRING(POINT(12,8),POINT(18,8),POINT(18,4),POINT(12,4),POINT(12,8)))));

SELECT
 a.id as 'poly1',
 b.id as 'poly2',
 cast(100/Area(a.poly)*Area(Intersection(a.poly,b.poly)) as decimal(4,2)) as '% of poly2 in poly1'
FROM
 poly1 a,
 poly2 b
WHERE
 MBROverlaps(b.poly,a.poly);

CREATE TABLE `poly1` (

`id` int(10) unsigned NOT NULL,

`poly` polygon NOT NULL,

PRIMARY KEY (`id`),

SPATIAL KEY `poly` (`poly`)

) ENGINE=MyISAM;

CREATE TABLE `poly2` (

`id` int(10) unsigned NOT NULL,

`poly` polygon NOT NULL,

PRIMARY KEY (`id`),

SPATIAL KEY `poly` (`poly`)

) ENGINE=MyISAM;

-- Insert some test data into the tables

insert into poly1 (id,poly) VALUES (1, GEOMFROMWKB(POLYGON(LINESTRING(POINT(1,0),POINT(4,0), POINT(4,2), POINT(1,2),POINT(1,0)))));

insert into poly1 (id,poly) VALUES (2, GEOMFROMWKB(POLYGON(LINESTRING(POINT(6,0),POINT(10,0),POINT(10,2),POINT(6,2),POINT(6,0)))));

insert into poly1 (id,poly) VALUES (3, GEOMFROMWKB(POLYGON(LINESTRING(POINT(10,10),POINT(20,10),POINT(20,5),POINT(10,5),POINT(10,10)))));

insert into poly2 (id,poly) VALUES (1, GEOMFROMWKB(POLYGON(LINESTRING(POINT(1,0),POINT(4,0), POINT(4,2), POINT(1,2),POINT(1,0)))));

insert into poly2 (id,poly) VALUES (2, GEOMFROMWKB(POLYGON(LINESTRING(POINT(3,0),POINT(7,0), POINT(7,2), POINT(3,2),POINT(3,0)))));

insert into poly2 (id,poly) VALUES (3, GEOMFROMWKB(POLYGON(LINESTRING(POINT(5,0),POINT(7,0), POINT(7,2), POINT(5,2),POINT(5,0)))));

insert into poly2 (id,poly) VALUES (4, GEOMFROMWKB(POLYGON(LINESTRING(POINT(8,0),POINT(10,0),POINT(10,2),POINT(8,2),POINT(8,0)))));

insert into poly2 (id,poly) VALUES (5, GEOMFROMWKB(POLYGON(LINESTRING(POINT(3,0),POINT(6,0), POINT(6,2), POINT(3,2),POINT(3,0)))));

insert into poly2 (id,poly) VALUES (6, GEOMFROMWKB(POLYGON(LINESTRING(POINT(5,0),POINT(11,0),POINT(11,2),POINT(5,2),POINT(5,0)))));

insert into poly2 (id,poly) VALUES (7, GEOMFROMWKB(POLYGON(LINESTRING(POINT(12,8),POINT(18,8),POINT(18,4),POINT(12,4),POINT(12,8)))));

SELECT

a.id as 'poly1',

b.id as 'poly2',

cast(100/Area(a.poly)*Area(Intersection(a.poly,b.poly)) as decimal(4,2)) as '% of poly2 in poly1'

FROM

poly1 a,

poly2 b

WHERE

MBROverlaps(b.poly,a.poly);

Result:

poly1	poly2	% of poly2 in poly1
1	2	33.33
1	5	33.33
2	2	25.00
2	3	25.00
3	7	36.00

poly1 poly2 % of poly2 in poly1

1 2 33.33

1 5 33.33

2 2 25.00

2 3 25.00

3 7 36.00