pantz.org banner
Configuration of NetFlow, Flowtools, pfflowd on OpenBSD
Posted on 05-16-2006 06:23:00 UTC | Updated on 04-05-2013 21:07:23 UTC
Section: /software/flowtools/ | Permanent Link

Have you ever wanted to keep track of every packet going through your firewall? How about getting some stats on the hosts using your network. Stats like most bandwidth used or most popular ports or ip's. Well NetFlow is what your looking for. NetFlow is an open but proprietary network protocol developed by Cisco Systems to run on Cisco IOS-enabled equipment for collecting IP traffic information.

Flow-tools is a suite of programs that takes Cisco NetFlow datagrams and generates reports based on the information it has recorded. The Cisco Netflow datagrams are generated by a program called Pfflowd.

The Pfflowd website describes it's programs functionality the best by saying: Converts OpenBSD PF status messages (sent via the pfsync interface) to Cisco NetFlow datagrams. These datagrams may be sent (via UDP) to a host of one's choice. Utilising the OpenBSD stateful packet filter infrastructure means that flow tracking is very fast and accurate.

Note: The PFflowd only accounts packets that get passed statefully. Since flow reporting is coupled to PF's state tracking, only traffic flows which are passed via a "keep state", "modulate state" or "synproxy state" rule are accounted. Blocked packets are not accounted for.

Below are some simple steps to getting this going. This install was done on OpenBSD 3.8.

1. Install python 2.4 from the version of OpenBSD your using. Follow the install instructions and make it the default Python for the system.

pkg_add -v http://openbsd.mirrors.pair.com/ftp/3.9/packages/i386/python-2.4.2p0.tgz

2. Install the Pfflowd package on any version of OpenBSD before 3.9.

pkg_add -v http://openbsd.mirrors.pair.com/ftp/3.8/packages/i386/pfflowd-0.6.tgz

If your using OpenBSD 3.9 then you will need to download the latest version from CVS from the authors website. A fix was committed to CVS on the authors site in June 2006 to fix it. If you need that the site is here. Just download the Makefile, pfflowd.8, pfflowd.c, and pfflowd.h into a directory. Then just issue the "make" command. It should compile no problem in OpenBSD 3.9. Then copy the made binary to /usr/local/sbin/ and the man page to /usr/local/man/man8/. Last step is to create the user "_pfflowd". If you don't then Pfflowd will not start.

3. Install the flow-tools .68 or higher. As of OpenBSD 3.8 & 3.9 you will need to download the source and compile it as both only have version .67. The reason for this is we will need the new flow-rptfmt program that only comes with .68 and later. You can get the source from the flow-tools site. If any later version of OpenBSD comes with .68 or later just install the package and forget the source install. The commands to make and install flow-tools are below. This will install flow-tools to /usr/local/netflow.

cd /tmp
wget ftp://ftp.eng.oar.net/pub/flow-tools/flow-tools-0.68.tar.gz
tar xvzf flow-tools-0.68.tar.gz
cd flow-tools-0.68
./configure
make
make install

4. Let's make the directory for the log files then start the flow-tools collector. This will listen on localhost for Netflow datagrams sent from Pfflowd. It uses the program flow-capture from the flow-tools package to collect datagrams sent from Pfflowd. It will keep binary logs of the collected datagrams in /var/log/netflow. It will begin a new file every 10 mins. It listens on localhost port 12345. Make sure you allow this in your PF rules if need be.

mkdir /var/log/netflow
/usr/local/netflow/bin/flow-capture -w /var/log/netflow -n 143 127.0.0.1/0/12345

5. Let's start Pfflowd so it will start sending data to the flow-capture daemon. This starts the Pfflowd daemon which will send Netflow datagrams to the flow-tools daemon on localhost port 12345. You might need bring up the pfsync0 interface so this will work. So we will do that also. To get it to come up on reboot just make the file /etc/hostname.pfsync0 and put the work "up" on the first line. Then save it.

ifconfig pfsync0 up
/usr/local/sbin/pfflowd -n 127.0.0.1:12345

6. Now that it's collecting data we will need to create a config file that will filter the data before we generate a report. The following lines need to be put in a file to be accessed by reports.conf config file. The name of my file is nfilter.conf. This is an example of a filter that could be used to narrow down the logs to ip ranges and ports if you would like to.

############################
filter-primitive popularports
 type ip-port
  permit 80
  permit 443
  permit 20
  permit 21
  permit 22
  permit 25
  permit 53
  permit 110
  permit 143
  default deny

filter-primitive intif
 type ip-address-prefix
  permit 192.168.0.0/24
 default deny

filter-primitive intif-ip
 type ip-address
  permit 192.168.0.10
  default deny

filter-primitive 10min-ago
 type time-date
 permit lt 600 seconds ago

##############################
filter-definition intif-src
 match ip-source-address intif

filter-definition intif-dst
 match ip-destination-address intif

filter-definition intif-src-popular
 match ip-source-address intif
 match ip-destination-port popularports

filter-definition intif-ip1
 match ip-source-address  intif-ip

7. Now that we have filters for the data we need to generate the reports from the filtered data. The following lines need to be put in a file and called from the command line. I named mine reports.conf.

#The filter lines below refer to this file
include-filter /etc/nfilter.conf

#### Start: Source IP -> Dest Port. Filter Name: intif-src ####
stat-report srcip-dstprt
 type ip-source-address/ip-destination-port
  filter intif-src
 output
  sort +octets
  records 15
stat-definition srcip-dstprt
 report srcip-dstprt
#### End: Source IP -> Dest Port. Filter Name: intif-src ####

#### Start: Destination IP -> Dest Port. Filter Name: intif-dst ####
stat-report dstip-dstprt
 type ip-destination-address/ip-destination-port
  filter intif-dst
 output
  sort +octets
  records 15
stat-definition dstip-dstprt
 report dstip-dstprt
#### End: Destination IP -> Dest Port. Filter Name: intif-dst ####
#### Start: Source IP -> Dest Port. Filter Name: intif-src-popular ####
stat-report srcip-dstprt-pop
 type ip-source-address/ip-destination-port
  filter intif-src-popular
 output
  sort +octets
  records 15
stat-definition srcip-dstprt-pop
 report srcip-dstprt-pop
#### End: Source IP -> Dest Port. Filter Name: intif-src-popular ####

8. Now we need a script to tie all this together so it runs the programs from flow-tools to generate the reports in html format. This is a korn shell script. Change the variables on the top to fit your needs. The output is very simple and can be changed to suit your needs. Save this to a file and execute it (chmod +x filename if need be). This should dump a very simple html file to the path you set in the script. It will show the output from the current day it is run.

#!/bin/sh

# Add to your path statement in /root/.profile: /usr/local/netflow/bin/

# Variables for settings below
REPORTPATH=/var/www/htdocs
CFGFILES=/etc
LOGPATH=/var/log/netflow
HTMLFILENAME=netflow.shtml
YEAR=`date "+%Y"`
MONTH=`date "+%m"`
DAY=`date "+%d"`

createreport () {
 flow-cat $LOGPATH/$YEAR/$YEAR-$MONTH/$YEAR-$MONTH-$DAY/ \
 |flow-report -s$CFGFILES/reports.conf  -S $1 \
 |flow-rptfmt -fhtml >>$REPORTPATH/$HTMLFILENAME
 echo "<br><br><br>" >>$REPORTPATH/$HTMLFILENAME
}

# Create html file
echo "<html><center>">$REPORTPATH/$HTMLFILENAME

# Run Create file function with report name
createreport dstip-dstprt
createreport srcip-dstprt
createreport srcip-dstprt-pop

# Close html file
echo "</center></html>">>$REPORTPATH/$HTMLFILENAME

9. Once you have verfied the script works then put it in a cron job to run every night. Since the script looks at the current day it is run I like to run mine right before the end of the day. Use the command "crontab -e" and insert the line below. This runs the script at 11:51 PM every night.

51      23      *       *       *       /var/www/cgi-bin/reports.sh >>/dev/null 2>&1

View the file netflow.shtml your favorite web browser to see the stats.

That is a simple example. But I wanted an email with the stats every night. I did not find a way to get it into a nice format so I wrote a perl script to put the output in a nice e-mail friendly format. The script takes the output from flow-report and gives the "octets" column more human readable numbers in Bytes, Megabytes, and Gigabytes. It also lines up all the columns neatly and just spits out acsii. You can then redirect the output of the script to an email that can be sent each day. The script will read all of the current days (the day it's run) stats. It's rough but it gets the job done. Just run the script by itself to see the output on the command line. Don't forget to put in the report file below in /etc/

Below is the report file I use. I use the file name and path /etc/report.cfg. There is no filter being used.

stat-report ip-address
 type ip-address
 output
  sort +octets
  records 15
stat-definition ip-address
 report ip-address

stat-report ip-source-address
 type ip-source-address
 output
  sort +octets
  records 15
stat-definition ip-source-address
 report ip-source-address

stat-report ip-destination-address
 type ip-destination-address
 output
  sort +octets
  records 15
stat-definition ip-destination-address
 report ip-destination-address

stat-report ip-source/destination-address
 type ip-source/destination-address
 output
  sort +octets
  records 15
stat-definition ip-source/destination-address
 report ip-source/destination-address

stat-report ip-source-port
 type ip-source-port
 output
  sort +octets
  records 15
stat-definition ip-source-port
 report ip-source-port

stat-report ip-destination-port
 type ip-destination-port
 output
  sort +octets
  records 15
stat-definition ip-destination-port
 report ip-destination-port

stat-report ip-port
 type ip-port
 output
  sort +octets
  records 15
stat-definition ip-port
 report ip-port

stat-report ip-source/destination-port
 type ip-source/destination-port
 output
  sort +octets
  records 15
stat-definition ip-source/destination-port
 report ip-source/destination-port

#########

stat-report ip-source-address/ip-source-port
 type ip-source-address/ip-source-port
 output
  sort +octets
  records 15
stat-definition ip-source-address/ip-source-port
 report ip-source-address/ip-source-port

stat-report ip-source-address/ip-destination-port
 type ip-source-address/ip-destination-port
 output
  sort +octets
  records 15
stat-definition ip-source-address/ip-destination-port
 report ip-source-address/ip-destination-port

stat-report ip-destination-address/ip-source-port
 type ip-destination-address/ip-source-port
 output
  sort +octets
  records 15
stat-definition ip-destination-address/ip-source-port
 report ip-destination-address/ip-source-port

stat-report ip-destination-address/ip-destination-port
 type ip-destination-address/ip-destination-port
 output
  sort +octets
  records 15
stat-definition ip-destination-address/ip-destination-port
 report ip-destination-address/ip-destination-port

stat-report ip-source-address/ip-source/destination-port
 type ip-source-address/ip-source/destination-port
 output
  sort +octets
  records 15
stat-definition ip-source-address/ip-source/destination-port
 report ip-source-address/ip-source/destination-port

stat-report ip-destination-address/ip-source/destination-port
 type ip-destination-address/ip-source/destination-port
 output
  sort +octets
  records 15
stat-definition ip-destination-address/ip-source/destination-port
 report ip-destination-address/ip-source/destination-port

stat-report ip-source/destination-address/ip-source-port
 type ip-source/destination-address/ip-source-port
 output
  sort +octets
  records 15
stat-definition ip-source/destination-address/ip-source-port
 report ip-source/destination-address/ip-source-port

stat-report ip-source/destination-address/ip-destination-port
 type ip-source/destination-address/ip-destination-port
 output
  sort +octets
  records 15
stat-definition ip-source/destination-address/ip-destination-port
 report ip-source/destination-address/ip-destination-port

stat-report ip-source/destination-address/ip-source/destination-port
 type ip-source/destination-address/ip-source/destination-port
 output
  sort +octets
  records 15
stat-definition ip-source/destination-address/ip-source/destination-port
 report ip-source/destination-address/ip-source/destination-port

Below is the perl script that uses the above report config file. Change the variables at the top to what you want. Currently it's set to look for the report file /etc/report.cfg. It is also set to look in the path /var/log/flow for it's log files.

#!/usr/bin/perl
# Make sure all the flow-tools binaries are in your shells path.

$cfg_file="/etc/report.cfg";
$log_path="/var/log/flow";

# Get and fix up current date.
($Second, $Minute, $Hour, $Day, $Month, $Year, $WeekDay, $DayOfYear, $IsDaylightSavings) = localtime(time);
$Month += 1;
$Year += 1900;
if($Month < 10) { $Month = "0" . $Month; }
if ($Hour < 10) { $Hour = "0" . $Hour; }
if ($Minute < 10) { $Minute = "0" . $Minute; }
if ($Second < 10) { $Second = "0" . $Second; }
if($Day < 10) { $Day = "0" . $Day; }

# Run Create file function with report name
createreport ("ip-source-address");
createreport ("ip-destination-address");
createreport ("ip-address");
createreport ("ip-source/destination-address");
createreport ("ip-source-port");
createreport ("ip-destination-port");
createreport ("ip-port");
createreport ("ip-source/destination-port");
createreport ("ip-source-address/ip-source-port");
createreport ("ip-source-address/ip-destination-port");
createreport ("ip-destination-address/ip-source-port");
createreport ("ip-destination-address/ip-destination-port");
createreport ("ip-source-address/ip-source/destination-port");
createreport ("ip-destination-address/ip-source/destination-port");
createreport ("ip-source/destination-address/ip-source-port");
createreport ("ip-source/destination-address/ip-destination-port");
createreport ("ip-source/destination-address/ip-source/destination-port");

sub createreport {
  @reportarray = (); #clear array
  open(IN, "flow-cat $log_path/$Year/$Year-$Month/$Year-$Month-$Day/ |flow-report -s$cfg_file -S $_[0] |") or die ("Can't open file. Permissions?");
    while( <IN> ) {
      if ( /# recn: / || !/^#/ ) {
        s/# recn: //;
        chomp ();
        push @reportarray, [ split "," ]; # Split elements into an array of arrays.
      }
    }
  close(IN);
  printout (); #print each report using the print subroutine
}


sub printout {
  # loop through each element in first line. count postion's over till we hit octets. exit loop.
  my $octethit=0;
  for my $titlecount ( @{$reportarray[0]} ) {
    if ( $titlecount eq "octets" ) {
      last;
    } else {
      $octethit++;
    }
  }

  # skip first array. then use octets count from above to find octets line. convert that to kb,mb,gb. write back to array.
  for my $i ( 1 .. $#reportarray ) {
    for my $j ( 0 .. $#{$reportarray[$i]} ) {
      if ( $j == $octethit ) {
        my $kilo = sprintf ("%.4f", ($reportarray[$i][$j]/1024));
        my $kiloint = $reportarray[$i][$j]/1024;
        my $mega = sprintf ("%.4f", ($kilo/1024));
        my $megaint = $kilo/1024;
        my $giga = sprintf ("%.4f", ($mega/1024));
        my $gigaint = $mega/1024;

        if ( $kiloint < 1 ) {
          $reportarray[$i][$j] = "$reportarray[$i][$j] BT";
        } elsif ( $megaint < 1 ) {
          $reportarray[$i][$j] = "$kilo KB";
        } elsif ( $gigaint < 1 ) {
          $reportarray[$i][$j] = "$mega MB";
        } else {
          $reportarray[$i][$j] = "$giga GB";
        }
          next;
      }
    }
  }

  @maxlength = (); # clear array
  for my $ii ( 0 .. $#reportarray ) {
    for my $jj ( 0 .. $#{$reportarray[$ii]} ) {
      $tmplength = length($reportarray[$ii][$jj]);
      if ( $tmplength > $maxlength[$jj] ) {
         $maxlength[$jj] = $tmplength;
      }
    }
  }

  # print to stdout all lines
  for my $q ( 0 .. $#reportarray ) {
    for my $r ( 0 .. $#{$reportarray[$q]} ) {
      my $diff = $maxlength[$r] - length ($reportarray[$q][$r]);
      print "$reportarray[$q][$r]  ";
      print ' ' x $diff;
    }
    print "\n";
  }
  print "\n";
}

This is the line from cron that emails the stats: /var/www/cgi-bin/flow-rptfmtprint.pl | /usr/bin/mailx -s "`uname -n` daily flow stats" root

Del.icio.us! | Digg Me! | Reddit!

Related stories


RSS Feed RSS feed logo
About


3com
3ware
alsa
alsactl
alsamixer
amd
android
apache
areca
arm
ati
auditd
awk
badblocks
bash
bind
bios
bonnie
cable
carp
cat5
cdrom
cellphone
centos
chart
chrome
cifs
cisco
cloudera
comcast
commands
comodo
compiz-fusion
corsair
cpufreq
cpufrequtils
cpuspeed
cron
crontab
crossover
cu
cups
cvs
database
dbus
dd
dd_rescue
ddclient
debian
decimal
dhclient
dhcp
diagnostic
diskexplorer
disks
dkim
dns
dos
dovecot
drac
dsniff
dvdauthor
e-mail
echo
editor
emerald
ethernet
expect
ext3
ext4
fat32
fedora
fetchmail
fiber
filesystems
firefox
firewall
flac
flexlm
floppy
flowtools
fonts
format
freebsd
ftp
gdm
gmail
gnome
greasemonkey
greylisting
growisofs
grub
hacking
hadoop
harddrive
hba
hex
hfsc
html
html5
http
https
idl
ie
ilo
intel
ios
iperf
ipmi
iptables
ipv6
irix
javascript
kde
kernel
kickstart
kmail
kprinter
krecord
kubuntu
kvm
lame
ldap
linux
logfile
lp
lpq
lpr
maradns
matlab
memory
mencoder
mhdd
mkinitrd
mkisofs
moinmoin
motherboard
mouse
movemail
mplayer
multitail
mutt
myodbc
mysql
mythtv
nagios
nameserver
netflix
netflow
nginx
nic
ntfs
ntp
nvidia
odbc
openbsd
openntpd
openoffice
openssh
openssl
opteron
parted
partimage
patch
perl
pf
pfflowd
pfsync
photorec
php
pop3
pop3s
ports
postfix
power
procmail
proftpd
proxy
pulseaudio
putty
pxe
python
qemu
r-studio
raid
recovery
redhat
router
rpc
rsync
ruby
saltstack
samba
schedule
screen
scsi
seagate
seatools
sed
sendmail
sgi
shell
siw
smtp
snort
solaris
soundcard
sox
spam
spamd
spf
sql
sqlite
squid
srs
ssh
ssh.com
ssl
su
subnet
subversion
sudo
sun
supermicro
switches
symbols
syslinux
syslog
systemrescuecd
t1
tcpip
tcpwrappers
telnet
terminal
testdisk
tftp
thttpd
thunderbird
timezone
ting
tls
tools
tr
trac
tuning
tunnel
ubuntu
unbound
vi
vpn
wget
wiki
windows
windowsxp
wireless
wpa_supplicant
x
xauth
xfree86
xfs
xinearama
xmms
youtube
zdump
zeromq
zic
zlib