pantz.org banner
Perl script to output block stats from PF log file
Posted on 10-07-2007 01:13:00 UTC | Updated on 11-01-2008 03:50:30 UTC
Section: /software/pf/ | Permanent Link

My OpenBSD firewalls all have default block policies. Any connection attempt not initiated from the inside will be blocked by the firewall and logged in the PF log files. Usually these hits are people/scripts scanning for open ports. I thought it would be fun to see what ports people are scanning for and what ip's they are from.

To do this I needed a script to parse the log files and add up all blocks. So I made a perl script to do this. I am not a fantastic Perl programmer but the script does what I need it to. There is not much error checking but like I said it works. It is not intended for gigantic log files but it does not do to bad with them. Feel free to make modifications and if your so inclined send them to me. I just might throw them up if I like them. The script runs from a cron job and dumps the stats to an html file on my webservers. I can access it from a browser on the firewall every night to see who's scanning for what. Watch out for the settings like thresholds. If you set them to high you might not get any data back. Same goes for turning on port filtering.

Script Output Example

Pantz PFlog Stats

Log file Data from 10-05-2007 21:00:14 to 10-06-2007 13:24:02

Showing data from 10-06-2007 13:20:00 to 10-06-2007 13:24:00

Script run on: 10-06-2007 17:54:08

Source IP # of blocks in
206.248.132.102 2
221.208.208.95 2
202.97.238.200 1
202.97.238.202 1
82.166.13.106 1
221.208.208.97 1
Destination Port# of blocks in
1026 4
1027 2
7381 2






206.248.132.102
Destination Port # of blocks in
7381 2
202.97.238.200
Destination Port # of blocks in
1026 1
202.97.238.202
Destination Port # of blocks in
1027 1
221.208.208.97
Destination Port # of blocks in
1026 1
82.166.13.106
Destination Port # of blocks in
1026 1
221.208.208.95
Destination Port # of blocks in
1027 1
1026 1






1027
Source IP # of blocks in
202.97.238.202 1
221.208.208.95 1
1026
Source IP # of blocks in
202.97.238.200 1
221.208.208.97 1
82.166.13.106 1
221.208.208.95 1
7381
Source IP # of blocks in
206.248.132.102 2

Below is the code for the perl script that generates the example above. Just configure it and run it. It's pretty well commented on what to do.

#!/usr/bin/perl -s

# This perl script generates some stats (in html format) for OpenBSD's firewall
# PF. It was made with a home firewall in mind. One that has a default block
# in policy. Which means any packets hitting the firewall inbound that were
# not allowed by other rules or not part of another connection don't belong.
# These are the ones we want to see. Most likely these have malicious intent.

# The stats are generated from the binary formated PF logs. OpenBSD's
# implementation of Tcpdump is the only Tcpdump that can read this log. This
# script was tested on an OpenBSD 4.2 machine with the version of PF and
# Tcpdump that comes with 4.2. The script only does TCP and UDP stats
# currently. Also, it only does stats for block in rules and only for
# destination ports and source IP addresses.

# The external programs needed to run this script are: Tcpdump (OpenBSD's
# version), gzcat, and of course Perl. The "host" program is need if you want
# to do hostname lookups. All programs mentioned come with OpenBSD by default.
# You will also need to run this script as root because the permissions on
# the pflog files can only be read by root.

# I have recently tested this script on FreeBSD 6.3 successfully. The
# only thing to watch out for is FreeBSD compresses it's logfiles using 
# bzip2 by default. You will need to change the gzcat line in the script
# to bzcat for it to work. FreeBSD's Tcpdump was modified to work with PF.

# I welcome any additions or added features to the script just send them to
# webmaster at pantz dot org. This code is free to distribute as long as credit
# is given to the website (pantz.org) in the modified code.

use Time::Local;

################################
# Start Configuration Settings #
################################

# You can set any variable below on the command line in the form of: -[variable]=[value]
# Example:  -interface=em0   would set the $interface variable to 'em0'.
# Arrays (begin with @) can't be set from command line.
# Command line settings overides variables set below.

# Interface to be evaluated (usually the external network interface).
$interface = "fxp0" unless $interface;

# Path to output html file.
$pfhtmlfile = "/var/www/pantz.org/stats/pfhtmlstats.html" unless $pfhtmlfile;

# Analyze one file in the dir or all files.
# Set to "one" for one file.
# Set to "all" for all pf files (even compressed ones)
# in the /var/log dir that start with  name "pflog".
$oneorallfiles = "all" unless $oneorallfiles;

# Full path to a single pflog file you want analyzed (compressed or uncompressed).
# Used if var above is set to "one". Example: /home/user/pflog.0.gz
$pflogfile = "/var/log/pflog" unless $pflogfile;

# Set to "exclude" to exclude only certain ports in stats output.
# "include" to include only certian ports in stats output.
# "off" to show all ports in stats output.
$exclude_include_ports = "off" unless $exclude_include_ports;

# Port list (array) for the above setting. For ranges use ".." in between the # range.
# Example of ports 1 thru 10 and 22 and 30 would be: 1..10,22,30
@in_ex_clude_ports = (99..100);

# Threshold values. Will not show or count "# of blocks in" totals from a host
# to a port or ip at or below the value set. Ex. Set to 10. No port or ip hit
# count (per host) that totaled 10 or below is counted.
$src_ip_threshold_value = 0 unless $src_ip_threshold_value;
$src_port_threshold_value = 0 unless $src_port_threshold_value;

# Set to "on" to have hostnames instead of ip's in the output. Set to "off" for just ip addresses.
# If this is turned on the script will take longer to run. Depending on the ammout of unique hosts.
$hostname_lookup = "off" unless $hostname_lookup;

# Filter by date? Settings are: Yes or No.
$date_filter = "no" unless $date_filter;

# Type of filter. Specific date range, current date and time run (now) backwards, or end of the current day backwards.
# Settings are: range, now, or curday.
$filter_type = "now" unless $filter_type;

# If now filter then set Units of time. Settings are: sec, min, hour, or day.
$time_unit = "hour" unless $time_unit;

# Amount of time units. How far back from now or end of current day? Whole number.
$time_amount = "12" unless $time_amount;

# If filtering by date range then specify date range. Form: YearMonthDayHourMinSec.
# Pad all single digits with zero. Ex 5 becomes 05.
# December 6th, 2006 20:05:01 would be: 20061206200501.
$lower_date = 20070205000000 unless $lower_date;
$upper_date = 20070206235959 unless $upper_date;

##############################
# End Configuration Settings #
##############################

#####Begin: Assembling date/time code.#####
$curtimeformat = format_epoch(time);

if ($date_filter eq "yes") {
  if ($filter_type eq "range") {
    ($low_year,$low_month,$low_day,$low_hour,$low_min,$low_sec) = ($lower_date =~ /(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})/);
    ($up_year,$up_month,$up_day,$up_hour,$up_min,$up_sec) = ($upper_date =~ /(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})/);
    # timelocal, it expects values with the same range as those which localtime returns.
    # Namely, months start at 0, and years need 1900 subtracted from them.
    $low_year -= 1900;
    $low_month -= 1;
    $up_year -= 1900;
    $up_month -= 1;
    # DMYHMS to epoch seconds corrected for year and month -1900 and -1 respectively.
    $low_epochsec = timelocal($low_sec, $low_min, $low_hour, $low_day, $low_month, $low_year);
    $up_epochsec = timelocal($up_sec,$up_min,$up_hour,$up_day,$up_month,$up_year);
  } elsif ($filter_type eq "now") {
    $up_epochsec = timelocal(localtime(time));
    calc_low_epoch_sec();
  } elsif ($filter_type eq "curday") {
    ($cur_second, $cur_minute,$cur_hour,$cur_Day,$cur_month,$cur_year) = localtime(time);
    $up_epochsec = timelocal(($cur_second, $cur_minute,$cur_hour,$cur_Day,$cur_month,$cur_year) = (59,59,23,$cur_Day,$cur_month,$cur_year));
    calc_low_epoch_sec();
  } else {
    die ("Date filter was requested but no filter type matched. Check your date filter type.");
  }
}

#####End: Assembling date/time code.#####

#####Begin: Get one or all pflog filenames.#####
if ($oneorallfiles eq "all") {
  @pflogfilenames = </var/log/pflog*>;
  if ($date_filter eq "yes") {
    foreach $pflogfilesstat (@pflogfilenames) {
      ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks) = stat $pflogfilesstat;
      # Don't open log files with dates not in our date range. Using last modify time on log file as date.
      if ( $mtime >= $low_epochsec ) {
        push(@newpflogfilenames, "$pflogfilesstat");
      }
    }
    @pflogfilenames = @newpflogfilenames;
  }
  @pflogfilenames = sort { $b cmp $a } @pflogfilenames;
} else {
  push(@pflogfilenames, "$pflogfile");
}
#####End: Get one or all pflog filenames.#####

#####Begin: Reading pflog file(s) and inputing data for sorting and minipulation.#####

foreach $pflogfilename (@pflogfilenames) {
open(IN, "gzcat -f $pflogfilename | tcpdump -nettqr - 2>&1 |") or die ("Can't open file. Permissions?");
  $isfirstline = 0;
  while( <IN> ) {

    ($line_date,$line_points,$line_rulenum,$line_action,$line_interface,$line_src_host,$line_src_port,$line_dst_host,$line_dst_port,$line_remainder)
    = ($_ =~ /(\d+)\.(\d+) rule (\d+).*\(match\)\:?? (\w+ \w+) \w+ (\w+)\: (\d+\.\d+\.\d+\.\d+)\.(\d+) > (\d+\.\d+\.\d+\.\d+)\.(\d+)\:(.*)/);

    # If line had no match with date it is probably no good. Go to next line. When a line finally matches increase by one.
    next unless $line_date;
    $isfirstline++;

    # Get date from first and last line of each file. Put in array.
    if ($isfirstline == "1" || eof(IN)) { push(@date_array, "$line_date"); }

    next unless (/block in/ && /$interface/);
    $exclude_include_port_match = "no";

    # If using date filter go to the next log line if date does not fall within set filters.
    if ($date_filter eq "yes") {
        next unless ($line_date >= $low_epochsec && $line_date <= $up_epochsec)
    }
    if ($line_remainder =~ /tcp/) {
      if ($exclude_include_ports ne "off") {
        foreach $exin_ports (@in_ex_clude_ports) {
          if ($exin_ports eq $line_dst_port) {
            $exclude_include_port_match = "yes";
          }
        }
        if (($exclude_include_port_match eq "yes") && ($exclude_include_ports eq "include")) {
          push_tcp();
          next;
        }
        if (($exclude_include_port_match eq "no") && ($exclude_include_ports eq "exclude")) {
          push_tcp();
          next;
        }
      } else {
        push_tcp();
        next;
      }
    }
    if ($line_remainder =~ /udp/) {
      if ($exclude_include_ports ne "off") {
        foreach $exin_ports (@in_ex_clude_ports) {
          if ($exin_ports eq $line_dst_port) {
            $exclude_include_port_match = "yes";
          }
        }
        if (($exclude_include_port_match eq "yes") && ($exclude_include_ports eq "include")) {
          push_udp();
          next;
        }
        if (($exclude_include_port_match eq "no") && ($exclude_include_ports eq "exclude")) {
          push_udp();
          next;
        }
      } else {
        push_udp();
        next;
      }
    }
    #if ($line_remainder =~ /icmp/) {
    #  push_icmp();
    #}
  }
close(IN);
}

#####End: Reading pflog file(s) and inputing data for sorting and minipulation.#####

#####Begin: Output of HTML file.#####

open(PFHTMLSTATS, ">$pfhtmlfile") or die ("Can't create html file");

print PFHTMLSTATS qq{
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html><head><title>Pantz PFlog Stats</title>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
</head>
<body style="text-align: center;">
<h2>Pantz PFlog Stats</h2>
};

print PFHTMLSTATS "<p>Log file Data from " . format_epoch($date_array[0]) . " to " . format_epoch($date_array[-1]) . "<br></p>\n";

if ($date_filter eq "yes") {
  print PFHTMLSTATS "<p>Showing data from " . format_epoch($low_epochsec) . " to " . format_epoch($up_epochsec) . "<br></p>\n";
}

print PFHTMLSTATS qq {
<p>Script run on: $curtimeformat <br></p>
<TABLE BORDER="0">
<tr><td valign="top">
<TABLE BORDER="1">
};

# Create host totals. Hash of hash. With threshold evaluation. Lookup host name if set. Sort most to least hits.
for $tcud_src_hstct_key ( keys %tcud_src_hstct ) {
  for $tcud_src_hst_dst_portct_key ( keys %{ $tcud_src_hstct{$tcud_src_hstct_key} } ) {
    # Check if our totals meet the threshold set. If so add it to a total for that ip.
    if ($tcud_src_hstct{$tcud_src_hstct_key}{$tcud_src_hst_dst_portct_key} > $src_ip_threshold_value) {
      $porthitcount += $tcud_src_hstct{$tcud_src_hstct_key}{$tcud_src_hst_dst_portct_key};
    }
  }
  # Lookup hostname if var set. Else set to host's ip.
  if ($hostname_lookup eq "on") {
    hostnamelookup ($tcud_src_hstct_key);
  } else {
    $key_hostname = "$tcud_src_hstct_key";
  }
  # Make hash of arrays. IP, hostname, and hit count. Only if we met thresholds.
  if ($porthitcount != 0) {
    $tcud_src_hst_tot_ct{$tcud_src_hstct_key} = [$key_hostname,$porthitcount];
  }
  $porthitcount = 0;
}

# Print heading line with hostname if var is set. If not just print the ip heading line.
if ($hostname_lookup eq "on") {
  print PFHTMLSTATS "<tr><td><b>Hostname</b></td><td><b>Source IP</b></td>
                     <td><b># of blocks in</b></td></tr>\n";
} else {
  print PFHTMLSTATS "<tr><td><b>Source IP</b>
                     </td><td><b># of blocks in</b></td></tr>\n";
}

# Print hostname, ip, and count if hostname var is set. Otherwise just print ip and count lines.
for $print_tot_src_ip_key1 ( sort {  $tcud_src_hst_tot_ct{$b}[1] <=> $tcud_src_hst_tot_ct{$a}[1] }  keys %tcud_src_hst_tot_ct ) {
  if ($hostname_lookup eq "on") {
    print PFHTMLSTATS "<tr><td>$tcud_src_hst_tot_ct{$print_tot_src_ip_key1}[0]</td>
                       <td><a href=\"#IP:$print_tot_src_ip_key1\">$print_tot_src_ip_key1</a>
                       </td><td>$tcud_src_hst_tot_ct{$print_tot_src_ip_key1}[1]</td></tr>\n";
  } else {
    print PFHTMLSTATS "<tr><td><a href=\"#IP:$print_tot_src_ip_key1\">$print_tot_src_ip_key1</a></td>
                       <td>$tcud_src_hst_tot_ct{$print_tot_src_ip_key1}[1]</td></tr>\n";
  }
}

print PFHTMLSTATS qq{
</table>
</td> <td></td> <td valign="top">
<TABLE BORDER="1">
<tr><td><b>Destination Port</b></td><td><b># of blocks in</b></td></tr>
};

# Create destination port totals. Hash of hash. With threshold evaluation. Sort most to least hits.
for $tcud_dst_portct_key1 ( keys %tcud_dst_portct ) {
  # Check if our totals meet the threshold set. If so add it to a total for that port.
  for $tcud_dst_portct_key2 ( keys %{ $tcud_dst_portct{$tcud_dst_portct_key1} } ) {
    if ($tcud_dst_portct{$tcud_dst_portct_key1}{$tcud_dst_portct_key2} > $src_port_threshold_value) {
      $iphitperportcount += $tcud_dst_portct{$tcud_dst_portct_key1}{$tcud_dst_portct_key2};
    }
  }
  # Create hash of ip and port count. Only if we met thresholds.
  if ($iphitperportcount != 0) {
    $tcud_dst_port_tot_ct{$tcud_dst_portct_key1} = $iphitperportcount;
  }
  $iphitperportcount = 0;
}

# Print the hash from most to least port hits.
foreach $print_tot_dst_port_key (sort { $tcud_dst_port_tot_ct {$b} <=> $tcud_dst_port_tot_ct {$a} } keys %tcud_dst_port_tot_ct) {
  print PFHTMLSTATS "<tr><td><a href=\"#PORT:$print_tot_dst_port_key\">$print_tot_dst_port_key</a></td>
                     <td>$tcud_dst_port_tot_ct{$print_tot_dst_port_key}</td></tr>\n";
}

print PFHTMLSTATS qq {\n</table>\n</td></tr>\n</table>\n\n<p><br><br></p><hr><p><br><br></p>};

$hostportcount = 4;
$hostporttablerowcount = 4;
$was_anything_printed = 0;

print PFHTMLSTATS "\n\n<TABLE BORDER=\"0\">\n\n";

# Start looping thru hash of hash. Print tables if thresholds were met.
for $tcud_src_hstct_key ( keys %tcud_src_hstct ) {
  # Start a new row if our counter is reset.
  if ($hostportcount == $hostporttablerowcount) {
    print PFHTMLSTATS "<tr valign=\"top\">\n\n";
    $hostportcount = 0;
  }
  for $tcud_src_hst_dst_portct_key (sort { $tcud_src_hstct{$tcud_src_hstct_key}{$b} <=> $tcud_src_hstct{$tcud_src_hstct_key}{$a} } keys %{ 
$tcud_src_hstct{$tcud_src_hstct_key} } ) {
    # Check if threshold is met for total host ip hits to a port.
    if ($tcud_src_hstct{$tcud_src_hstct_key}{$tcud_src_hst_dst_portct_key} > $src_ip_threshold_value) {
      # Trip the counter if threshold was met.
      $was_anything_printed++;
      # If this is our first time through and we met the threshold print the table heading.
      if ($was_anything_printed == 1) {
        print PFHTMLSTATS "<td><TABLE BORDER=\"1\">\n";
        print PFHTMLSTATS "<tr><td colspan=\"2\"><a name=\"IP:$tcud_src_hstct_key\"></a>
                           <a href=\"http://ws.arin.net/cgi-bin/whois.pl\?queryinput=$tcud_src_hstct_key\">
                           <b>$tcud_src_hstct_key</b></a></td></tr><tr><td><b>Destination Port</b>
                           </td><td> <b># of blocks in</b></td></tr>\n";
      }
      # Print a table row.
      print PFHTMLSTATS "<tr><td><a href=\"http://isc.sans.org/port_details.php\?port=$tcud_src_hst_dst_portct_key\">
                         $tcud_src_hst_dst_portct_key</a></td><td>
                         $tcud_src_hstct{$tcud_src_hstct_key}{$tcud_src_hst_dst_portct_key}</td>
                         </tr>\n";
    }
  }

  # If something was printed at least once increase column counter. Close our table.
  if ($was_anything_printed > 0) {
    $hostportcount++;
    print PFHTMLSTATS "</table></td><td> </td>\n\n";
  }
  # If host port counter has it its max (set above) close the whole row.
  if ($hostportcount == $hostporttablerowcount) {
    print PFHTMLSTATS "</tr><tr><td> </td></tr>\n\n";
  }
  $was_anything_printed = 0;
}

# If hash ended before row finished then close it.
if ($hostportcount < $hostporttablerowcount) {
  print PFHTMLSTATS "<td> </td></tr>\n";
}


print PFHTMLSTATS "\n</table>\n\n<p><br><br></p><hr><p><br><br></p>\n";

$porthostcount = 4;
$porthosttablerowcount = 4;
$was_anything_printed = 0;

print PFHTMLSTATS "\n\n<TABLE BORDER=\"0\">\n";

for $tcud_dst_portct_key3 ( keys %tcud_dst_portct ) {
  # Start a new row if our counter is reset
  if ($porthostcount == $porthosttablerowcount) {
    print PFHTMLSTATS "<tr valign=\"top\">\n\n";
    $porthostcount = 0;
  }
  for $tcud_dst_portct_key4 (sort { $tcud_dst_portct{$tcud_dst_portct_key3}{$b} <=> $tcud_dst_portct{$tcud_dst_portct_key3}{$a} } keys 
%{$tcud_dst_portct{$tcud_dst_portct_key3} } ) {
    # Check if threshold is met for total host port hits to a ip.
    if ($tcud_dst_portct{$tcud_dst_portct_key3}{$tcud_dst_portct_key4} > $src_port_threshold_value) {
      # Trip the counter if threshold was met.
      $was_anything_printed++;
      # If this is our first time through and we met the threshold print the table heading.
      if ($was_anything_printed == 1) {
        print PFHTMLSTATS "<td><TABLE BORDER=\"1\">\n";
        print PFHTMLSTATS "<tr><td colspan=\"2\"><b><a name=\"PORT:$tcud_dst_portct_key3\"></a>
                           <a href=\"http://isc.sans.org/port_details.php\?port=$tcud_dst_portct_key3\">
                           $tcud_dst_portct_key3</a></b></td></tr><tr><td><b>Source IP</b></td>
                           <td> <b># of blocks in</b></td></tr>\n";
      }
      # Print a table row.
      print PFHTMLSTATS "<tr><td><a href=\"http://ws.arin.net/cgi-bin/whois.pl\?queryinput=$tcud_dst_portct_key4\">
                         $tcud_dst_portct_key4</a></td><td>
                         $tcud_dst_portct{$tcud_dst_portct_key3}{$tcud_dst_portct_key4}</td></tr>\n";
    }
  }
  # If something was printed increase column counter
  if ($was_anything_printed > 0) {
    $porthostcount++;
    print PFHTMLSTATS "</table></td><td> </td>\n\n";
  }
  # If host port counter has it its max (set above) close the whole row.
  if ($porthostcount == $porthosttablerowcount) {
    print PFHTMLSTATS "</tr><tr><td> </td></tr>\n\n";
  }
  $was_anything_printed = 0;
}

# If hash ended before row finished then close it.
if ($porthostcount < $porthosttablerowcount) {
  print PFHTMLSTATS "<td> </td></tr>\n\n";
}

print PFHTMLSTATS "\n</table>\n</body></html>";

close(PFHTMLSTATS);

#####End: Output of HTML file.#####

#####Start: Subroutines.#####
sub push_tcp {
  # Hash of a hash. Value is increasing counter.
  $tcud_src_hstct{$line_src_host}{$line_dst_port}++;
  $tcud_dst_portct{$line_dst_port}{$line_src_host}++;
}
sub push_udp {
  # Hash of a hash. Value is increasing counter.
  $tcud_src_hstct{$line_src_host}{$line_dst_port}++;
  $tcud_dst_portct{$line_dst_port}{$line_src_host}++;
}
sub push_icmp {
  #push(@icmp_src_hst_array, "$line_src_host");
  #push(@icmp_dst_hst_array, "$line_dst_host");
}
sub hostnamelookup {
  # Use the program "host" to do reverse lookups on ip's.
  $key_hostname = $_[0];
  $key_hostname = `host $key_hostname`;
  if ($key_hostname =~ /pointer/) {
    ($key_hostname) = ($key_hostname =~ /pointer (.*)/);
    $key_hostname = substr($key_hostname,0, -1);
  } else {
    $key_hostname = "No PTR Record";
  }
}
sub format_epoch {
  my ($fmt_second, $fmt_minute, $fmt_hour, $fmt_Day, $fmt_month, $fmt_year) = localtime($_[0]);
  $fmt_month += 1;
  $fmt_year += 1900;
  if($fmt_month < 10) { $fmt_month = "0" . $fmt_month; }
  if ($fmt_hour < 10) { $fmt_hour = "0" . $fmt_hour; }
  if ($fmt_minute < 10) { $fmt_minute = "0" . $fmt_minute; }
  if ($fmt_second < 10) { $fmt_second = "0" . $fmt_second; }
  if($fmt_Day < 10) { $fmt_Day = "0" . $fmt_Day; }
  return "$fmt_month-$fmt_Day-$fmt_year $fmt_hour:$fmt_minute:$fmt_second";
}
sub calc_low_epoch_sec {
  if ($time_unit eq "min") { $time_amount *= 60; }
  if ($time_unit eq "hour") { $time_amount *= 3600; }
  if ($time_unit eq "day") { $time_amount *= 86400; }
  return $low_epochsec = $up_epochsec - $time_amount;
}
#####End: Subroutines.#####

Changes

07-01-04 - First change log entry. Added date sort so filenames are read from
           oldest to newest. Done via filenames numbers in order.
         - Fixed date range date so it evaluates all lines not just block in lines.
         - Added interface filter.

11-23-04 - Added port breakdown by IP address's.
         - Added quick navigation to IP address port breakdown.

11-27-04 - Added IP breakdown by port numbers.
         - Added quick navigation to ports IP address breakdown.

12-06-04 - Added IP and Port thresholds for output.

12-01-05 - Fixed pattern matching for PF logs in 3.8.

02-01-06 - Rewrote code to make it simpler and slightly faster.
           Fixed thresholds so all counts reflect correct numbers.
10-25-06 - changed tcpdump switch to dump dates as unix epoch time.
         - changed regex to use new date format.
         - put in "next" lines to skip evaluation of rules we don't want
           or have matched. This increased speed and resource usage.

10-26-06 - Added ability to set config settings from commandline. perl -s.
         - Added basic module to convert DMYHMS to epoch seconds.
         - Added ability to only process lines in a specific date range.

10-06-07 - Fixed html output so it is HTML 4.01 Strict.

11-12-07 - Fixed problem where if first line of a log file was not a matched
           line the date would not be set correctly for the "Log data from" line.

10-31-08 - Fixed include exclude ports that was broken because I forgot to rename
           some old vars to the new name.

Possible To Do's

1. Number of blocks in by days of the week or just date. Do a ascii graphic with symbol.
2. Do something with ICMP.
3. Keep cache of stats from last run to help with large log files.
4. Check log file extention. If it is bzip2 then use bzcat instead of zcat.

Del.icio.us! | Digg Me! | Reddit!

Related stories


RSS Feed RSS feed logo
About


3com
3ware
alsa
alsactl
alsamixer
amd
android
apache
areca
arm
ati
auditd
awk
badblocks
bash
bind
bios
bonnie
cable
carp
cat5
cdrom
cellphone
centos
chart
chrome
cifs
cisco
cloudera
comcast
commands
comodo
compiz-fusion
corsair
cpufreq
cpufrequtils
cpuspeed
cron
crontab
crossover
cu
cups
cvs
database
dbus
dd
dd_rescue
ddclient
debian
decimal
dhclient
dhcp
diagnostic
diskexplorer
disks
dkim
dns
dos
dovecot
drac
dsniff
dvdauthor
e-mail
echo
editor
emerald
ethernet
expect
ext3
ext4
fat32
fedora
fetchmail
fiber
filesystems
firefox
firewall
flac
flexlm
floppy
flowtools
fonts
format
freebsd
ftp
gdm
gmail
gnome
greasemonkey
greylisting
growisofs
grub
hacking
hadoop
harddrive
hba
hex
hfsc
html
html5
http
https
idl
ie
ilo
intel
ios
iperf
ipmi
iptables
ipv6
irix
javascript
kde
kernel
kickstart
kmail
kprinter
krecord
kubuntu
kvm
lame
ldap
linux
logfile
lp
lpq
lpr
maradns
matlab
memory
mencoder
mhdd
mkinitrd
mkisofs
moinmoin
motherboard
mouse
movemail
mplayer
multitail
mutt
myodbc
mysql
mythtv
nagios
nameserver
netflix
netflow
nginx
nic
ntfs
ntp
nvidia
odbc
openbsd
openntpd
openoffice
openssh
openssl
openvpn
opteron
parted
partimage
patch
perl
pf
pfflowd
pfsync
photorec
php
pop3
pop3s
ports
postfix
power
procmail
proftpd
proxy
pulseaudio
putty
pxe
python
qemu
r-studio
raid
recovery
redhat
router
rpc
rsync
ruby
saltstack
samba
schedule
screen
scsi
seagate
seatools
sed
sendmail
sgi
shell
siw
smtp
snort
solaris
soundcard
sox
spam
spamd
spf
sql
sqlite
squid
srs
ssh
ssh.com
ssl
su
subnet
subversion
sudo
sun
supermicro
switches
symbols
syslinux
syslog
systemrescuecd
t1
tcpip
tcpwrappers
telnet
terminal
testdisk
tftp
thttpd
thunderbird
timezone
ting
tls
tools
tr
trac
tuning
tunnel
ubuntu
unbound
vi
vpn
wget
wiki
windows
windowsxp
wireless
wpa_supplicant
x
xauth
xfree86
xfs
xinearama
xmms
youtube
zdump
zeromq
zic
zlib