pantz.org banner
Setting up spamd
Posted on 12-02-2005 03:33:00 UTC | Updated on 12-02-2005 03:33:00 UTC
Section: /software/spamd/ | Permanent Link

Spamd is a fake sendmail-like daemon created by the OpenBSD developers which rejects false mail (spam). It will attempt to waste the time and resources of the spam sender. Spamd is designed to work in conjunction with OpenBSD's pf, and should function on any system that has access to pf. The way it will be implimented below is through a feature called greylisting. It will use black, white, and grey lists to determine weather or not to pass the mail onto the actual mail server.

The first mail delivery attempt (by a server that has never attempted delivery before) connects to port 25 is redirected to port 8025 on localhost. This is done with pf redirect rules. If the delivering server is not currently in the whitelist pf table it goes directly to spamd. The delivering mail server is denied delivery by spamd and told to come back at a later time. The mail servers visit is logged in spamd database and a timer starts. During this time the marked server is not allowed to deliver it's mail. It can come back as many times as it wants in this time period but will always be denied delivery. The connecting server is in what is called a "grey state" and is put on a greylist. This means it is not blacklisted (never allowed) nor whitelisted (always allowed).

Once our specified deny time period ends the delivering mail server will hopefully attempt another delivery (most legit servers will). When it does it will be denied delivery once again. The difference this time is it's status in the spamd database will be changed from grey to white. It's ip address is put in the pf whitelist table and any delivery from this server is redirected straight to the mail server (port 25) on any future delivery attempts. The future delivery attempts for a whitelisted server is done via the whitelist table in pf. Once spamd puts the server's ip in this table the best match in the firewall rules is the whitelist table match. Any servers in this table are redirected to the mail server.

The pf whitelist table is mantained by the spamd deamon. It will insert and remove entries based on it's database. Spamd removes whitelist entries from the spamd database if no mail delivery activity has been seen from the whitelisted address by spamlogd within our specified number of hours from the initial time an address is whitelisted.

After being white listed the delivering server will try another delivery attempt. This time it will be redirected right to mail server.

Spamd is installed on OpenBSD by default. The following are configuration options we are going to use to run spamd. The "-5" option is for black-listed entries, it returns an error code 550 to the spammer. The "-S12" option will stutter greylisted connections for the specified amount of seconds, after which the connection is not stuttered. The next option is "-G25:4:864". The first 25 stands for pass time in mins. Spamd will not accept mail from the initial connection attempt time up to this set time (25). At this point the server is grey listed. The second number 4 is called grey expiration time (hrs). This is the maximum amount of time the server to try to make another delivery after the 25 min pass time period. So, the server can attempt a delivery between 25 mins and 4 hrs after the initial connection attempt and it will be whitelisted. But remember when it connects during this time period it will be denied and whitelisted at the same time. It will have to come back one last time to delivery it's mail. It can come back any time after it is whitelisted. The last number is called the whitelist experation time. It is the time you have in hours to send a message to the server after you have been whitelisted. If you don't send a message to the server from the time your whitelisted up to this time then you will be removed from the whitelist and have to go through the whole filtering process over again. "-b" tells spamd what ip to bind to. "-v" turns on verbose logging. "-nOpenMailServer" is the SMTP version banner that is reported upon initial connection. Change that to whatever you want. The spamd line is put in the /etc/rc.conf file. An example of the revelant lines from this file are below.

The /etc/rc.conf below has pf being turned on and points to it's config file. It will also turn on spamd (in greylisting mode) and sendmail (or if you installed and configured postfix then that). The rest of the lines are not needed to get spamd working but are there for reference.

# These are the settings needed for spamd and postfix and or sendmail to work.
pf=YES
pf_rules=/etc/pf.conf
sendmail_flags="-bd -q30m"
spamd_grey=YES
spamd_flags="-5 -S20 -G25:4:864 -b127.0.0.1 -v -nOpenMailServer"
syslogd_flags="-a /var/spool/postfix/dev/log"

# These settings are not needed to get spamd and postfix working they are just what I use.
# The rest of the options in the rc.conf are either left blank or turned off.
dhcpd_flags="fxp1"
ntpd_flags="-s"
sshd_flags="-4"

These are the lines you will need in your pf.conf file to get spamd working. Only the lines that involve spamd are below. You will have to fit these into your ruleset in the correct order (Macros, tables, options, traffic normalization, queuing, traslation, and packet filtering). The comments explain what the rules do. I have left in the rules for my MTA Postfix as an example of what you would need to do when using spamd and Postfix together. Postfix and spamd listen on localhost and connections for both are redirected to localhost from the outside. After putting the rules in restart PF (pfctl -f /etc/pf.conf).

# use a macro for the interface name, so it can be changed easily
ext_if = "fxp0"

# Spamd whitelist table
table <spamd-white> persist

# Outside servers whitelist. Uncomment if you use a generated whitelist like in script at bottom of page.
# table <whitelist> persist file "/etc/whitelist.txt"

# Outside servers whitelist redirect to postfix. Uncomment if used/needed. Line above goes with this.
# rdr on $ext_if proto tcp from  <whitelist> to ($ext_if) port smtp -> lo0 port smtp

# Spamd and Postfix redirect on external interface. 
# If in whitelist table go to Postfix. If not in whitelist table go to spamd.
rdr on $ext_if proto tcp from  <spamd-white> to ($ext_if) port smtp -> lo0 port smtp
rdr on $ext_if proto tcp from !<spamd-white> to ($ext_if) port smtp -> lo0 port spamd

# Postfix Mail Server (internal interface) redirect for internal hosts.
rdr on $int_if proto tcp from any to  $int_if  port smtp -> lo0 port smtp

# Allow traffic for spamd from outside. Synproxy the connection.
pass in quick on $ext_if inet proto tcp from any to lo0 port spamd flags S/SAFPRU synproxy state

# Allow traffic for postfix mail server from anyone on the outside and the inside.
pass in log on $ext_if inet proto tcp from any to lo0 port smtp flags S/SAFPRU synproxy state
pass in     on $int_if inet proto tcp from any to lo0 port smtp flags S/SAFPRU keep     state

Setup and save a very simple /etc/spamd.conf file for now. We will do blacklist checks with the MTA.

all:\
        ::

You should be able to just reboot your machine to have all of these services start. I would suggest doing it to be sure they all start when the machine is rebooted. After rebooting you should see spamd (spamd,tcpdump) running. Hopefully you will see your MTA running also (postfix or sendmail or whatever). Tcpdump is running for spamd to watch incoming connections from mail servers. It uses it to keep the spam database (spamdb) up to date.

After being setup you can try to send an e-mail to your machine. Here's to hoping you setup all your dns records correctly. If you did and the e-mail gets to your machine spamd should respond with the a general message of "Come back later". Now you have an entry in your spam database (spamdb). To check this just execute the program: spamdb. You will see output like the following below.

GREY|10.20.30.40|||1152284462|1152298862|1152298862|1|0

Then after comming back after our pass time but less then our greyexp time you will see a whitelist entry like the following

WHITE|10.20.30.40|||1152284462|1152286088|1155396513|2|0

For GREY or WHITE entries, the format is:

type|source ip|from|to|first|pass|expire|block|pass

The fields are as follows:

type       WHITE if whitelisted or GREY if greylisted.
source ip  IP address the connection originated from.
from       envelope-from address for GREY (empty for WHITE entries).
to         envelope-to address for GREY (empty for WHITE entries).
first      time the entry was first seen.
pass       time the entry passed from being GREY to being WHITE.
expire     time the entry will expire and be removed from the database.
block      number of times a corresponding connection received a
           temporary failure from spamd. Usually the blocked delivery
           attempts up to the passtime mins time. 
pass       number of times a corresponding connection has been seen
           to pass to the real MTA by spamlogd. Successful e-mails
           passed to postfix.

To see the time entries in a human readable format just use the date command with "-r" on the system. Times are in seconds since the Unix Epoch. For example:

date -r 1152286088

Will give the output:

Fri Jul  7 11:28:08 EDT 2006

Once the delivering server is whitelisted it can deliver the mail unhindered. The server will make the delivery and MTA will handle the mail transfer. That should be it. Spamd should be up and rolling.

One thing about Spamd. It can be a problem with webmail services like gmail and hotmail. The problem goes like this. When someone sends and e-mail from this service the first delivery is sent out by lets say server A. It will be greylisted like it is supposed to. Then the webmail system waits and tries delivery again but this time from server B. Sever B is then greylisted. It the webmail system waits for some time again like it is supposed to and tries to deliver the mail again this time from server C. Server C is then greylisted. It can go on and on like this never having the same server get back to our server in the right amount of time before the greylisting is removed. Then the process would have to start over again. This happens because large e-mail services use pools of servers to deliver mail. You can never expect to get the mail delivered from the same server. People who run greylisting servers try to get around this by whitelisting as many webmail servers then can find. You can comb through past e-mails from people that sent mail from these services and try to guess the naming scheme. There is a nice script that checks the SPF records and a mantained whilelist file. This will help in trying to whitelist valid servers. The results of this file need to be put in a whitelist table for PF. For anyone intrested the script looks like:

!/bin/sh
# Script from http://home.xnet.com/~ansible/openbsd_spamd_conf.html

FILE=/etc/whitelist.txt

rm -f $FILE
touch $FILE

for domain in \
        aol.com \
        apple.com \
        amazon.com \
        gmx.net \
        _spf.google.com \
        spf-a.hotmail.com \
        spf-b.hotmail.com \
        spf-c.hotmail.com \
        spf-d.hotmail.com \
        _spf-a.microsoft.com \
        _spf-b.microsoft.com \
        _spf-c.microsoft.com \
        mynethost.com

do
echo \#$domain >> $FILE;
dig $domain TXT +short | tr "\ " "\n" | grep ^ip4: | cut -d: -f2 >> $FILE;
done

echo '# http://cvs.puremagic.com/viewcvs/*checkout*/greylisting/schema/whitelist_ip.txt' >> $FILE;
ftp -o -
http://cvs.puremagic.com/viewcvs/*checkout*/greylisting/schema/whitelist_ip.txt \
    | awk '/^[^#]/ {print $0}' >> $FILE;

Whatever you do it's a pain but it may be a small price to pay to for the reduction in spam.

Reddit!

Related stories


RSS Feed RSS feed logo

About


3com

3ware

alsa

alsactl

alsamixer

amd

android

apache

areca

arm

ati

auditd

awk

badblocks

bash

bind

bios

bonnie

cable

carp

cat5

cdrom

cellphone

centos

chart

chrome

chromebook

cifs

cisco

cloudera

comcast

commands

comodo

compiz-fusion

corsair

cpufreq

cpufrequtils

cpuspeed

cron

crontab

crossover

cu

cups

cvs

database

dbus

dd

dd_rescue

ddclient

debian

decimal

dhclient

dhcp

diagnostic

diskexplorer

disks

dkim

dns

dos

dovecot

drac

dsniff

dvdauthor

e-mail

echo

editor

emerald

ethernet

expect

ext3

ext4

fat32

fedora

fetchmail

fiber

filesystems

firefox

firewall

flac

flexlm

floppy

flowtools

fonts

format

freebsd

ftp

gdm

gmail

gnome

google

greasemonkey

greylisting

growisofs

grub

hacking

hadoop

harddrive

hba

hex

hfsc

html

html5

http

https

hulu

idl

ie

ilo

intel

ios

iperf

ipmi

iptables

ipv6

irix

javascript

kde

kernel

kickstart

kmail

kprinter

krecord

kubuntu

kvm

lame

ldap

linux

logfile

lp

lpq

lpr

maradns

matlab

memory

mencoder

mhdd

mkinitrd

mkisofs

moinmoin

motherboard

mouse

movemail

mplayer

multitail

mutt

myodbc

mysql

mythtv

nagios

nameserver

netflix

netflow

nginx

nic

ntfs

ntp

nvidia

odbc

openbsd

openntpd

openoffice

openssh

openssl

openvpn

opteron

parted

partimage

patch

perl

pf

pfflowd

pfsync

photorec

php

pop3

pop3s

ports

postfix

power

procmail

proftpd

proxy

pulseaudio

putty

pxe

python

qemu

r-studio

raid

recovery

redhat

router

rpc

rsync

ruby

saltstack

samba

schedule

screen

scsi

seagate

seatools

sed

sendmail

sgi

shell

siw

smtp

snort

solaris

soundcard

sox

spam

spamd

spf

sql

sqlite

squid

srs

ssh

ssh.com

ssl

su

subnet

subversion

sudo

sun

supermicro

switches

symbols

syslinux

syslog

systemrescuecd

t1

tcpip

tcpwrappers

telnet

terminal

testdisk

tftp

thttpd

thunderbird

timezone

ting

tls

tools

tr

trac

tuning

tunnel

ubuntu

unbound

vi

vpn

wget

wiki

windows

windowsxp

wireless

wpa_supplicant

x

xauth

xfree86

xfs

xinearama

xmms

youtube

zdump

zeromq

zic

zlib