pantz.org banner
Expect examples and tips
Posted on 06-24-2012 21:42:19 UTC | Updated on 06-24-2012 23:05:37 UTC
Section: /software/expect/ | Permanent Link

Expect is an automation and testing tool used to automate a process that receives interactive commands. If you can connect to any machine using ssh, telnet, ftp, etc then you can automate the process with an expect script. This works even with local programs where you would have to interact with a script or program on the command line.

When I first starting using Expect I wanted a short but detailed description on how Expect performed its magic. I had to read the Expect O'Reilly book to gather all of the information I needed, but that was not very short. I'm going to give my shortened version of how it works, and answer the questions I had at the time I was learning it.

Installing and getting started

Expect is written as a Tcl extension and can be installed on almost any Unix type distribution. The package on most distributions should just be called "expect" so look for that when installing it. For example on Linux distros that use apt-get to install packages just issue "sudo apt-get install expect".

After Expect is installed find where it was installed to by issuing the command "which expect". This should give you the path to the binary. If not try becoming root and issuing the same command. We need to know the path of the binary as it is the first line of our script. Note this path as we will be needing it soon.

How expect works

Expect scripts are written like many other scripts. Like Bash or Perl the binary is called at the top "#!/usr/bin/expect" and the scripts commands below it. Expect is an extention to the Tcl scripting language so it uses all of Tcl's syntax.

Most scripts begin by spawning the process they want to interact with. This is done with the "spawn" command. Like "spawn ssh user@host or spawn ftp host". Then you can begin checking for output with the command "expect". Expect uses regular expressions to find patterns in the output, and when it matches a pattern you send it command with the "send" command. In the simplest form that is how expect works. You start the process and look for patterns in the output. Then send commands based on the matches from the output.

Most used commands and descriptions

Below I will list the commands (with descriptions) I use the most in Expect. They should get you through the simple scripts that most people need.

Expect scripts design

Before we get to actual Expect script examples let me lay out how I design my scripts and what I have learned works best for me.

  1. I love giving good feedback when an error condition is reached. To do this with expect try to use the eof and timeout keywords, or default pattern keyword in each expect statement. If you set a timeout and eof message and action in each expect block you can use a send_user message and tell the user what part of the script failed or if the process exited. If you use exit as your action for the keywords then you can be sure the script does not go any further. This helps make debugging what went wrong faster at the expense of a few extra lines per statement.I think its worth it.
  2. I try not to use the while loops with the expect command. I see people do this a lot and many times it is not needed. The expect command itself is a loop. It will keep looping through the output looking for a match. You can act on any matches and keep performing more matches and actions. Expect has a default timeout of 10 secs when looping through looking for a match. If there is never a match it will timeout and you can set an action to this timeout if you want.
  3. To set your Expect scripts apart from other scripts use the file extension .exp
  4. Always try to follow a send command with an expect statement if possible. This helps first with timing where the next command will not be executed until that last one has been completed and checked with the expect statement. You don't need sleep statements if you have the luxury of knowing what output you should be seeing. You can be sure the command completed correctly if you know what should be there when it is finished.

Example #1.

The first example will be logging into a linux box with ssh to see if the given accounts password works. By doing this we are also checking other things like if your machine is up and if you can login to it and get a prompt back. So in actuality this script does a bunch of things.

#!/usr/bin/expect
set timeout 9
set username [lindex $argv 0]
set password [lindex $argv 1]
set hostname [lindex $argv 2]
log_user 0

if {[llength $argv] == 0} {
  send_user "Usage: scriptname username \'password\' hostname\n"
  exit 1
}

send_user "\n#####\n# $hostname\n#####\n"

spawn ssh -q -o StrictHostKeyChecking=no $username@$hostname

expect {
  timeout { send_user "\nFailed to get password prompt\n"; exit 1 }
  eof { send_user "\nSSH failure for $hostname\n"; exit 1 }
  "*assword"
}

send "$password\r"

expect {
  timeout { send_user "\nLogin failed. Password incorrect.\n"; exit 1}
  "*\$ "
}

send_user "\nPassword is correct\n"
send "exit\r"
close

Let me describe what is going on in this script line by line.

Line 1 executes the start of the script with the path to the expect binary.

Line 2 sets a timeout for each expect statement to 9 seconds.

Line 3,4,5 set variables for username, password, and hostname which are taken from the command line for when the script is run.

Line 6 turns off the output to STDOUT (printing the output to the screen). If we remove this line you will see the whole login process.

Line 8 is the first if statement. It checks to see if any arguments have been given to the script and if not it will print out how to use the script. If you do this with every script you will always be able to remind yourself of how to use it.

Line 13 prints a banner with the hosts name.

Line 15 starts the ssh process and turns on quiet mode and turns off host key checking (only turn off key checking on a trusted network). It uses the variables we give it on the command line to ssh to the host.

Line 17 is our first expect statement. First we set our timeout message in case our expect statement can not find a match for what its looking for. According to our timeout we set this will occur in 9 seconds. This means it will loop through the ssh output for 9 seconds and if it can not find a match it will print this error an then exit as we have told it to do. We also set our EOF value here. If our ssh session does not connect or gets disconnected EOF will be returned and our error message will print. This will then exit like we tell it to do. The last and most important part of this statement is line 20 the pattern to match which is "*assword". This is a regular expression that looks for 0 or more characters with the letters assword after it. Many password login prompts for ssh show up as "Password: " so this should match that. If it does match then it moves to the next statement. If it does not match we will hit the timeout, then our error message will print and the script will exit.

Line 23 will actually send the password we specified on the command line to ssh. We know we can send the password now because we verified we had a matching password prompt.

Line 25 starts our next expect statement. We set the timeout error message to say that the login must be incorrect. Why? Because the matching loop has started again looking for output. This time we are looking for a command prompt. If we can not match a command prompt in the output then that means we never got one. That means our password must have failed. Line 27 after the timeout line is the matching expression "*\$ ". The prompt I was matching for looked like this "user@hostname:~$ ". So the match says 0 or more of any character (*) and then a dollar sign and then a space. Notice the dollar sign has a \ before it. We have to escape this because it is a special character for regular expressions which means end of line match. If we don't put the \ before the $ it will still work, but the match is to broad and matches anything. We want it to be a specific match to be sure it is a correct prompt so we want to look for any character with a $ and a space. Your prompts are likely to vary so change your matching expression to meet your needs.

Line 30 sends output to the screen to tell the person the we got a correct prompt back. We can send this because we know we successfully got this far and our prompt matched from the previous expect statement.

Line 31 and 32. Line 31 sends the exit command to the linux machine. Line 32 then closes the connection to the ssh process. That's it!

This script uses command line arguments for a reason. If you can build your scripts like this then you can use it to check many different accounts across many different hosts. It does not lock you into one host and account per script. If you have a list of hosts you can use it in a bash loop and check many machines at once. Like: for x in server1 server2; do ./script.exp opt1 opt2 $x; done

Notice we have to put the password on the command line with this script. First off make sure you put your passwords in single quotes. That removes any issue with special characters being interpreted by your shell. Second, is a bad idea to use password prompts on the command line because many shells (like bash) keep a history of your commands (like ~/.bash_history). That means your password is sitting in a file in plaintext in your home dir. That's bad. To help mitigate this with bash you can put the following line in your ~/.bashrc file or ~/.profile. "export HISTCONTROL=ignorespace" . This will then execute on login and after it is run will allow you to put a space before any command and it will not be kept in your history. The password will still be on your screen after you have run it but usually this is scrolled off soon after the script is run. Typing "clear" will scroll it off the current screen, but just keep it in mind.

Example #2

This is a more complicated example that has to deal with different situations depending on the device type. The following script will SSH into 2 different types of Cisco routers and switches and change the SNMP password. The reason this is more complicated is that the 2 different types of devices uses 2 different operating systems with some similar and some different commands.

#!/usr/bin/expect
set timeout 15
set hostname [lindex $argv 0]
set snmpshapass [lindex $argv 1]
set snmpaespass [lindex $argv 2]
set username [lindex $argv 3]
set password [lindex $argv 4]
set enable [lindex $argv 5]
set send_slow {10 .001}
log_user 0

if {[llength $argv] == 0} {
  send_user "Usage: scriptname hostname \'snmpshapass\' \'snmpaespass\' username \'userpassword\' \'enablepassword\'\n"
  send_user "For Cisco Nexus devices just give hostname snmpshapass and snmpaespass if you have ssh keys installed\n"
  exit 1
}

send_user "\n#####\n# $hostname\n#####\n"

if { [info exists $username] } { 
  spawn ssh -q -o StrictHostKeyChecking=no $username@$hostname
} else {
  spawn ssh -q -o StrictHostKeyChecking=no $hostname
}

expect {
  timeout { send_user "\nFailed to get password prompt\n"; exit 1 }
  eof { send_user "\nSSH failure for $hostname\n"; exit 1 }
  "*#" {}
  "*assword:" {
    send "$password\r"
  }
}

send "/r"

expect {
  default { send_user "\nCould not get into enabled mode. Password problem?\n"; exit 1 }
  "*#" {}
  "*>" {
    send "enable\r"
    expect "*assword"
    send "$enable\r"
    expect "*#"
  }
}

send "show ver | inc Cisco\r"

expect {
  default { send_user "\nFailed to determine OS or get back correct prompt while changing pass.\n"; exit 1 }
  "Nexus" {
    send "config t\r"
    expect "*(config)#"
    send snmp-server user snmpUser network-operator auth sha $snmpshapass priv AES-128 $snmpaespass\r"
    expect "*(config)#"
    send "exit\r"
    expect "*#"
    send "copy run start\r"
    expect "100%"
    expect "*#"
  }
  "IOS" {
    send "config t\r"
    expect "*(config)#"
    send snmp-server user snmpUser snmpGroup v3 auth sha $snmpshapass priv AES 128 $snmpaespass\r"
    expect "*(config)#"
    send "exit\r"
    expect "*#"
    send "write mem\r"
    expect "*#"
  }
}

send "exit\r"
send_user "\nSuccessfully changed SNMP password on $hostname\n"
close

I will not go through this script line by line like it did the first one but I will describe what it is doing at some key parts. You notice this script checks for a specific command line option on line 19. If the username variable is set then it will execute the ssh line with a username. If no username is given then it assumes your using ssh keys and uses the username you logged in with. Nexus devices allow ssh keys per user, IOS does not.

After starting the ssh session we have to make sure we are logged and in enabled mode. Cisco and Nexus devices show your in enabled mode with the prompt matching "*#". When we login to a Nexus device with ssh keys we are already at that prompt, this is not true for IOS devices or Nexus devices that do not use keys. Line 25 is our first expect statement and first checks to see if have the correct prompt "*#". If so then nothing is executed and the expect statement is short circuited with {}. If we don't match that prompt expect moves onto the second match line "*assword". That will match an IOS device or a Nexus device that is not configured with ssh keys. It will then give the correct password. If that fails we get the error message.

On line 36 we use expect again to see if we in the correct mode by checking for the prompt matching "*#". For a Nexus device with ssh keys setup we are already at that prompt so we short circuit again and move on to the next area. If there is no match we move to looking for the correct IOS prompt "*>". If we find it then we issue an enable command and give the password to finally get to the prompt we have been looking for that matches "*#". We check for that pattern one last time. if it matches then we move onto the next block.

On line 47 we send the command "show ver | inc Cisco\r". This command works for both devices and will give us a line that we can match on that shows us which operating system we are using. It will either say Nexus in the line or IOS in the line. We will use this output in our next expect statement

On line 49 we start our expect statement and check for one of 2 OS types. Either IOS or Nexus. When one matches we enter configuration mode and run the command to change the SNMP password for that device type. After the command is run we write our configuration to memory and make sure we get back to prompt successfully. The save command takes a few seconds to run. By making sure we get back to a prompt after we issue the save command it assures we have given the save command the right amount of time to run successfully. It is possible there could be an error in the save, but right now we are not checking for that.

Then we finally exit out of the router or switch. If we got this far and did not hit any exit timeouts we set then it is very likely this worked, so we send our success message. Then close our process down.

Tips and Gotchas

Del.icio.us! | Digg Me! | Reddit!

Related stories


RSS Feed RSS feed logo
About


3com
3ware
alsa
alsactl
alsamixer
amd
android
apache
areca
arm
ati
auditd
awk
badblocks
bash
bind
bios
bonnie
cable
carp
cat5
cdrom
cellphone
centos
chart
cifs
cisco
cloudera
comcast
commands
comodo
compiz-fusion
corsair
cpufreq
cpufrequtils
cpuspeed
cron
crontab
crossover
cu
cups
cvs
database
dbus
dd
dd_rescue
ddclient
debian
decimal
dhclient
dhcp
diagnostic
diskexplorer
disks
dns
dos
dovecot
drac
dsniff
dvdauthor
e-mail
echo
editor
emerald
ethernet
expect
ext3
ext4
fat32
fedora
fetchmail
fiber
filesystems
firefox
firewall
flac
flexlm
floppy
flowtools
fonts
format
freebsd
ftp
gdm
gnome
greasemonkey
greylisting
growisofs
grub
hacking
hadoop
harddrive
hba
hex
hfsc
html
html5
http
idl
ie
ilo
intel
ios
iperf
ipmi
iptables
ipv6
irix
javascript
kde
kernel
kickstart
kmail
kprinter
krecord
kubuntu
kvm
lame
ldap
linux
logfile
lp
lpq
lpr
maradns
matlab
memory
mencoder
mhdd
mkinitrd
mkisofs
moinmoin
motherboard
mouse
movemail
mplayer
multitail
mutt
myodbc
mysql
mythtv
nagios
nameserver
netflow
nginx
nic
ntfs
ntp
nvidia
odbc
openbsd
openntpd
openoffice
openssh
openssl
opteron
parted
partimage
patch
perl
pf
pfflowd
pfsync
photorec
php
pop3
pop3s
ports
postfix
power
procmail
proftpd
proxy
pulseaudio
putty
pxe
python
qemu
r-studio
raid
recovery
redhat
router
rpc
rsync
samba
schedule
scsi
seagate
seatools
sed
sendmail
sgi
shell
siw
smtp
snort
solaris
soundcard
sox
spam
spamd
sql
sqlite
squid
ssh
ssh.com
ssl
su
subnet
subversion
sudo
sun
supermicro
switches
symbols
syslinux
syslog
systemrescuecd
t1
tcpip
tcpwrappers
telnet
terminal
testdisk
tftp
thttpd
thunderbird
timezone
ting
tools
tr
trac
tuning
tunnel
ubuntu
vi
wget
wiki
windows
windowsxp
wireless
wpa_supplicant
x
xauth
xfree86
xfs
xinearama
xmms
youtube
zdump
zic
zlib