[rancid] PfSense Package [semi-solved!]

[James Bensley]

Hi Danilo

Thanks for that link to the pfSense package. Finding a pfSense plugin
was on my to do list, I just hadn't gotten that far yet. I have this
working now although I had a few issues.

To use this, unpack the three files into your RANCID bin directory.
This is likely something like /usr/lib/rancid/bin/ or
/usr/local/rancid/bin/. In there you will find an existing file
"rancid-fe", replace or merge with the new one to update your devices
definitions. Now you can add pfsense firewalls to your devices.db file
with the type of "m0n0", which is what you will use for pfSense. Even
it says m0n0 as pfSense is a fork of m0n0wall, m0n0walls don't support
SSH and this script tries telnet/ssh/rsh so it won't work on them.

Also, note that you must enable SSH on your pfSense box if it isn't
already. I then added user which only has the right to SSH in.

These scripts are a bit broken though and my scripting skills aren't
the best; so I am in fact stuck. I have hacked them about a bit and
now get the following output in my hourly rancid emails (which you can
trigger manually with (rancid-run -r my-pfsense-device.fqdn.com)

Index: configs/my-pfsense-device.fqdn.com
===================================================================
retrieving revision 1.2
diff -U 4 -r1.2 my-pfsense-device.fqdn.com
@@ -1 +1,1769 @@
- exec m0n0login -t 120 -c "uname -a;cat /cf/conf/config.xml"
my-pfsense-device.fqdn.com
+ my-pfsense-device.fqdn.com
+ spawn ssh -2 -x -l rancid my-pfsense-device.fqdn.com
+ Password:
+ Last login: Wed Dec 19 10:28:47 2012 from 89.21.224.35
+ Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994
+ The Regents of the University of California. All rights reserved.
+
+
+ [0;1;33m[ [0;1;37m2.0.1-RELEASE [0;1;33m] [0;1;33m[ [0;1;37mrancid
[0;1;31m@ [0;1;37mmy-pfsense-device.fqdn.com [0;1;33m]
[0;1;32m/home/rancid [0;1;33m( [0;1;37m1 [0;1;33m) [0;1;36m [0;1;31m:
[0;40;37m
uname -a
+ FreeBSD my-pfsense-device.fqdn.com 8.1-RELEASE-p6 FreeBSD
8.1-RELEASE-p6 #0: Mon Dec 12 18:59:41 EST 2011
***@FreeBSD_8.0_pfSense_2.0-snaps.pfsense.org:/usr/obj./usr/pfSensesrc/src/sys/pfSense_wrap.8.i386
i386
+ [0;1;33m[ [0;1;37m2.0.1-RELEASE [0;1;33m] [0;1;33m[ [0;1;37mrancid
[0;1;31m@ [0;1;37mmy-pfsense-device.fqdn.com [0;1;33m]
[0;1;32m/home/rancid [0;1;33m( [0;1;37m2 [0;1;33m) [0;1;36m [0;1;31m:
[0;40;37m
cat /cf/conf/config.xml
+ <?xml version="1.0"?>
+ <pfsense>

As you can see from this opening snippet there are two problems;
Firstly, the expect script m0n0login is including the SSH MTOD/Banner
stuff (I said my scripting wasn't great, although this doesn't really
matter). Secondly, an issue which does actually matter, when you SSH
to a pfSense box they have coloured terminal output and SSH is
spitting this out (the colouring info) in to the expect script, so the
prompt on my test pfSense box which usually looks like this;

[2.0.1-RELEASE][***@my-pfsense-device.fqdn.com]/home/username(1):

Now looks like this;

[0;1;33m[ [0;1;37m2.0.1-RELEASE [0;1;33m] [0;1;33m[ [0;1;37mrancid
[0;1;31m@ [0;1;37mmy-pfsense-device.fqdn.com [0;1;33m]
[0;1;32m/home/rancid [0;1;33m( [0;1;37m1 [0;1;33m) [0;1;36m [0;1;31m:
[0;40;37m

I will continue to try and fix this by either of the below and post
back the fix here once it is solved, but I have no idea how long that
will take;
1 - Someone wiser than me here can tell me how to stop SSH from either
accepting the colouring info from the pfSense box or not display it on
stdout
2 - I find help else where

In the mean time, for you and anyone else that NEEDs to be backing up
pfSense boxes right now I have modified a m0n0wall bash script making
which makes commits to the rancid CVS for me and it works just fine
(in a different "branch" though"). Similar to the SSH method, add a
rancid user and allow them just access to the diagnostic backup page.
This works with curl over HTTPS:

I hope that helps someone, and I hope someone can help me,
Cheers,
James.

#!/bin/bash
# backup up a pfsense config and puts it into cvs
# depends on: bash, curl, cvs, date, rm

CVSROOT=/var/lib/cvs
export CVSROOT
CVSPROJ=pfsense

## HTTPS firewalls on port 8080...

DEVICES="my-pfsense-device.fqdn.com \
another-pfsense-device.fqdn.com \
3rd-pfsense-device.fqdn.com"

PROTO=https
PORT=8080
USER=rancid
PASS=rancidpassword

for DEVICE in $DEVICES; do
TMPDIR=/tmp/$$
mkdir $TMPDIR
cd $TMPDIR
cvs -Q co $CVSPROJ
cd $CVSPROJ
# Login
curl -k -o /dev/null --cookie cjar --cookie-jar cjar --data
"login=Login" --data "usernamefld=$USER" --data "passwordfld=$PASS"
--location $PROTO://$DEVICE:$PORT/index.php
# Download config file
curl -k -o config-$DEVICE.xml --cookie cjar --cookie-jar cjar --data
"Submit=download" --data "donotbackuprrd=yes" --location
$PROTO://$DEVICE:$PORT/diag_backup.php
# Log out
curl -k -o /dev/null --cookie cjar --cookie-jar cjar --location
$PROTO://$DEVICE:$PORT/index.php?logout
echo config-$DEVICE.xml
NOW=`date +%Y-%m-%d@%H:%M:%S`
cvs -Q commit -m "backup of $DEVICE config.xml [$NOW]"
cvs -Q import -m "backup of $DEVICE config.xml [$NOW]" voswall configs release
cd /tmp
rm -rf $TMPDIR
done

[Bill Fenner]

Post by James Bensley
Hi Danilo
Thanks for that link to the pfSense package. Finding a pfSense plugin
was on my to do list, I just hadn't gotten that far yet. I have this
working now although I had a few issues.
To use this, unpack the three files into your RANCID bin directory.
This is likely something like /usr/lib/rancid/bin/ or
/usr/local/rancid/bin/. In there you will find an existing file
"rancid-fe", replace or merge with the new one to update your devices
definitions. Now you can add pfsense firewalls to your devices.db file
with the type of "m0n0", which is what you will use for pfSense. Even
it says m0n0 as pfSense is a fork of m0n0wall, m0n0walls don't support
SSH and this script tries telnet/ssh/rsh so it won't work on them.
Also, note that you must enable SSH on your pfSense box if it isn't
already. I then added user which only has the right to SSH in.
These scripts are a bit broken though and my scripting skills aren't
the best; so I am in fact stuck. I have hacked them about a bit and
now get the following output in my hourly rancid emails (which you can
trigger manually with (rancid-run -r my-pfsense-device.fqdn.com)
Index: configs/my-pfsense-device.fqdn.com
===================================================================
retrieving revision 1.2
diff -U 4 -r1.2 my-pfsense-device.fqdn.com
@@ -1 +1,1769 @@
- exec m0n0login -t 120 -c "uname -a;cat /cf/conf/config.xml"
my-pfsense-device.fqdn.com
+ my-pfsense-device.fqdn.com
+ spawn ssh -2 -x -l rancid my-pfsense-device.fqdn.com
+ Last login: Wed Dec 19 10:28:47 2012 from 89.21.224.35
+ Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994
+ The Regents of the University of California. All rights reserved.
+
+
+ [0;1;33m[ [0;1;37m2.0.1-RELEASE [0;1;33m] [0;1;33m[ [0;1;37mrancid
[0;40;37m
uname -a
+ FreeBSD my-pfsense-device.fqdn.com 8.1-RELEASE-p6 FreeBSD
8.1-RELEASE-p6 #0: Mon Dec 12 18:59:41 EST 2011
i386
+ [0;1;33m[ [0;1;37m2.0.1-RELEASE [0;1;33m] [0;1;33m[ [0;1;37mrancid
[0;40;37m
cat /cf/conf/config.xml
+ <?xml version="1.0"?>
+ <pfsense>
As you can see from this opening snippet there are two problems;
Firstly, the expect script m0n0login is including the SSH MTOD/Banner
stuff (I said my scripting wasn't great, although this doesn't really
matter). Secondly, an issue which does actually matter, when you SSH
to a pfSense box they have coloured terminal output and SSH is
spitting this out (the colouring info) in to the expect script, so the
prompt on my test pfSense box which usually looks like this;
Now looks like this;
[0;1;33m[ [0;1;37m2.0.1-RELEASE [0;1;33m] [0;1;33m[ [0;1;37mrancid
[0;40;37m
I will continue to try and fix this by either of the below and post
back the fix here once it is solved, but I have no idea how long that
will take;
1 - Someone wiser than me here can tell me how to stop SSH from either
accepting the colouring info from the pfSense box or not display it on
stdout
2 - I find help else where

My solution to this is to edit the rancid user's ~/.tcshrc on the
pfsense box to only contain:

set prompt="pfsense# "

Not hugely scalable, but I only have a handful of pfsense boxes to collect from.

I agree that it'd be nicer for m0n0login to be able to strip the ANSI
escape sequences.

Bill

[heasley]

Post by Bill Fenner
I agree that it'd be nicer for m0n0login to be able to strip the ANSI
escape sequences.

try using hpuifilter, as in hlogin.

i have yet to be sucessful getting pfsense to work in a vm, so i havent
tried this myself.