[rancid] post-processing the diff listing before mailing?

[Lee]

Anyone like the idea of processing the diff listing before mailing it out?

I'd rather have rancid collect the configs and not mess with them -
ie: rancid.conf has
FILTER_PWDS=NO
NOCOMMSTR=NO
ACLSORT=NO
but I don't want things like SNMP community strings or
encrypted/obfuscated passwords being mailed out.. So my idea is
post-processing the diff listing before emailing it out. Which would
also let me know which configs changed so I could check the new
configs & mail out a 'local config standards violations' email :)

The code is really ugly, but as a proof of concept..

"control_rancid" line 447
# This has been different for different machines...
# Diff the directory and then checkin.
trap 'rm -fr $TMP $TMP.diff $DIR/routers.single;' 1 2 15
cd $DIR
if [ $RCSSYS = "cvs" ] ; then
# --LR-- cvs -f diff -U 4 -ko | sed -e '/^RCS file: /d' -e '/^--- /d' \
# --LR-- 4 is not enough, 8 seems a bit much so try 6
cvs -f diff -U 6 -ko | sed -e '/^RCS file: /d' -e '/^--- /d' \
-e '/^+++ /d' -e 's/^\([-+ ]\)/\1 /' >$TMP.diff
else
svn diff | sed -e '/^+++ /d' -e 's/^\([-+ ]\)/\1 /' >$TMP.diff
fi

# --LR-- begin: check configs of changed devices
grep "^Index: " $TMP.diff |\
awk '/^Index: configs/{
if ( ! got1 ) {
printf("echo\necho\necho Config check results:\n~/bin/ckConfig.sh ")
got1=1
}
printf("%s ", $2)
}
END{ if (got1) printf("\necho == results end ==\n") }
' >$TMP.doit
/bin/sh $TMP.doit >> $TMP.diff
~/bin/sanitize.sh $TMP.diff >$TMP.doit
/bin/mv $TMP.doit $TMP.diff
# --LR-- end: check configs of changed devices

if [ $alt_mailrcpt -eq 1 ] ; then
subject="router config diffs - courtesy of $mailrcpt"
else
subject="router config diffs"
fi


Lee

[Chris Gauthier]

I do like the idea of that quite a bit. I intentionally turn off the masking in the repository, but prefer it not be mailed out. Perhaps that should just be the default setting? In any event, I like the concept a lot.

Chris G.

-----Original Message-----
From: rancid-discuss-***@shrubbery.net [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of Lee
Sent: Wednesday, September 22, 2010 10:04 AM
To: rancid-***@shrubbery.net
Subject: [rancid] post-processing the diff listing before mailing?

Anyone like the idea of processing the diff listing before mailing it out?

I'd rather have rancid collect the configs and not mess with them -
ie: rancid.conf has
FILTER_PWDS=NO
NOCOMMSTR=NO
ACLSORT=NO
but I don't want things like SNMP community strings or encrypted/obfuscated passwords being mailed out.. So my idea is post-processing the diff listing before emailing it out. Which would also let me know which configs changed so I could check the new configs & mail out a 'local config standards violations' email :)

The code is really ugly, but as a proof of concept..

"control_rancid" line 447
# This has been different for different machines...
# Diff the directory and then checkin.
trap 'rm -fr $TMP $TMP.diff $DIR/routers.single;' 1 2 15 cd $DIR if [ $RCSSYS = "cvs" ] ; then
# --LR-- cvs -f diff -U 4 -ko | sed -e '/^RCS file: /d' -e '/^--- /d' \
# --LR-- 4 is not enough, 8 seems a bit much so try 6
cvs -f diff -U 6 -ko | sed -e '/^RCS file: /d' -e '/^--- /d' \
-e '/^+++ /d' -e 's/^\([-+ ]\)/\1 /' >$TMP.diff else
svn diff | sed -e '/^+++ /d' -e 's/^\([-+ ]\)/\1 /' >$TMP.diff fi

# --LR-- begin: check configs of changed devices grep "^Index: " $TMP.diff |\ awk '/^Index: configs/{
if ( ! got1 ) {
printf("echo\necho\necho Config check results:\n~/bin/ckConfig.sh ")
got1=1
}
printf("%s ", $2)
}
END{ if (got1) printf("\necho == results end ==\n") } ' >$TMP.doit /bin/sh $TMP.doit >> $TMP.diff ~/bin/sanitize.sh $TMP.diff >$TMP.doit /bin/mv $TMP.doit $TMP.diff
# --LR-- end: check configs of changed devices

if [ $alt_mailrcpt -eq 1 ] ; then
subject="router config diffs - courtesy of $mailrcpt"
else
subject="router config diffs"
fi


Lee
_______________________________________________
Rancid-discuss mailing list
Rancid-***@shrubbery.net
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CONFIDENTIALITY NOTICE
Attention: The information contained in this email and/or attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any system and destroy any copies.

[john heasley]

Post by Lee
Anyone like the idea of processing the diff listing before mailing it out?
I'd rather have rancid collect the configs and not mess with them -
ie: rancid.conf has
FILTER_PWDS=NO
NOCOMMSTR=NO
ACLSORT=NO
but I don't want things like SNMP community strings or
encrypted/obfuscated passwords being mailed out.. So my idea is
post-processing the diff listing before emailing it out. Which would
also let me know which configs changed so I could check the new
configs & mail out a 'local config standards violations' email :)
The code is really ugly, but as a proof of concept..

why not do this outside of rancid; mail it to a script that processes
it's input, such as you've done here, and emails the result to your
diff-receivers list. everyone can have their own flavor of diff
post-processing.

Post by Lee
"control_rancid" line 447
# This has been different for different machines...
# Diff the directory and then checkin.
trap 'rm -fr $TMP $TMP.diff $DIR/routers.single;' 1 2 15
cd $DIR
if [ $RCSSYS = "cvs" ] ; then
# --LR-- cvs -f diff -U 4 -ko | sed -e '/^RCS file: /d' -e '/^--- /d' \
# --LR-- 4 is not enough, 8 seems a bit much so try 6
cvs -f diff -U 6 -ko | sed -e '/^RCS file: /d' -e '/^--- /d' \
-e '/^+++ /d' -e 's/^\([-+ ]\)/\1 /' >$TMP.diff
else
svn diff | sed -e '/^+++ /d' -e 's/^\([-+ ]\)/\1 /' >$TMP.diff
fi
# --LR-- begin: check configs of changed devices
grep "^Index: " $TMP.diff |\
awk '/^Index: configs/{
if ( ! got1 ) {
printf("echo\necho\necho Config check results:\n~/bin/ckConfig.sh ")
got1=1
}
printf("%s ", $2)
}
END{ if (got1) printf("\necho == results end ==\n") }
' >$TMP.doit
/bin/sh $TMP.doit >> $TMP.diff
~/bin/sanitize.sh $TMP.diff >$TMP.doit
/bin/mv $TMP.doit $TMP.diff
# --LR-- end: check configs of changed devices
if [ $alt_mailrcpt -eq 1 ] ; then
subject="router config diffs - courtesy of $mailrcpt"
else
subject="router config diffs"
fi
Lee
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss

[john heasley]

Post by john heasley

Post by Lee
Anyone like the idea of processing the diff listing before mailing it out?
I'd rather have rancid collect the configs and not mess with them -
ie: rancid.conf has
FILTER_PWDS=NO
NOCOMMSTR=NO
ACLSORT=NO
but I don't want things like SNMP community strings or
encrypted/obfuscated passwords being mailed out.. So my idea is
post-processing the diff listing before emailing it out. Which would
also let me know which configs changed so I could check the new
configs & mail out a 'local config standards violations' email :)
The code is really ugly, but as a proof of concept..

why not do this outside of rancid; mail it to a script that processes
it's input, such as you've done here, and emails the result to your
diff-receivers list. everyone can have their own flavor of diff
post-processing.

btw, also see rancid-<vers>/share/rtrfilter.

[Lee]

Post by john heasley

Post by john heasley

Post by Lee
Anyone like the idea of processing the diff listing before mailing it out?
I'd rather have rancid collect the configs and not mess with them -
ie: rancid.conf has
FILTER_PWDS=NO
NOCOMMSTR=NO
ACLSORT=NO
but I don't want things like SNMP community strings or
encrypted/obfuscated passwords being mailed out.. So my idea is
post-processing the diff listing before emailing it out. Which would
also let me know which configs changed so I could check the new
configs & mail out a 'local config standards violations' email :)
The code is really ugly, but as a proof of concept..

why not do this outside of rancid; mail it to a script that processes
it's input, such as you've done here, and emails the result to your
diff-receivers list.

Mainly because I'm not confident of my ability to get that working
correctly 100% of the time. Adding another line or two to the
existing script seems much less error-prone :)

Post by john heasley
everyone can have their own flavor of diff

Post by john heasley
post-processing.

btw, also see rancid-<vers>/share/rtrfilter.

Thanks for the pointer. The problem I have with rtrfilter is that it
deletes lines & I want to know if something is added or changed, but I
don't want sensitive info mailed out.

here's my sanitize.sh - any suggestions for what else should be blanked out?
============
#!/bin/sh
# script to sanitize a ciscso config
#

awk '
/^Index: configs/ { if ( NR != 1) printf("\n\n\n") }
{
# IOS stuph
strip("key 7 ", "xXxXxX")
strip("md5 7 ", "xXxXxX")
strip("password 7 ", "xXxXxX")
strip("secret 5 ", "xXxXxX")
strip("snmp-server community ", "xXxXxX")
strip("crypto isakmp key ", "xXxXxX")
strip("tacacs-server key 7 ", "xXxXxX")

# CatOS stuph
strip("set password " , "xXxXxX")
strip("set enablepass ", "xXxXxX")
strip("set snmp community read-only " , "xXxXxX")
strip("set snmp community read-write " , "xXxXxX")
strip("set snmp community read-write-all ", "xXxXxX")
strip("set tacacs key ", "xXxXxX")

print $0
}

func strip(s, r) {
sub(s " *[^ ]*", s r, $0)
}
' $1
=============

Thanks,
Lee