Discussion:
[rancid] persistent alerts - but nothing was changed ... ?
Wilkinson, Alex
2013-07-15 09:28:10 UTC
Permalink
Hi all,

I am consistently getting rancid alerts (diffs) that config has changed on
a number of Cisco Nexus devices. However, the diffs in the email are exactly
the same each rancid-run(1) and are bogus.

Is there anything that would cause rancid to see bogus diffs ?

-Alex

************** IMPORTANT MESSAGE *****************************
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential.
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
We can be contacted through our web site: commbank.com.au.
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line.
**************************************************************
Alan McKinnon
2013-07-15 09:37:35 UTC
Permalink
Post by Wilkinson, Alex
Hi all,
I am consistently getting rancid alerts (diffs) that config has changed on
a number of Cisco Nexus devices. However, the diffs in the email are exactly
the same each rancid-run(1) and are bogus.
Is there anything that would cause rancid to see bogus diffs ?
A diff is a diff, and it has content because something is different.
Maybe whitespace.

If you post one of these diffs, we can help examine it for you.
--
Alan McKinnon
***@gmail.com
Wilkinson, Alex
2013-07-16 00:41:08 UTC
Permalink
Post by Alan McKinnon
Post by Wilkinson, Alex
Hi all,
I am consistently getting rancid alerts (diffs) that config has changed on
a number of Cisco Nexus devices. However, the diffs in the email are exactly
the same each rancid-run(1) and are bogus.
Is there anything that would cause rancid to see bogus diffs ?
A diff is a diff, and it has content because something is different.
Maybe whitespace.
If you post one of these diffs, we can help examine it for you.
I'm using SVN not CVS.

Here is an example diff - I have not touched these lines whatsoever but get alerts every day about them:

Index: configs/nexus4k1-5
===================================================================
- -- configs/nexus1-5 (revision 85)
@@ -94,6 +94,17 @@
ele-fwd pause rate threshold is 1000 pps


+ interface mgmt0
+ speed 1000
+ duplex full
+ vrf member management
+ ip address 192.168.240.35/24
+
+ interface mgmt1
+ boot kickstart bootflash:/n4000-bk9-kickstart.4.1.2.E1.1i.bin
+ boot system bootflash:/n4000-bk9.4.1.2.E1.1i.bin
+ system health loopback frequency 60
+
interface Ethernet1/1
link state group 1 downstream
spanning-tree port type edge
@@ -194,17 +205,6 @@
interface Ethernet1/20
speed 10000

- interface mgmt0
- speed 1000
- duplex full
- vrf member management
- ip address 192.168.240.35/24
-
- interface mgmt1
- boot kickstart bootflash:/n4000-bk9-kickstart.4.1.2.E1.1i.bin
- boot system bootflash:/n4000-bk9.4.1.2.E1.1i.bin
- system health loopback frequency 60
-

Any ideas of how to debug ?

-Alex

************** IMPORTANT MESSAGE *****************************
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential.
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
We can be contacted through our web site: commbank.com.au.
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line.
**************************************************************
Roy
2013-07-16 02:47:55 UTC
Permalink
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
Post by Wilkinson, Alex
Post by Alan McKinnon
Post by Wilkinson, Alex
Hi all,
I am consistently getting rancid alerts (diffs) that config has changed on
a number of Cisco Nexus devices. However, the diffs in the email are exactly
the same each rancid-run(1) and are bogus.
Is there anything that would cause rancid to see bogus diffs ?
A diff is a diff, and it has content because something is different.
Maybe whitespace.
If you post one of these diffs, we can help examine it for you.
I'm using SVN not CVS.
Index: configs/nexus4k1-5
===================================================================
- -- configs/nexus1-5 (revision 85)
@@ -94,6 +94,17 @@
ele-fwd pause rate threshold is 1000 pps
+ interface mgmt0
+ speed 1000
+ duplex full
+ vrf member management
+ ip address 192.168.240.35/24
+
+ interface mgmt1
+ boot kickstart bootflash:/n4000-bk9-kickstart.4.1.2.E1.1i.bin
+ boot system bootflash:/n4000-bk9.4.1.2.E1.1i.bin
+ system health loopback frequency 60
+
interface Ethernet1/1
link state group 1 downstream
spanning-tree port type edge
@@ -194,17 +205,6 @@
interface Ethernet1/20
speed 10000
- interface mgmt0
- speed 1000
- duplex full
- vrf member management
- ip address 192.168.240.35/24
-
- interface mgmt1
- boot kickstart bootflash:/n4000-bk9-kickstart.4.1.2.E1.1i.bin
- boot system bootflash:/n4000-bk9.4.1.2.E1.1i.bin
- system health loopback frequency 60
-
Any ideas of how to debug ?
-Alex
Wilkinson, Alex
2013-07-16 03:53:23 UTC
Permalink
Post by Roy
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
You are exactly right. When comparing the diffs via OpenGrok its very clear that
the line locations consistently change causing a diff + rancid alert. If i'm not
changing these devices and the line locations are supposedly changing - what
could cause this ? rancid ?

-Alex

************** IMPORTANT MESSAGE *****************************
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential.
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
We can be contacted through our web site: commbank.com.au.
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line.
**************************************************************
heasley
2013-07-16 20:53:45 UTC
Permalink
Post by Wilkinson, Alex
Post by Roy
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
You are exactly right. When comparing the diffs via OpenGrok its very clear that
the line locations consistently change causing a diff + rancid alert. If i'm not
changing these devices and the line locations are supposedly changing - what
could cause this ? rancid ?
it wouldnt be rancid, it's the device itself. report the bug to the TAC.
Wilkinson, Alex
2013-07-17 01:46:15 UTC
Permalink
Post by heasley
Post by Wilkinson, Alex
Post by Roy
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
You are exactly right. When comparing the diffs via OpenGrok its very clear that
the line locations consistently change causing a diff + rancid alert. If i'm not
changing these devices and the line locations are supposedly changing - what
could cause this ? rancid ?
it wouldnt be rancid, it's the device itself. report the bug to the TAC.
Something I have noticed is that for all my Nexus devices (7K, 5K, 4K) none of
them have the 'RANCID-CONTENT-TYPE' of 'cisco-nx' but rather plain old 'cisco'.
Could this be the reason behind the my problem ? And even if not, why would
Rancid not be using cisco-nx automagically ? Or do I have to set it manually ?

-Alex

************** IMPORTANT MESSAGE *****************************
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential.
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
We can be contacted through our web site: commbank.com.au.
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line.
**************************************************************
Alan McKinnon
2013-07-17 05:49:50 UTC
Permalink
Post by Wilkinson, Alex
Post by heasley
Post by Wilkinson, Alex
Post by Roy
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
You are exactly right. When comparing the diffs via OpenGrok its very clear that
the line locations consistently change causing a diff + rancid alert. If i'm not
changing these devices and the line locations are supposedly changing - what
could cause this ? rancid ?
it wouldnt be rancid, it's the device itself. report the bug to the TAC.
Something I have noticed is that for all my Nexus devices (7K, 5K, 4K) none of
them have the 'RANCID-CONTENT-TYPE' of 'cisco-nx' but rather plain old 'cisco'.
Could this be the reason behind the my problem ? And even if not, why would
Rancid not be using cisco-nx automagically ? Or do I have to set it manually ?
You have to set it manually in router.db

Rancid has no auto-detection of device type
--
Alan McKinnon
***@gmail.com
Wilkinson, Alex
2013-07-17 07:24:43 UTC
Permalink
Post by Alan McKinnon
Post by Wilkinson, Alex
Post by heasley
Post by Wilkinson, Alex
Post by Roy
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
You are exactly right. When comparing the diffs via OpenGrok its very clear that
the line locations consistently change causing a diff + rancid alert. If i'm not
changing these devices and the line locations are supposedly changing - what
could cause this ? rancid ?
it wouldnt be rancid, it's the device itself. report the bug to the TAC.
Something I have noticed is that for all my Nexus devices (7K, 5K, 4K) none of
them have the 'RANCID-CONTENT-TYPE' of 'cisco-nx' but rather plain old 'cisco'.
Could this be the reason behind the my problem ? And even if not, why would
Rancid not be using cisco-nx automagically ? Or do I have to set it manually ?
You have to set it manually in router.db
Rancid has no auto-detection of device type
Ah, great! I changed all NX devices to cisco-nx. This seems to fix the false alert
noise but now introduces a new problem for our Nexus 5000s e.g

Trying to get all of the configs.
nexus5k1-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-1: End of run not found
!
nexus5k1-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-2: End of run not found
!
nexus5k2-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-2: End of run not found
!
nexus5k2-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-1: End of run not found

All other Nexus devices work fine now (7Ks, 4Ks).

Any ideas ?

-Alex

************** IMPORTANT MESSAGE *****************************
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential.
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
We can be contacted through our web site: commbank.com.au.
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line.
**************************************************************
Alan McKinnon
2013-07-17 07:59:27 UTC
Permalink
Post by Wilkinson, Alex
Post by Alan McKinnon
Post by Wilkinson, Alex
Post by heasley
Post by Wilkinson, Alex
Post by Roy
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
You are exactly right. When comparing the diffs via OpenGrok its very clear that
the line locations consistently change causing a diff + rancid alert. If i'm not
changing these devices and the line locations are supposedly changing - what
could cause this ? rancid ?
it wouldnt be rancid, it's the device itself. report the bug to the TAC.
Something I have noticed is that for all my Nexus devices (7K, 5K, 4K) none of
them have the 'RANCID-CONTENT-TYPE' of 'cisco-nx' but rather plain old 'cisco'.
Could this be the reason behind the my problem ? And even if not, why would
Rancid not be using cisco-nx automagically ? Or do I have to set it manually ?
You have to set it manually in router.db
Rancid has no auto-detection of device type
Ah, great! I changed all NX devices to cisco-nx. This seems to fix the false alert
noise but now introduces a new problem for our Nexus 5000s e.g
Trying to get all of the configs.
nexus5k1-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-1: End of run not found
!
nexus5k1-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-2: End of run not found
!
nexus5k2-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-2: End of run not found
!
nexus5k2-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-1: End of run not found
All other Nexus devices work fine now (7Ks, 4Ks).
Any ideas ?
The failing commands are the last three, so I'd suspect the command just
before those - show debug.

Stuff to check:

1. Does clogin properly work and properly enable the login?
2. What is in the log files (${RANCIDDIR}/var/logs/*) for those devices?
3. Does the rancid user have proper permissions to run those commands?
(keep in mind that if you use tacacs for auth, that Nexus are *very*
different from IOS wrt authorization).


I found that the default behaviour in the nxrancid code was to quit the
script entirely on permission denied errors, which caused me huge
issues. So now I apply this patch (beware, it's long):

(my notes about what it does):
* The nexus parser is incomplete and "Permission denied" errors cause
the parser to fail and quit. We want to continue regardless and mark the
command as having failed.
* Oddly enough, "show fex" and "show module fex" are not supported out
of the box.
* Oddly enough, the routine to sanitize SNMP community strings is
commented out.
* Changed the code to redact tacacs keys
* Expanded password redaction regex to exclude "mpls ldp neighbor"
Post by Wilkinson, Alex
--- nxrancid.orig 2012-02-28 12:21:51.000000000 +0200
+++ nxrancid 2013-05-15 11:14:14.000000000 +0200
@@ -174,7 +174,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(-1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
if (/^Cisco Nexus Operating System/) { $type = "NXOS";}
@@ -241,7 +242,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
/^Built By / && ProcessHistory("COMMENTS","","", "!Build: $_");
@@ -264,7 +266,8 @@
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if /(Invalid input detected|Type help or )/;
return(1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
/^-+$/ && next; # Skip lines of all dashes.
@@ -286,7 +289,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
s/ +$//; # Drop trailing ' '
@@ -307,7 +311,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
s/ +$//; # Drop trailing ' '
@@ -328,7 +333,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(-1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
# Cut out CurTemp - drop the 2nd to last field.
@@ -358,7 +364,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
# Cut out Actual Output/Draw.
@@ -404,7 +411,8 @@
return(1) if /(Invalid input detected|Type help or )/;
return(1) if /Ambiguous command/i;
return(-1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
s/ variable = / = /;
@@ -434,7 +442,8 @@
return(1) if / is either not present or not formatted/;
return(-1) if /\%Error calling/;
return(-1) if /(: device being squeezed|ATA_Status time out)/i; # busy
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
return(1) if /(Open device \S+ failed|Error opening \S+:)/;
@@ -460,7 +469,8 @@
last if (/^$prompt/);
next if (/^\s*$cmd\s*$/);
return(1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
s/(.*) \*$/$1/; # Drop a trailing '*'
@@ -485,7 +495,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
if (/^(NAME: "[^"]*",)\s+(DESCR: "[^"]+")/) {
@@ -532,7 +543,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
next if (/^Configuration last modified by/);
# the pager can not be disabled per-session on the PIX
@@ -568,7 +575,8 @@
# newer releases (~12.1(9)) place the vlan config in the normal
# configuration (write term).
return(1) if ($type =~ /^(3550|4500)$/);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
# the pager can not be disabled per-session on the PIX
if (/^(<-+ More -+>)/) {
@@ -594,7 +602,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(-1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
/^No matching debug flags set$/ && next;
@@ -619,7 +628,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
ProcessHistory("COMMENTS","","","!CORES: $_");
@@ -639,7 +649,8 @@
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(-1) if (/\% Invalid command at /);
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
ProcessHistory("COMMENTS","","","!PROC_LOGS: $_");
@@ -648,6 +659,27 @@
return(0);
}
+# This routine parses "show fex" and "show module fex"
+sub ShowFex {
+ print STDERR " In ShowFex: $_" if ($debug);
+
+ while (<INPUT>) {
+ tr/\015//d;
+ last if (/^$prompt/);
+ next if (/^(\s*|\s*$cmd\s*)$/);
+ return(1) if /Line has invalid autocommand /;
+ return(1) if /(Invalid input detected|Type help or )/;
+ return(1) if (/\% Invalid command at /);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
+ return(-1) if (/command authorization failed/i);
+
+ ProcessHistory("COMMENTS","","","!FEX: $_");
+ }
+ ProcessHistory("COMMENTS","","","!\n");
+ return(0);
+}
+
# This routine processes a "write term"
sub WriteTerm {
print STDERR " In WriteTerm: $_" if ($debug);
@@ -660,7 +692,8 @@
return(1) if (/(Invalid input detected|Type help or )/i);
return(-1) if (/\% Invalid command at /);
return(0) if ($found_end); # Only do this routine once
- return(-1) if (/\% Permission denied/);
+# return(-1) if (/\% Permission denied/);
+ return(1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);
# /Non-Volatile memory is in use/ && return(-1); # NvRAM is locked
@@ -784,6 +822,10 @@
# ProcessHistory("","","","! neighbor $1 password <removed>\n");
# next;
# }
+ if (/^\s*(.*?neighbor \S*) password / && $filter_pwds >= 1) {
+ ProcessHistory("","","","! $1 password <removed>\n");
+ next;
+ }
# if (/^(ppp .* password) 7 .*/ && $filter_pwds >= 1) {
# ProcessHistory("","","","!$1 <removed>\n"); next;
# }
@@ -929,18 +962,25 @@
# }
# next;
# }
-# if (/^(snmp-server community) (\S+)/) {
-# if ($filter_commstr) {
-# ProcessHistory("SNMPSERVERCOMM","keysort","$_","!$1 <removed>$'") && next;
-# } else {
-# ProcessHistory("SNMPSERVERCOMM","keysort","$_","$_") && next;
-# }
-# }
+ # Why was this commented out? It shows up in the raw text...
+ if (/^(snmp-server community) (\S+)/) {
+ if ($filter_commstr) {
+ ProcessHistory("SNMPSERVERCOMM","keysort","$_","!$1 <removed>$'") && next;
+ } else {
+ ProcessHistory("SNMPSERVERCOMM","keysort","$_","$_") && next;
+ }
+ }
# # prune tacacs/radius server keys
# if (/^((tacacs|radius)-server\s(\w*[-\s(\s\S+])*\s?key) (\d )?\w+/
# && $filter_pwds >= 1) {
# ProcessHistory("","","","!$1 <removed>$'"); next;
# }
+ # tacacs-server host 196.23.0.13 key 7 "xxxxxxx" port 50 timeout 10
+ if (/^((tacacs|radius)-server.*?\bkey\b.*?) ".*?"(.*)/
+ && $filter_pwds >= 1) {
+ ProcessHistory("","","","!$1 <removed>$3\n"); next;
+ }
# # order clns host statements
# /^clns host \S+ (\S+)/ &&
# ProcessHistory("CLNS","keysort","$1","$_") && next;
@@ -1035,7 +1075,7 @@
{'show boot' => 'ShowBoot'},
{'dir bootflash:' => 'DirSlotN'},
{'dir debug:' => 'DirSlotN'},
- {'dir logflash:' => 'DirSlotN'},
+# {'dir logflash:' => 'DirSlotN'},
{'dir slot0:' => 'DirSlotN'},
{'dir usb1:' => 'DirSlotN'},
{'dir usb2:' => 'DirSlotN'},
@@ -1048,6 +1088,8 @@
{'show debug' => 'ShowDebug'},
{'show cores vdc-all' => 'ShowCores'},
{'show processes log vdc-all' => 'ShowProcLog'},
+ {'show module fex' => 'ShowFex'},
+ {'show fex' => 'ShowFex'},
{'show running-config' => 'WriteTerm'},
);
# Use an array to preserve the order of the commands and a hash for mapping
[edit] bin/rancid
Removed dynamic address data from a description line for Ethernet interfaces.
Expanded password redaction regex to exclude "mpls ldp neighbor"
--- rancid.orig 2012-12-20 22:46:04.000000000 +0200
+++ rancid 2012-12-20 22:48:51.000000000 +0200
@@ -835,6 +835,10 @@
/^AM79970 / && ProcessHistory("INT","","","!Interface: $_") && next;
/^buffer size \d+ (Universal Serial: .*)/ &&
ProcessHistory("INT","","","!\t$1\n") && next;
+ # !Interface: FastEthernet0/0, GT96K FE ADDR: 62AFB684, FASTSEND: 61579E4C, MCI_INDEX: 0
+ /^Hardware is (.*?)($| ADDR: .*| at 0x.*)/ &&
+ ProcessHistory("INT","","","!Interface: $INT$1\n") && next;
/^Hardware is (.*)/ &&
ProcessHistory("INT","","","!Interface: $INT$1\n") && next;
/^(QUICC Serial unit \d),/ &&
@@ -1741,8 +1745,8 @@
ProcessHistory("LINE-PASS","","","!$1secret <removed>\n");
next;
}
- if (/^\s*neighbor (\S*) password / && $filter_pwds >= 1) {
- ProcessHistory("","","","! neighbor $1 password <removed>\n");
+ if (/^\s*(.*?neighbor.*?) password / && $filter_pwds >= 1) {
+ ProcessHistory("","","","! $1 password <removed>\n");
next;
}
if (/^(\s*ppp .* hostname) .*/ && $filter_pwds >= 1) {
--
Alan McKinnon
***@gmail.com
Wilkinson, Alex
2013-07-17 08:46:58 UTC
Permalink
Post by Alan McKinnon
Post by Wilkinson, Alex
Post by Alan McKinnon
Post by Wilkinson, Alex
Post by heasley
Post by Wilkinson, Alex
Post by Roy
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
You are exactly right. When comparing the diffs via OpenGrok its very clear that
the line locations consistently change causing a diff + rancid alert. If i'm not
changing these devices and the line locations are supposedly changing - what
could cause this ? rancid ?
it wouldnt be rancid, it's the device itself. report the bug to the TAC.
Something I have noticed is that for all my Nexus devices (7K, 5K, 4K) none of
them have the 'RANCID-CONTENT-TYPE' of 'cisco-nx' but rather plain old 'cisco'.
Could this be the reason behind the my problem ? And even if not, why would
Rancid not be using cisco-nx automagically ? Or do I have to set it manually ?
You have to set it manually in router.db
Rancid has no auto-detection of device type
Ah, great! I changed all NX devices to cisco-nx. This seems to fix the false alert
noise but now introduces a new problem for our Nexus 5000s e.g
Trying to get all of the configs.
nexus5k1-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-1: End of run not found
!
nexus5k1-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-2: End of run not found
!
nexus5k2-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-2: End of run not found
!
nexus5k2-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-1: End of run not found
All other Nexus devices work fine now (7Ks, 4Ks).
Any ideas ?
The failing commands are the last three, so I'd suspect the command just
before those - show debug.
Awesome! You where right. 'show debug' was failing. I forgot to add a role for
rancid user. All sorted now! Neat patch also!

-Alex

************** IMPORTANT MESSAGE *****************************
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential.
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
We can be contacted through our web site: commbank.com.au.
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line.
**************************************************************
Alan McKinnon
2013-07-17 09:12:05 UTC
Permalink
Post by Wilkinson, Alex
Awesome! You where right. 'show debug' was failing. I forgot to add a role for
rancid user. All sorted now! Neat patch also!
Thanks :-)


I'd review the patch very carefully if I were you - it's specific to my
needs and might not work for you
--
Alan McKinnon
***@gmail.com
Wilkinson, Alex
2013-07-23 01:10:32 UTC
Permalink
Post by Wilkinson, Alex
Post by Alan McKinnon
Post by Wilkinson, Alex
Post by heasley
Post by Wilkinson, Alex
Post by Roy
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
You are exactly right. When comparing the diffs via OpenGrok its very clear that
the line locations consistently change causing a diff + rancid alert. If i'm not
changing these devices and the line locations are supposedly changing - what
could cause this ? rancid ?
it wouldnt be rancid, it's the device itself. report the bug to the TAC.
Something I have noticed is that for all my Nexus devices (7K, 5K, 4K) none of
them have the 'RANCID-CONTENT-TYPE' of 'cisco-nx' but rather plain old 'cisco'.
Could this be the reason behind the my problem ? And even if not, why would
Rancid not be using cisco-nx automagically ? Or do I have to set it manually ?
You have to set it manually in router.db
Rancid has no auto-detection of device type
Ah, great! I changed all NX devices to cisco-nx. This seems to fix the false alert
noise but now introduces a new problem for our Nexus 5000s e.g
Trying to get all of the configs.
nexus5k1-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-1: End of run not found
!
nexus5k1-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-2: End of run not found
!
nexus5k2-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-2: End of run not found
!
nexus5k2-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-1: End of run not found
mmm... so false positives are back!

e.g.

Index: configs/nexus7k
===================================================================
- -- configs/nexus7k (revision 380)
@@ -299,10 +299,10 @@
!Env: Xb3 N7K-C7010-FAB-1 60 W Powered-Up
!Env: Xb4 xbar 60 W Absent
!Env: Xb5 xbar 60 W Absent
- !Env: fan1 N7K-C7010-FAN-S 720 W Powered-Up
- !Env: fan2 N7K-C7010-FAN-S 720 W Powered-Up
- !Env: fan3 N7K-C7010-FAN-F 120 W Powered-Up
- !Env: fan4 N7K-C7010-FAN-F 120 W Powered-Up
+ !Env: fan1 N7K-C7010-FAN-S 720 W Powered-Up
+ !Env: fan2 N7K-C7010-FAN-S 720 W Powered-Up
+ !Env: fan3 N7K-C7010-FAN-F 120 W Powered-Up
+ !Env: fan4 N7K-C7010-FAN-F 120 W Powered-Up
!Env: N/A - Per module power not available
!Env: Power Usage Summary:
!Env: --------------------

So I get this alert all day because of white space changes. Is there anyway to stop/ignore this ?

-Alex

************** IMPORTANT MESSAGE *****************************
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential.
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
We can be contacted through our web site: commbank.com.au.
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line.
**************************************************************
Vincent Hoffman-Kazlauskas
2013-07-23 09:07:05 UTC
Permalink
Post by Wilkinson, Alex
Post by Wilkinson, Alex
Post by Alan McKinnon
Post by Wilkinson, Alex
Post by heasley
Post by Wilkinson, Alex
Post by Roy
I don't know the box but the diff seems to indicate that the location of
mgmt0 and mgmt1 interfaces in the config file is changing. In the first
diff the interfaces are before Ethernet1/1 and in the second diff, they
are after Ethernet 1/20
You are exactly right. When comparing the diffs via OpenGrok its very clear that
the line locations consistently change causing a diff + rancid alert. If i'm not
changing these devices and the line locations are supposedly changing - what
could cause this ? rancid ?
it wouldnt be rancid, it's the device itself. report the bug to the TAC.
Something I have noticed is that for all my Nexus devices (7K, 5K, 4K) none of
them have the 'RANCID-CONTENT-TYPE' of 'cisco-nx' but rather plain old 'cisco'.
Could this be the reason behind the my problem ? And even if not, why would
Rancid not be using cisco-nx automagically ? Or do I have to set it manually ?
You have to set it manually in router.db
Rancid has no auto-detection of device type
Ah, great! I changed all NX devices to cisco-nx. This seems to fix the false alert
noise but now introduces a new problem for our Nexus 5000s e.g
Trying to get all of the configs.
nexus5k1-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-1: End of run not found
!
nexus5k1-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k1-2: End of run not found
!
nexus5k2-2: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-2: End of run not found
!
nexus5k2-1: missed cmd(s): show cores vdc-all,show processes log vdc-all,show running-config
nexus5k2-1: End of run not found
mmm... so false positives are back!
e.g.
Index: configs/nexus7k
===================================================================
- -- configs/nexus7k (revision 380)
@@ -299,10 +299,10 @@
!Env: Xb3 N7K-C7010-FAB-1 60 W Powered-Up
!Env: Xb4 xbar 60 W Absent
!Env: Xb5 xbar 60 W Absent
- !Env: fan1 N7K-C7010-FAN-S 720 W Powered-Up
- !Env: fan2 N7K-C7010-FAN-S 720 W Powered-Up
- !Env: fan3 N7K-C7010-FAN-F 120 W Powered-Up
- !Env: fan4 N7K-C7010-FAN-F 120 W Powered-Up
+ !Env: fan1 N7K-C7010-FAN-S 720 W Powered-Up
+ !Env: fan2 N7K-C7010-FAN-S 720 W Powered-Up
+ !Env: fan3 N7K-C7010-FAN-F 120 W Powered-Up
+ !Env: fan4 N7K-C7010-FAN-F 120 W Powered-Up
!Env: N/A - Per module power not available
!Env: --------------------
So I get this alert all day because of white space changes. Is there anyway to stop/ignore this ?
Totally untested/pulled out of somewhere or other but diff (or svn diff,
dunno about cvs diff) can take a -b flag that ignores changes in the
amount of white space.
A very brief grep in the /usr/libexec/rancid/* scripts (on a centos
install not sure where for other distros/OS) leads me to think you could
try adding the -b flag to the relevent diff commands in control_rancid.

Cant promise this will fix it or is a good idea but it could work.

Vince
Post by Wilkinson, Alex
-Alex
************** IMPORTANT MESSAGE *****************************
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential.
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
We can be contacted through our web site: commbank.com.au.
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line.
**************************************************************
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
Chris Moody
2013-07-16 05:06:14 UTC
Permalink
I see TONS of buggy outputs that cause this same behavior from our Nexus
boxes.

It's almost entirely due to whitespace changes in outputs.

You'd seriously think that the output of something like show commands
would be consistent...but I guess that's too hard to do. ;o)

-Chris
Post by Alan McKinnon
Post by Wilkinson, Alex
Hi all,
I am consistently getting rancid alerts (diffs) that config has changed on
a number of Cisco Nexus devices. However, the diffs in the email are exactly
the same each rancid-run(1) and are bogus.
Is there anything that would cause rancid to see bogus diffs ?
A diff is a diff, and it has content because something is different.
Maybe whitespace.
If you post one of these diffs, we can help examine it for you.
Daniel Schmidt
2013-07-16 14:28:10 UTC
Permalink
Odd, I don't have many issues. On the subject of white space, I believe it
was John Jetmore who proposed this white space fix, is it in your code?

#########################
--- nxrancid-238-dist 2012-06-14 10:58:55.000000000 -0500
+++ nxrancid-238-local 2012-06-14 11:00:00.000000000 -0500
@@ -380,7 +380,7 @@
s/ Draw / /;
s/ ----------- / /;
s/ N\/A / / ||
- s/ \d+ W / /; # Does not chop enough to line up.
+ s/ [ \d]{9} W / /; # Does not chop enough to line
up. (does now)

/actual draw/ && next; # Drop changing total power output.

###########################
Post by Chris Moody
I see TONS of buggy outputs that cause this same behavior from our Nexus
boxes.
It's almost entirely due to whitespace changes in outputs.
You'd seriously think that the output of something like show commands
would be consistent...but I guess that's too hard to do. ;o)
-Chris
Post by Alan McKinnon
Post by Wilkinson, Alex
Hi all,
I am consistently getting rancid alerts (diffs) that config has changed on
a number of Cisco Nexus devices. However, the diffs in the email are exactly
the same each rancid-run(1) and are bogus.
Is there anything that would cause rancid to see bogus diffs ?
A diff is a diff, and it has content because something is different.
Maybe whitespace.
If you post one of these diffs, we can help examine it for you.
______________________________**_________________
Rancid-discuss mailing list
http://www.shrubbery.net/**mailman/listinfo.cgi/rancid-**discuss<http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss>
E-Mail to and from me, in connection with the transaction
of public business, is subject to the Wyoming Public Records
Act and may be disclosed to third parties.
heasley
2013-07-16 17:14:52 UTC
Permalink
Post by Daniel Schmidt
Odd, I don't have many issues. On the subject of white space, I believe it
was John Jetmore who proposed this white space fix, is it in your code?
#########################
--- nxrancid-238-dist 2012-06-14 10:58:55.000000000 -0500
+++ nxrancid-238-local 2012-06-14 11:00:00.000000000 -0500
@@ -380,7 +380,7 @@
s/ Draw / /;
s/ ----------- / /;
s/ N\/A / / ||
- s/ \d+ W / /; # Does not chop enough to line up.
+ s/ [ \d]{9} W / /; # Does not chop enough to line
up. (does now)
/actual draw/ && next; # Drop changing total power output.
###########################
ShowEnvPower currently looks like this (from Zenon Mousmoulas/myself), which
I hope fixes the problem for folks:

# This routine parses "show environment power"
sub ShowEnvPower {
print STDERR " In ShowEnvPower: $_" if ($debug);

while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
next if (/^\s*\^\s*$/);
return(1) if /Line has invalid autocommand /;
return(1) if /(Invalid input detected|Type help or )/;
return(1) if (/\% Invalid command at /);
return(-1) if (/\% Permission denied/);
return(-1) if (/command authorization failed/i);

# Cut out Actual Output/Draw.
#Power Actual Total
#Supply Model Output Capacity Status
# (Watts ) (Watts )
#------- ------------------- ----------- ----------- --------------
#1 ------------ 0 W 0 W Absent
#3 749 W 5480 W Ok
# Actual Power
#Module Model Draw Allocated Status
# (Watts ) (Watts )
#------- ------------------- ----------- ----------- --------------
#2 NURBURGRING N/A 573 W Powered-Up
#fan1 N/A 720 W Powered-Up
if ( /(.* +)(\d+ W)( +\d+ W.*)/) {
$_ = sprintf("%s%-". length($2)."s%s\n", $1, "", $3);
}

/actual draw/ && next; # Drop changing total power output.

s/ +$//; # Drop trailing ' '
ProcessHistory("COMMENTS","","","!Env: $_");
}
ProcessHistory("COMMENTS","","","!\n");
return(0);
}
Post by Daniel Schmidt
Post by Chris Moody
I see TONS of buggy outputs that cause this same behavior from our Nexus
boxes.
It's almost entirely due to whitespace changes in outputs.
You'd seriously think that the output of something like show commands
would be consistent...but I guess that's too hard to do. ;o)
-Chris
Post by Alan McKinnon
Post by Wilkinson, Alex
Hi all,
I am consistently getting rancid alerts (diffs) that config has changed on
a number of Cisco Nexus devices. However, the diffs in the email are exactly
the same each rancid-run(1) and are bogus.
Is there anything that would cause rancid to see bogus diffs ?
A diff is a diff, and it has content because something is different.
Maybe whitespace.
If you post one of these diffs, we can help examine it for you.
______________________________**_________________
Rancid-discuss mailing list
http://www.shrubbery.net/**mailman/listinfo.cgi/rancid-**discuss<http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss>
E-Mail to and from me, in connection with the transaction
of public business, is subject to the Wyoming Public Records
Act and may be disclosed to third parties.
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
Matthew J Wilson
2013-07-24 11:22:54 UTC
Permalink
Post by Wilkinson, Alex
Index: configs/nexus7k
===================================================================
- -- configs/nexus7k (revision 380)
@@ -299,10 +299,10 @@
!Env: Xb3 N7K-C7010-FAB-1 60 W Powered-Up
!Env: Xb4 xbar 60 W Absent
!Env: Xb5 xbar 60 W Absent
- !Env: fan1 N7K-C7010-FAN-S 720 W
Powered-Up
- !Env: fan2 N7K-C7010-FAN-S 720 W
Powered-Up
- !Env: fan3 N7K-C7010-FAN-F 120 W
Powered-Up
- !Env: fan4 N7K-C7010-FAN-F 120 W
Powered-Up
+ !Env: fan1 N7K-C7010-FAN-S 720 W Powered-Up
+ !Env: fan2 N7K-C7010-FAN-S 720 W Powered-Up
+ !Env: fan3 N7K-C7010-FAN-F 120 W
Powered-Up
+ !Env: fan4 N7K-C7010-FAN-F 120 W
Powered-Up
!Env: N/A - Per module power not available
!Env: --------------------
So I get this alert all day because of white space changes. Is there
anyway to stop/ignore this ?
We ran into this as well. Would a patch to the nxrancid script like the
following work for you?

-Matt



Index: nxrancid
===================================================================
--- nxrancid (revision 852)
+++ nxrancid (revision 853)
@@ -372,7 +372,7 @@
s/ Draw / /;
s/ ----------- / /;
s/ N\/A / / ||
- s/ \d+ W / /; # Does not chop enough to line up.
+ s/ (\d+) W /" " x length($1)/e; # Replace with same length

/actual draw/ && next; # Drop changing total power output.

Loading...