[rancid] Doubling characters on some devices

[Walt Howd]

I'm running rancid 2.3.6-8 with expect 5.44.1.14 on Ubuntu 10.04.3 LTS 32bit.

I'm backing up configurations on approximately 200 devices without issues but two devices will sometimes double or omit a character on one or more lines. Example:

diff --git a/cisco/configs/4507-1 b/cisco/configs/4507-1
index 97a63fc..30c6d3e 100644
--- a/cisco/configs/4507-1
+++ b/cisco/configs/4507-1
@@ -2915,7 +2915,7 @@
interface GigabitEthernet6/22
switchport port-security maximum 4
switchport port-security
storm-control broadcast level 10.00

- sstorm-control action shutdown
+ storm-control action shutdown

spanning-tree portfast
!
<snip>
@@ -4704,7 +4704,7 @@
interface GigabitEthernet9/41
switchport mode access
switchport block multicast
switchport block unicast

- switchport port-security maximumm 4
+ switchport port-security maximum 4

switchport port-security
storm-control broadcast level 10.00
storm-control action shutdown



I've looked through the archives but havent' found others experiencing this problem. My rancid.conf is below:



TERM=network;export TERM
umask 027
TMPDIR=/tmp; export TMPDIR
BASEDIR=/var/lib/rancid; export BASEDIR
PATH=/usr/lib/rancid/bin:/usr/bin:/usr/sbin:/bin:/usr/local/bin:/usr/bin; export PATH
CVSROOT=$BASEDIR/CVS; export CVSROOT
LOGDIR=$BASEDIR/logs; export LOGDIR
RCSSYS=git; export RCSSYS
NOPIPE=YES; export NOPIPE
FILTER_PWDS=YES; export FILTER_PWDS
NOCOMMSTR=YES; export NOCOMMSTR
PAR_COUNT=3; export PAR_COUNT
LIST_OF_GROUPS="cisco"
HTMLMAILS=YES; export HTMLMAILS



I've turned on NOPIPE and also turned PAR_COUNT down to 3 but the issue still randomly occurs.

I have also tested manually pulling configs (via TFTP) from the problematic switches and have not found any unexpected differences.




Walt Howd
Network Manager
Information Technology Services
Truman State University
100 East Normal Street
Kirksville, MO 63501

[Walt Howd]

I'm running rancid 2.3.6-8 with expect 5.44.1.14 on Ubuntu 10.04.3 LTS 32bit.

I'm backing up configurations on approximately 200 devices without issues but two devices will sometimes double or omit a character on one or more lines. Example:

diff --git a/cisco/configs/4507-1 b/cisco/configs/4507-1
index 97a63fc..30c6d3e 100644
--- a/cisco/configs/4507-1
+++ b/cisco/configs/4507-1
@@ -2915,7 +2915,7 @@
interface GigabitEthernet6/22
switchport port-security maximum 4
switchport port-security
storm-control broadcast level 10.00

- sstorm-control action shutdown
+ storm-control action shutdown

spanning-tree portfast
!
<snip>
@@ -4704,7 +4704,7 @@
interface GigabitEthernet9/41
switchport mode access
switchport block multicast
switchport block unicast

- switchport port-security maximumm 4
+ switchport port-security maximum 4

switchport port-security
storm-control broadcast level 10.00
storm-control action shutdown



I've looked through the archives but havent' found others experiencing this problem. My rancid.conf is below:



TERM=network;export TERM
umask 027
TMPDIR=/tmp; export TMPDIR
BASEDIR=/var/lib/rancid; export BASEDIR
PATH=/usr/lib/rancid/bin:/usr/bin:/usr/sbin:/bin:/usr/local/bin:/usr/bin; export PATH
CVSROOT=$BASEDIR/CVS; export CVSROOT
LOGDIR=$BASEDIR/logs; export LOGDIR
RCSSYS=git; export RCSSYS
NOPIPE=YES; export NOPIPE
FILTER_PWDS=YES; export FILTER_PWDS
NOCOMMSTR=YES; export NOCOMMSTR
PAR_COUNT=3; export PAR_COUNT
LIST_OF_GROUPS="cisco"
HTMLMAILS=YES; export HTMLMAILS



I've turned on NOPIPE and also turned PAR_COUNT down to 3 but the issue still randomly occurs.

I have also tested manually pulling configs (via TFTP) from the problematic switches and have not found any unexpected differences.




Walt Howd
Network Manager
Information Technology Services
Truman State University
100 East Normal Street
Kirksville, MO 63501

[Walt Howd]

The pager is disabled successfully at logon. I tried "show version" through clogin on the two problematic devices as well as several others.

The two devices in question have the largest config (at 104K) and third largest config (at 64K) in my environment.

I also apologize for the double posting of my original message.

Example clogin:

***@rancid:~$ bin/clogin -c 'show version' 4507-1
4507-1
spawn telnet 4507-1
Trying 10.1.1.96...
Connected to 4507-1 (10.1.1.96).
Escape character is '^]'.

WARNING: All systems are monitored.
Unauthorized access to this system is prohibited.
Violators are subject to all criminal and civil penalties.


User Access Verification

Username: switchuser
Password:
4507-1>enable
Password:
4507-1#
4507-1#terminal length 0
4507-1#show version
Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500-ENTSERVICESK9-M), Version 12.2(40)SG, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Wed 07-Nov-07 18:00 by prod_rel_team
Image text-base: 0x10000000, data-base: 0x11ADF0B8

ROM: 12.2(20r)EW1
Dagobah Revision 226, Swamp Revision 34

4507-1 uptime is 26 weeks, 1 day, 8 hours, 2 minutes
Uptime for this control processor is 26 weeks, 1 day, 8 hours, 10 minutes
System returned to ROM by power-on
System restarted at 08:50:05 CDT Tue Aug 9 2011
System image file is "bootflash:cat4500-entservicesk9-mz.122-40.SG.bin"


This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
***@cisco.com.

cisco WS-C4510R (REDACTED) processor (revision 4) with 524288K bytes of memory.
Processor board ID REDACTED
MPC8245 CPU at 400Mhz, Supervisor V
Last reset from PowerUp
2 Virtual Ethernet interfaces
48 FastEthernet interfaces
290 Gigabit Ethernet interfaces
511K bytes of non-volatile configuration memory.

Configuration register is 0x2

4507-1#exit
Connection closed by foreign host.
***@rancid:~$


Walt Howd
Network Manager
Information Technology Services
Truman State University
100 East Normal Street
Kirksville, MO 63501

Post by Walt Howd
I'm running rancid 2.3.6-8 with expect 5.44.1.14 on Ubuntu 10.04.3 LTS 32bit.

Is the pager succcessfully disabled at login? test with
clogin -c 'show version' hostname

Post by Walt Howd
diff --git a/cisco/configs/4507-1 b/cisco/configs/4507-1
index 97a63fc..30c6d3e 100644
--- a/cisco/configs/4507-1
+++ b/cisco/configs/4507-1
@@ -2915,7 +2915,7 @@
interface GigabitEthernet6/22
switchport port-security maximum 4
switchport port-security
storm-control broadcast level 10.00
- sstorm-control action shutdown
+ storm-control action shutdown
spanning-tree portfast
!
<snip>
@@ -4704,7 +4704,7 @@
interface GigabitEthernet9/41
switchport mode access
switchport block multicast
switchport block unicast
- switchport port-security maximumm 4
+ switchport port-security maximum 4
switchport port-security
storm-control broadcast level 10.00
storm-control action shutdown
TERM=network;export TERM
umask 027
TMPDIR=/tmp; export TMPDIR
BASEDIR=/var/lib/rancid; export BASEDIR
PATH=/usr/lib/rancid/bin:/usr/bin:/usr/sbin:/bin:/usr/local/bin:/usr/bin; export PATH
CVSROOT=$BASEDIR/CVS; export CVSROOT
LOGDIR=$BASEDIR/logs; export LOGDIR
RCSSYS=git; export RCSSYS
NOPIPE=YES; export NOPIPE
FILTER_PWDS=YES; export FILTER_PWDS
NOCOMMSTR=YES; export NOCOMMSTR
PAR_COUNT=3; export PAR_COUNT
LIST_OF_GROUPS="cisco"
HTMLMAILS=YES; export HTMLMAILS
I've turned on NOPIPE and also turned PAR_COUNT down to 3 but the issue still randomly occurs.
I have also tested manually pulling configs (via TFTP) from the problematic switches and have not found any unexpected differences.
Walt Howd
Network Manager
Information Technology Services
Truman State University
100 East Normal Street
Kirksville, MO 63501
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss

[Walt Howd]

With some help from a kind third party I was able to track this down to a bug with the telnet.krb5 package in Ubuntu/Debian. This was referenced last year on the mailing list:

http://ftp.shrubbery.net/pipermail/rancid-discuss/2011-September/005929.html

To solve this issue, rather then remove the kerberos packages, I changed my default telnet to the netkit version of telnet as show below:

***@rancid:~# update-alternatives --config telnet
There are 2 choices for the alternative telnet (providing /usr/bin/telnet).

Selection Path Priority Status
------------------------------------------------------------
* 0 /usr/bin/telnet.krb5 105 auto mode
1 /usr/bin/telnet.krb5 105 manual mode
2 /usr/bin/telnet.netkit 100 manual mode

Press enter to keep the current choice[*], or type selection number: 2
update-alternatives: using /usr/bin/telnet.netkit to provide /usr/bin/telnet (telnet) in manual mode.

***@rancid:~# update-alternatives --display telnet
telnet - manual mode
link currently points to /usr/bin/telnet.netkit
/usr/bin/telnet.krb5 - priority 105
slave telnet.1.gz: /usr/share/man/man1/telnet.krb5.1.gz
/usr/bin/telnet.netkit - priority 100
slave telnet.1.gz: /usr/share/man/man1/telnet.netkit.1.gz
Current 'best' version is '/usr/bin/telnet.krb5'.

***@rancid:~# ls -sal /etc/alternatives/telnet
0 lrwxrwxrwx 1 root root 22 2012-02-13 15:35 /etc/alternatives/telnet -> /usr/bin/telnet.netkit

Walt Howd
Network Manager
Information Technology Services
Truman State University
100 East Normal Street
Kirksville, MO 63501