[rancid] Rancid 2.3.4 issue

[Nelson, Leo - Dell]

It appears I have two issues.

Cisco NX devices never back up.
New Cisco 2960G not working either.

Any help would be greatly appreciated.

Here is the output from the Rancid log the AS1 is a Nexus 5020 and the IS1 is a 2960.

!Env: ^
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*as1-0408-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
as1-0408-colo3-edc: missed cmd(s): show module,dir usb1:,dir debug:,show debug,show cores vdc-all,show vtp st
atus,show environment fan,show module xbar,show environment power,show inventory,dir usb2:,show vlan,dir vola
tile:,dir bootflash:,dir slot0:,show processes log vdc-all,dir logflash:,show running-config,show environment
temperature,show boot
as1-0408-colo3-edc: End of run not found


!
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*is1-0102-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
is1-0102-colo3-edc: missed cmd(s): dir /all slavedisk2:,dir /all sec-slot2:,show diag,dir /all disk1:,dir /al
l sec-nvram:,dir /all disk2:,dir /all sec-bootflash:,show spe version,dir /all slaveslot2:,dir /all disk0:,di
r /all slaveslot0:,dir /all sec-slot1:,dir /all harddiska:,dir /all slavenvram:,dir /all sec-disk2:,dir /all
slavesup-bootflash:,dir /all sec-disk0:,dir /all harddiskb:,show inventory raw,dir /all slavedisk1:,show modu
le,show controllers,show diagbus,more system:running-config,dir /all slavedisk0:,show debug,dir /all bootflas
h:,dir /all sec-slot0:,dir /all sec-disk1:,write term,show vtp status,dir /all sup-bootflash:,dir /all slot2:
,dir /all harddisk:,dir /all slot0:,dir /all sup-microcode:,show vlan,dir /all slavebootflash:,show controlle
rs cbus,dir /all slaveslot1:,show vlan-switch,show running-config,show c7200,dir /all slot1:
is1-0102-colo3-edc: End of run not found

Leo Nelson
Network Design Sr. Advisor

Information Technology
Catholic Healthcare West

Ph: 480-263-2880
***@chw.edu<mailto:***@chw.edu>

[Nelson, Leo - Dell]

More info. If I run rancid independent of the cron it pulls a config down just fine for both devices.

Leo Nelson
Network Design Sr. Advisor

Information Technology
Catholic Healthcare West

Main: 602-407-3570
***@chw.edu<mailto:***@chw.edu>

From: rancid-discuss-***@shrubbery.net [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of Nelson, Leo - Dell
Sent: Tuesday, January 18, 2011 9:40 AM
To: rancid-***@shrubbery.net
Subject: [rancid] Rancid 2.3.4 issue

It appears I have two issues.

Cisco NX devices never back up.
New Cisco 2960G not working either.

Any help would be greatly appreciated.

Here is the output from the Rancid log the AS1 is a Nexus 5020 and the IS1 is a 2960.

!Env: ^
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*as1-0408-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
as1-0408-colo3-edc: missed cmd(s): show module,dir usb1:,dir debug:,show debug,show cores vdc-all,show vtp st
atus,show environment fan,show module xbar,show environment power,show inventory,dir usb2:,show vlan,dir vola
tile:,dir bootflash:,dir slot0:,show processes log vdc-all,dir logflash:,show running-config,show environment
temperature,show boot
as1-0408-colo3-edc: End of run not found


!
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*is1-0102-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
is1-0102-colo3-edc: missed cmd(s): dir /all slavedisk2:,dir /all sec-slot2:,show diag,dir /all disk1:,dir /al
l sec-nvram:,dir /all disk2:,dir /all sec-bootflash:,show spe version,dir /all slaveslot2:,dir /all disk0:,di
r /all slaveslot0:,dir /all sec-slot1:,dir /all harddiska:,dir /all slavenvram:,dir /all sec-disk2:,dir /all
slavesup-bootflash:,dir /all sec-disk0:,dir /all harddiskb:,show inventory raw,dir /all slavedisk1:,show modu
le,show controllers,show diagbus,more system:running-config,dir /all slavedisk0:,show debug,dir /all bootflas
h:,dir /all sec-slot0:,dir /all sec-disk1:,write term,show vtp status,dir /all sup-bootflash:,dir /all slot2:
,dir /all harddisk:,dir /all slot0:,dir /all sup-microcode:,show vlan,dir /all slavebootflash:,show controlle
rs cbus,dir /all slaveslot1:,show vlan-switch,show running-config,show c7200,dir /all slot1:
is1-0102-colo3-edc: End of run not found

Leo Nelson
Network Design Sr. Advisor

Information Technology
Catholic Healthcare West

Ph: 480-263-2880
***@chw.edu<mailto:***@chw.edu>

[Nelson, Leo - Dell]

Hi John,
I have done this and I only find a single instance of Rancid installed. Here is a snippet from the log file when I run rancid-run. I really can't identify the reason why Rancid work on some devices and not others. I have check the router.db file and dns etc. The first device is IOS, second is NxOS.

Anymore help would be appreciated.

more EDC.20110121.115832
starting: Fri Jan 21 11:58:32 MST 2011

/storage/rancid/dead.letter... Saved message in /storage/rancid/dead.letter
cvs add: is1-0204-colo3-edc already exists, with version number 1.1
Added is1-0204-colo3-edc
cvs add: s1-pscchw already exists, with version number 1.1
Added s1-pscchw
cvs add: s1-sjhmc-edc already exists, with version number 1.89
Added s1-sjhmc-edc
cvs add: s2-sjhmc-edc already exists, with version number 1.107
Added s2-sjhmc-edc



Trying to get all of the configs.
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*as1-0102-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
as1-0102-colo3-edc: missed cmd(s): dir /all slavedisk2:,dir /all sec-slot2:,show diag,dir /all disk1:,dir /al
l sec-nvram:,dir /all disk2:,dir /all sec-bootflash:,show spe version,dir /all slaveslot2:,dir /all disk0:,di
r /all slaveslot0:,dir /all sec-slot1:,dir /all harddiska:,dir /all slavenvram:,dir /all sec-disk2:,dir /all
slavesup-bootflash:,dir /all sec-disk0:,dir /all harddiskb:,show inventory raw,dir /all slavedisk1:,show modu
le,show controllers,show diagbus,more system:running-config,dir /all slavedisk0:,show debug,dir /all bootflas
h:,dir /all sec-slot0:,dir /all sec-disk1:,write term,show vtp status,dir /all sup-bootflash:,dir /all slot2:
,dir /all harddisk:,dir /all slot0:,dir /all sup-microcode:,show vlan,dir /all slavebootflash:,show controlle
rs cbus,dir /all slaveslot1:,show vlan-switch,show running-config,show c7200,dir /all slot1:
as1-0102-colo3-edc: End of run not found
!
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*as1-0108-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
as1-0108-colo3-edc: missed cmd(s): show module,dir usb1:,dir debug:,show debug,show cores vdc-all,show vtp st
atus,show environment fan,show module xbar,show environment power,show inventory,dir usb2:,show vlan,dir vola
tile:,dir bootflash:,dir slot0:,show processes log vdc-all,dir logflash:,show running-config,show environment
temperature,show boot
as1-0108-colo3-edc: End of run not found
!Env: ^

Leo Nelson
Network Design Sr. Advisor
 
Information Technology
Catholic Healthcare West
 
Main: 602-407-3570
***@chw.edu


-----Original Message-----
From: john heasley [mailto:***@shrubbery.net]
Sent: Tuesday, January 18, 2011 12:56 PM
To: Nelson, Leo - Dell
Subject: Re: [rancid] Rancid 2.3.4 issue

Post by Nelson, Leo - Dell
More info. If I run rancid independent of the cron it pulls a config down just fine for both devices.

please make sure that you do not have multiple installations of rancid,
esp clogin, on your machine.

[john heasley]

Post by Nelson, Leo - Dell
Hi John,
I have done this and I only find a single instance of Rancid installed. Here is a snippet from the log file when I run rancid-run. I really can't identify the reason why Rancid work on some devices and not others. I have check the router.db file and dns etc. The first device is IOS, second is NxOS.
Anymore help would be appreciated.
more EDC.20110121.115832
starting: Fri Jan 21 11:58:32 MST 2011
/storage/rancid/dead.letter... Saved message in /storage/rancid/dead.letter
cvs add: is1-0204-colo3-edc already exists, with version number 1.1
Added is1-0204-colo3-edc
cvs add: s1-pscchw already exists, with version number 1.1
Added s1-pscchw
cvs add: s1-sjhmc-edc already exists, with version number 1.89
Added s1-sjhmc-edc
cvs add: s2-sjhmc-edc already exists, with version number 1.107
Added s2-sjhmc-edc

unrelated, but i think you need to fix these two errors first. first, why
is the mail going to dead.letter? are you not able to use sendmail (the
program, whether it be postfix, sendmail, or something else) from this
machine? second, why is rancid trying to cvs add files that apparently
already exist. this is usually because the configure script made an
error determining the proper argument for diff(1) on what i presume is a
linux box or there are two diffs on the machine with differing option
syntax and its finding another due to the PATH that configure constructed
for/in rancid.conf. see routers.all for this group. its should look
similar to
is1-0204-colo3-edc:cisco
s1-pscchw:cisco
....

Post by Nelson, Leo - Dell
Trying to get all of the configs.
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""

this means that either the rancid script (assuming cisco ios) or the network
client program (ssh, telnet, rsh) exited prematurely. given its apprently
from send_user, i suspect its the script that exited, but i have not
inclination why that might be. telnet/ssh is far more likely to exit
prematurely due to the network device closing the connection, etc.

you could try setting NOPIPE to YES in rancid.conf. or try strace'ing cron
to figure out positively whats exiting and possibly why.

Post by Nelson, Leo - Dell
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*as1-0102-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
as1-0102-colo3-edc: missed cmd(s): dir /all slavedisk2:,dir /all sec-slot2:,show diag,dir /all disk1:,dir /al
l sec-nvram:,dir /all disk2:,dir /all sec-bootflash:,show spe version,dir /all slaveslot2:,dir /all disk0:,di
r /all slaveslot0:,dir /all sec-slot1:,dir /all harddiska:,dir /all slavenvram:,dir /all sec-disk2:,dir /all
slavesup-bootflash:,dir /all sec-disk0:,dir /all harddiskb:,show inventory raw,dir /all slavedisk1:,show modu
le,show controllers,show diagbus,more system:running-config,dir /all slavedisk0:,show debug,dir /all bootflas
,dir /all harddisk:,dir /all slot0:,dir /all sup-microcode:,show vlan,dir /all slavebootflash:,show controlle
as1-0102-colo3-edc: End of run not found
!
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*as1-0108-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
as1-0108-colo3-edc: missed cmd(s): show module,dir usb1:,dir debug:,show debug,show cores vdc-all,show vtp st
atus,show environment fan,show module xbar,show environment power,show inventory,dir usb2:,show vlan,dir vola
tile:,dir bootflash:,dir slot0:,show processes log vdc-all,dir logflash:,show running-config,show environment
temperature,show boot
as1-0108-colo3-edc: End of run not found
!Env: ^
Leo Nelson
Network Design Sr. Advisor
?
Information Technology
Catholic Healthcare West
?
Main: 602-407-3570
-----Original Message-----
Sent: Tuesday, January 18, 2011 12:56 PM
To: Nelson, Leo - Dell
Subject: Re: [rancid] Rancid 2.3.4 issue

Post by Nelson, Leo - Dell
More info. If I run rancid independent of the cron it pulls a config down just fine for both devices.

please make sure that you do not have multiple installations of rancid,
esp clogin, on your machine.

[Nelson, Leo - Dell]

Ok it appears the problem I was having was partly the tacacs goup rights I had the user assigned to for the IOS devices. For the NXOS devices, I had to comment out commands that seemed to cause an exit on the 5Ks. Those were;
"show environmrnt clock"
"show cores vdc-all"

As for the dead.letter issue, I have reached out to the previous admin. When I use sendmail from the cmd line, while it still tells me it is going to dead.letter I get the message on my pc.

I would like to make sure I can utilize the mail system fully, so if youhave any suggestions I appreciate it.

FYI
I am NOT a linux admin nor do I claim to know what I am doing, but I have tinkered, can write basic scripts, and can read and understand config files so please be kind. :)

Leo Nelson
Network Design Sr. Advisor
 
Information Technology
Catholic Healthcare West
 
Main: 602-407-3570
***@chw.edu

-----Original Message-----
From: john heasley [mailto:***@shrubbery.net]
Sent: Friday, January 21, 2011 1:36 PM
To: Nelson, Leo - Dell
Cc: john heasley; rancid-***@shrubbery.net
Subject: Re: [rancid] Rancid 2.3.4 issue

Post by Nelson, Leo - Dell
Hi John,
I have done this and I only find a single instance of Rancid installed. Here is a snippet from the log file when I run rancid-run. I really can't identify the reason why Rancid work on some devices and not others. I have check the router.db file and dns etc. The first device is IOS, second is NxOS.
Anymore help would be appreciated.
more EDC.20110121.115832
starting: Fri Jan 21 11:58:32 MST 2011
/storage/rancid/dead.letter... Saved message in /storage/rancid/dead.letter
cvs add: is1-0204-colo3-edc already exists, with version number 1.1
Added is1-0204-colo3-edc
cvs add: s1-pscchw already exists, with version number 1.1
Added s1-pscchw
cvs add: s1-sjhmc-edc already exists, with version number 1.89
Added s1-sjhmc-edc
cvs add: s2-sjhmc-edc already exists, with version number 1.107
Added s2-sjhmc-edc

unrelated, but i think you need to fix these two errors first. first, why
is the mail going to dead.letter? are you not able to use sendmail (the
program, whether it be postfix, sendmail, or something else) from this
machine? second, why is rancid trying to cvs add files that apparently
already exist. this is usually because the configure script made an
error determining the proper argument for diff(1) on what i presume is a
linux box or there are two diffs on the machine with differing option
syntax and its finding another due to the PATH that configure constructed
for/in rancid.conf. see routers.all for this group. its should look
similar to
is1-0204-colo3-edc:cisco
s1-pscchw:cisco
....

Post by Nelson, Leo - Dell
Trying to get all of the configs.
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""

this means that either the rancid script (assuming cisco ios) or the network
client program (ssh, telnet, rsh) exited prematurely. given its apprently
from send_user, i suspect its the script that exited, but i have not
inclination why that might be. telnet/ssh is far more likely to exit
prematurely due to the network device closing the connection, etc.

you could try setting NOPIPE to YES in rancid.conf. or try strace'ing cron
to figure out positively whats exiting and possibly why.

Post by Nelson, Leo - Dell
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*as1-0102-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
as1-0102-colo3-edc: missed cmd(s): dir /all slavedisk2:,dir /all sec-slot2:,show diag,dir /all disk1:,dir /al
l sec-nvram:,dir /all disk2:,dir /all sec-bootflash:,show spe version,dir /all slaveslot2:,dir /all disk0:,di
r /all slaveslot0:,dir /all sec-slot1:,dir /all harddiska:,dir /all slavenvram:,dir /all sec-disk2:,dir /all
slavesup-bootflash:,dir /all sec-disk0:,dir /all harddiskb:,show inventory raw,dir /all slavedisk1:,show modu
le,show controllers,show diagbus,more system:running-config,dir /all slavedisk0:,show debug,dir /all bootflas
,dir /all harddisk:,dir /all slot0:,dir /all sup-microcode:,show vlan,dir /all slavebootflash:,show controlle
as1-0102-colo3-edc: End of run not found
!
write(spawn_id=1): broken pipe
while executing
"send_user -- "$expect_out(buffer)""
invoked from within
"expect -nobrace -re+ { exp_continue } -re {^[^
*]*as1-0108-co([^#>\r\n]+)?[#>](\([^)\r\n]+\))?} { send_user -- "$expect_out(buffer)"
} -re {..."
invoked from within
"expect {
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
-re "^\[^\n\r]*$reprom..."
(procedure "run_commands" line 39)
invoked from within
"run_commands $prompt $command"
("foreach" body line 155)
invoked from within
"foreach router [lrange $argv $i end] {
set router [string tolower $router]
# attempt at platform switching.
set platform ""
send_user ..."
(file "/usr/bin/clogin" line 722)
as1-0108-colo3-edc: missed cmd(s): show module,dir usb1:,dir debug:,show debug,show cores vdc-all,show vtp st
atus,show environment fan,show module xbar,show environment power,show inventory,dir usb2:,show vlan,dir vola
tile:,dir bootflash:,dir slot0:,show processes log vdc-all,dir logflash:,show running-config,show environment
temperature,show boot
as1-0108-colo3-edc: End of run not found
!Env: ^
Leo Nelson
Network Design Sr. Advisor
?
Information Technology
Catholic Healthcare West
?
Main: 602-407-3570
-----Original Message-----
Sent: Tuesday, January 18, 2011 12:56 PM
To: Nelson, Leo - Dell
Subject: Re: [rancid] Rancid 2.3.4 issue

Post by Nelson, Leo - Dell
More info. If I run rancid independent of the cron it pulls a config down just fine for both devices.

please make sure that you do not have multiple installations of rancid,
esp clogin, on your machine.