[rancid] Questions about debugging a failing F5 backup

[Chris Moody]

I'm running into a strange problem with a couple F5 boxes I'm trying to
backup.

Running RANCID v. 2.3.8.

I currently have over over 50 F5 load-balancers that are backing up
beautifully in RANCID. I seem to be running into an issue with a
particular pair of devices however, and am scratching my head trying to
figure out why these systems are failing. This pair is a set of
Viprions running 10.2.2. I have other Viprions that are backing up fine.

In the logs, I continually see the following:
=============================================
Trying to get all of the configs.
lb1-pre-prod-w2-qcy_eth0: missed cmd(s): bigpipe route static show,cat
/config/bigip.license,bigpipe monitor list all
lb2-pre-prod-w2-qcy_eth0: missed cmd(s): bigpipe route static show,cat
/config/bigip.license
=====================================
Getting missed routers: round 1.
lb2-pre-prod-w2-qcy_eth0: missed cmd(s): bigpipe route static show,cat
/config/bigip.license
lb1-pre-prod-w2-qcy_eth0: missed cmd(s): bigpipe route static show,cat
/config/bigip.license,bigpipe monitor list all
=====================================
etc...



However, if I actually login to the device and issue these commands,
they all return data just fine.

I've tried running f5rancid with debugging enabled, but it's not giving
me any clues as it says it's hitting all commands ok.

=============================================
sprdfrypa301 ~ $ f5rancid -d lb1-pre-prod-w2-qcy_eth0
executing clogin -t 90 -c"bigpipe version;bigpipe platform;cat
/config/bigip.license;bigpipe monitor list all;bigpipe profile
list;bigpipe base list;bigpipe db show;bigpipe route static show;bigpipe
list" lb1-pre-prod-w2-qcy_eth0
PROMPT MATCH: \[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby\] ~ #
HIT COMMAND:[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ #
bigpipe version
In ShowVersion: [***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~
# bigpipe version
HIT COMMAND:[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ # bigpipe
platform
In ShowPlatform: [***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~
# bigpipe platform
HIT COMMAND:[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ # cat
/config/bigip.license
In ShowLicense: [***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~
# cat /config/bigip.license
HIT COMMAND:[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ # bigpipe
monitor list all
In ShowMonitor: [***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~
# bigpipe monitor list all
HIT COMMAND:[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ # bigpipe
profile list
In ShowProfile: [***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~
# bigpipe profile list
HIT COMMAND:[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ # bigpipe
base list
In ShowBaseRun: [***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~
# bigpipe base list
HIT COMMAND:[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ # bigpipe
db show
In ShowDb: [***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ #
bigpipe db show
HIT COMMAND:[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ # bigpipe
route static show
In ShowRouteStatic:
[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ # bigpipe route
static show
HIT COMMAND:[***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ # bigpipe
list
In WriteTerm: [***@lb1-pre-prod-w2-qcy:/S1-green-P:Standby] ~ #
bigpipe list
sprdfrypa301 ~ $
=============================================

What are some next steps I can try to nail what's triggering the backup
of these couple devices to fail?

Cheers,
-Chris