[rancid] F5 Rancid | BIGIP 12.0

This problem almost always comes down to the usual issues with cron:

- cron runs from a non-interactive non-login shell, meaning that the
environment is empty (the environment being nothing more than a handy
*user* convenience). Make sure your cron env is set up correctly and
don't run ./rancid-run, specify the full correct path

- which user's cron is this in? It must go in rancid's crontab, not
root. If this is the case with your site, make sure you chown all
relevant data files back to the rancid user as some of them will now be
owned by root

Post by Shaun Krok
Hi
I have tried to look into forums where other users of rancid experience the same issue but could not find anything !
We have been running rancid successfully against F5 LTM clusters (HA) for a few years now.
The issue with partitions and TMSH was resolved and we have had no issues accept with the recent upgrade to BIGIP image : BIG-IP 12.0.0
The issue is below.
What we have noticed that if we use the cli command : ./rancid-run -r lon-f5-pri GLN & the collection has no issues, diff is done, entered into CVS and emails received !
The problem is when the general cron runs and we receive the below in the logs.
Using the NOPIPE I see the .raw file and have looked into the file and do not see any issues.
The f5rancid script has been working as I said for years until version 12.0 of BIGIP.
It works against several other production sites but running version 11.6 of BIGIP
I realize the below is just the process saying there is an issue but if anyone has suggestions would be much appreciated !
Version of rancid : ## rancid 3.2.99
***************************************
a snip of the crontab
# Check every 30 mins
*/30 * * * * /usr/local/rancid/bin/rancid-run &
**************************************
**************************************
A snip of the f5rancid script
# tmsh commands, BIGIP v11
@tmsh_commandtable = (
{'tmsh show /sys version' => 'ShowVersion'},
{'tmsh show /sys hardware' => 'ShowHardware'},
{'tmsh show /sys license' => 'ShowLicense'},
#{'cat /config/ZebOS.conf' => 'ShowZebOSconf'},
#{'lsof -i :179' => 'ShowZebOSsockets'},
{'tmsh show /net route static' => 'ShowRouteStatic'},
#{'ls --full-time --color=never /config/ssl/ssl.crt' => 'ShowSslCrt'},
#{'ls --full-time --color=never /config/ssl/ssl.key' => 'ShowSslKey'},
#{'tmsh -q list' => 'WriteTermTMSH'},
#{'tmsh -q -c /"cd /;list recursive"/' => 'WriteTermTMSH'},
{'./f5part' => 'WriteTermTMSH'},
**************************************
Trying to get all of the configs.
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys version,tmsh show /sys hardware,tmsh show /sys license
=====================================
Getting missed routers: round 1.
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys version,tmsh show /sys hardware,tmsh show /sys license
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
=====================================
Getting missed routers: round 2.
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys version,tmsh show /sys hardware,tmsh show /sys license
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
=====================================
Getting missed routers: round 3.
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys version,tmsh show /sys hardware,tmsh show /sys license
=====================================
Getting missed routers: round 4.
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys version,tmsh show /sys hardware,tmsh show /sys license
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
regards
Shaun Krok
Network Team
Herzliya Business Park
Herzliya Pituach 46140
Israel
Telephone: +972 (0)732889406
Mobile: +972 (0)50 2424381
This email message and its attachments are for the sole use of the intended recipient(s) and may not be shared with any other party. They may contain confidential information of 888 Holdings plc or its direct and indirect subsidiaries (together, the “888 Group”) and are to be regarded as confidential information under any non-disclosure agreement. Any review, use, disclosure or distribution by persons or entities other than the intended recipient(s) is prohibited. Nothing in this message is capable of or intended to create any legally binding obligation. The 888 Group will only ever assume a legally binding obligation where recorded in a written agreement duly executed by the authorized signatories of the relevant 888 Group company. The 888 Group accepts no liability for any personal views expressed in this message. If you are not the intended recipient, please contact the sender by return and destroy all copies of the original message and its attachments. Thank you
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

Shaun Krok

2016-09-12 08:24:54 UTC

Thanks Alan

Always have been running as *user* = rancid
[***@Rancid bin]$ whoami
Rancid

# Check every 30 mins
*/30 * * * * /usr/local/rancid/bin/rancid-run &

What do you suggest with regards setting up the cron environment for rancid --- never had to prior BIGIP 12.0 and all other device types: Cisco, Juniper, Riverbed , F5 working 100%

-----Original Message-----
From: Rancid-discuss [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of Alan McKinnon
Sent: Monday, September 12, 2016 11:15 AM
To: rancid-***@shrubbery.net
Subject: Re: [rancid] F5 Rancid | BIGIP 12.0 | missed cmd(s)

This problem almost always comes down to the usual issues with cron:

- cron runs from a non-interactive non-login shell, meaning that the
environment is empty (the environment being nothing more than a handy
*user* convenience). Make sure your cron env is set up correctly and
don't run ./rancid-run, specify the full correct path

- which user's cron is this in? It must go in rancid's crontab, not
root. If this is the case with your site, make sure you chown all
relevant data files back to the rancid user as some of them will now be
owned by root

_______________________________________________
Rancid-discuss mailing list
Rancid-***@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

This email message and its attachments are for the sole use of the intended recipient(s) and may not be shared with any other party. They may contain confidential information of 888 Holdings plc or its direct and indirect subsidiaries (together, the “888 Group”) and are to be regarded as confidential information under any non-disclosure agreement. Any review, use, disclosure or distribution by persons or entities other than the intended recipient(s) is prohibited. Nothing in this message is capable of or intended to create any legally binding obligation. The 888 Group will only ever assume a legally binding obligation where recorded in a written agreement duly executed by the authorized signatories of the relevant 888 Group company. The 888 Group accepts no liability for any personal views expressed in this message. If you are not the intended recipient, please contact the sender by return and destroy all copies of the original message and its attachments. Thank you

Alan McKinnon

2016-09-12 08:40:41 UTC

Shaun,

Usually setting PATH correctly is enough, but I don't think that is your
problem. To check that I understand the problem correctly:

All your devices work correctly except BIGIP 12. Even BIGIP 11 works
properly. BIGIP 12 fails from a cronjob but works correctly from the
command line as rancid user. This then excludes mistakes in router.db

Correct?

Nothing in cron or the environment strikes me as being unique to the
BIGIP 12. For completeness, what is in the rancid user's environment in
a regular login shell?

If that doesn't reveal anything useful, I'd generate .raw files for both
device types and compare them. Then read those files directly into
rancid; at some point a difference in behaviour must show up

Post by Shaun Krok
Thanks Alan
Always have been running as *user* = rancid
Rancid
# Check every 30 mins
*/30 * * * * /usr/local/rancid/bin/rancid-run &
What do you suggest with regards setting up the cron environment for rancid --- never had to prior BIGIP 12.0 and all other device types: Cisco, Juniper, Riverbed , F5 working 100%
-----Original Message-----
Sent: Monday, September 12, 2016 11:15 AM
Subject: Re: [rancid] F5 Rancid | BIGIP 12.0 | missed cmd(s)
- cron runs from a non-interactive non-login shell, meaning that the
environment is empty (the environment being nothing more than a handy
*user* convenience). Make sure your cron env is set up correctly and
don't run ./rancid-run, specify the full correct path
- which user's cron is this in? It must go in rancid's crontab, not
root. If this is the case with your site, make sure you chown all
relevant data files back to the rancid user as some of them will now be
owned by root

_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
This email message and its attachments are for the sole use of the intended recipient(s) and may not be shared with any other party. They may contain confidential information of 888 Holdings plc or its direct and indirect subsidiaries (together, the “888 Group”) and are to be regarded as confidential information under any non-disclosure agreement. Any review, use, disclosure or distribution by persons or entities other than the intended recipient(s) is prohibited. Nothing in this message is capable of or intended to create any legally binding obligation. The 888 Group will only ever assume a legally binding obligation where recorded in a written agreement duly executed by the authorized signatories of the relevant 888 Group company. The 888 Group accepts no liability for any personal views expressed in this message. If you are not the intended recipient, please contact the sender by return and destroy all copies of the original message and its attachments. Thank you

Daniel Kerse

2016-09-12 12:40:10 UTC

Hello Shaun et.al.,

Are you playing with the TERM environment variable at all?
Have you tried enabling debug on your cron job and then (re)checking:
1. the logs?
2. the resulting .raw files?

I'm also running version 3.2 (the latest available under EPEL) with
some F5s of various vintage.

Firstly to recap what is already known by this mailing list:
- circa BIG-IP v9.x -> v10.x the tmsh shell was introduced in addition
to the pre-existing bigpipe.
- circa BIG-IP v10.x -> v11.x the old bigpipe shell was removed.
- (At least for BIG-IP v11.x and earlier) you must set TERM=vt100-w in
order to run some commands. This problem would only become apparent
when running via CRON, When running rancid manually at the CLI
everything looks honky dory.. In my case the following commands would
fail (via CRON):
"ls --full-time --color=never /config/ssl/ssl.crt"
"ls --full-time --color=never /config/ssl/ssl.key"
"tmsh show /net route static"
- Someone mentioned that you need to run "tmsh -q -c 'cd / \;list
recursive" to get everything out of the partitions.
- Was this circa v11.x? I forget. I need to do more research here
as I've almost got around that issue that had people setting up shell
scripts on their F5s to make it work... more on that later.

Now here's the interesting thing I've noticed on v12.x . Perhaps this
is widely known in the F5 community but I haven't read about it
anywhere.
- Somewhere around BIG-IP v11.x -> v12.x the vt100-w terminal started
misbehaving.
- If I have TERM set to vt100-w and enable debug I see this error
message in the logs:
"Warning, can't fully initialize terminal, TERM is set to
"vt100-w", status (0)"
- Now I think again if you run via CLI it might appear to work OK
but not via CRON.

So now you end up in a difficult situation in BIG-IP v12:
- TERM=vt100-w fails via CRON because it's broken
- TERM=network (or similar) fails via CRON because some commands
require it to stop the junk showing up mid-line.
- Either case works via CLI.

So you can end up with a rather fiddly rancid.types.conf file.
Something like this (work in progress):

# F5s running 9.3.1 seem to lack tmsh support so we will use a
(slightly modified) version of f5rancid
bigip-v9;script;f5rancid-v9
bigip-v9;login;clogin

# F5s running 11.4.1 and 12.0.0 seem to lack bigpipe support.
# For some reason turning off debug flag causes problems for "tmsh -q
-c 'cd / \;list recursive'" I don't know why.
bigip-v10-11;script;rancid -t bigip-v10-11
bigip-v10-11;login;clogin
bigip-v10-11;module;bigip
bigip-v10-11;inloop;bigip::inloop
bigip-v10-11;command;bigip::ShowVersion;tmsh show /sys version
bigip-v10-11;command;bigip::ShowHardware;tmsh show /sys hardware
bigip-v10-11;command;bigip::ShowLicense;tmsh show /sys license
bigip-v10-11;command;bigip::ShowSslCrt;ls --full-time --color=never
/config/ssl/ssl.crt
bigip-v10-11;command;bigip::ShowSslKey;ls --full-time --color=never
/config/ssl/ssl.key
bigip-v10-11;command;bigip::ShowZebOSconf;cat /config/ZebOS.conf
bigip-v10-11;command;bigip::ShowZebOSsockets;lsof -i :179
bigip-v10-11;command;bigip::ShowRouteStatic;tmsh show /net route static
# This one seems to get confused
#bigip-v10-11;command;bigip::WriteTerm;tmsh -q -c 'cd / \;list recursive'
bigip-v10-11;command;bigip::WriteTerm;tmsh -q list

# F5s running 12.0.0 seem to lack support for vt100-w terminal type.
# Error Message:
# "Warning, can't fully initialize terminal, TERM is set to "vt100-w",
status (0)"
# As a result ShowSslCrt, ShowSslKey and ShowRouteStatic don't work.
# For some reason turning off debug flag causes problems for "tmsh -q
-c 'cd / \;list recursive'" I don't know why.
bigip-v12;script;rancid -dt bigip-v12
bigip-v12;login;clogin
bigip-v12;module;bigip12
bigip-v12;inloop;bigip12::inloop
bigip-v12;command;bigip12::ShowVersion;tmsh show /sys version
bigip-v12;command;bigip12::ShowHardware;tmsh show /sys hardware
bigip-v12;command;bigip12::ShowLicense;tmsh show /sys license
#bigip-v12;command;bigip12::ShowSslCrt;ls --full-time --color=never
/config/ssl/ssl.crt
#bigip-v12;command;bigip12::ShowSslKey;ls --full-time --color=never
/config/ssl/ssl.key
bigip-v12;command;bigip12::ShowZebOSconf;cat /config/ZebOS.conf
bigip-v12;command;bigip12::ShowZebOSsockets;lsof -i :179
#bigip-v12;command;bigip12::ShowRouteStatic;tmsh show /net route static
#bigip-v12;command;bigip12::WriteTerm;tmsh -q -c 'cd / \;list recursive'
bigip-v12;command;bigip12::WriteTerm;tmsh -q list

# Huawei support not provided out of the box, added after the fact.
# Let me know if you want me to post this, it's a bit OT for this thread..
huawei;script;rancid -t huawei
huawei;login;hulogin
huawei;module;huawei
huawei;inloop;huawei::inloop
huawei;command;huawei::DisplayVersion;display version
huawei;command;huawei::DisplayPatchInfo;display patch-information
huawei;command;huawei::DisplayDevice;display esn
huawei;command;huawei::DisplayDevice;display device
huawei;command;huawei::DisplayDevice;display device manufacture-info
huawei;command;huawei::DisplayDevice;display device pic-status
huawei;command;huawei::DisplayElabel;display device elabel
huawei;command;huawei::DisplayElabel;display elabel
huawei;command;huawei::DisplayTransceiver;display interface transceiver
huawei;command;huawei::DisplayLicense;display license
huawei;command;huawei::WriteTerm;display current-configuration

With also the following customizations:

1) To make "tmsh -q -c 'cd / \;list recursive'" do something useful
the following change to rancid.pm is required, otherwise the split is
too aggressive.
*******
< my($type, $directive, $value, $value2) = split('\;');
---

#my($type, $directive, $value, $value2) = split('\;');
my($type, $directive, $value, $value2) = split('\;', $_, 4);

*******
2) Following changes made to bigip.pm (from rancid 3.5). There might
be a bug here still as uncommenting the ";tmsh -q -c 'cd / \;list
recursive'" line above seems to lead to unnecessary reattempts at
config collection - but only when debug is disabled for some reason.
*******
5c5
< ## rancid 3.5
---

## rancid 3.5 plus some customisations.

66c66,67
< use rancid 3.5;
---

#use rancid 3.5;
use rancid 3.2;

74c75
< $ENV{'TERM'} = "vt100";
---

$ENV{'TERM'} = "vt100-w";

183a185

s/^\ \ ([0-9]+)(\ +).*Air\ Outlet/ $1$2REMOVED Air Outlet/i;

184a187,188

s/^\ \ ([0-9]+)(\ +).*ADM1026/ $1$2REMOVED ADM1026/i;
s/^\ \ ([0-9]+)(\ +).*Main\ board(.*)$/ $1$2REMOVED Main board$3/i;

185a190

s/^\ \ ([0-9]+)(\ +)[0-9]+\ +[0-9]+\ +[0-9]+/ $1$2REMOVED REMOVED REMOVED/;

218a224,272

# This routine parses "ls --full-time --color=never /config/ssl/ssl.crt"
sub ShowSslCrt {
my($line) = (0);
print STDERR " In ShowSslCrt: $_" if ($debug);
while (<$INPUT>) {
tr/\015//d;
# v9 software license does not have CR at EOF
s/^#-+($prompt.*)/$1/;
last if (/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if /^\s*\^\s*$/;
return(1) if /(Invalid input detected|Type help or )/;
return(-1) if (/command authorization failed/i);
if (!$line++) {
ProcessHistory("ShowSslCrt","","","#\n#/config/ssl/ssl.crt:\n");
}
ProcessHistory("ShowSslCrt","","","# $_") && next;
}
return(0);
}
# This routine parses "ls --full-time --color=never /config/ssl/ssl.key"
sub ShowSslKey {
my($line) = (0);
print STDERR " In ShowSslKey: $_" if ($debug);
while (<$INPUT>) {
tr/\015//d;
# v9 software license does not have CR at EOF
s/^#-+($prompt.*)/$1/;
last if (/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if /^\s*\^\s*$/;
return(1) if /(Invalid input detected|Type help or )/;
return(-1) if (/command authorization failed/i);
if (!$line++) {
ProcessHistory("ShowSslKey","","","#\n#/config/ssl/ssl.key:\n");
}
ProcessHistory("ShowSslKey","","","# $_") && next;
}
return(0);
}

290a345,347

return (1) if (/Syntax Error: unexpected argument/);
return (0) if ($found_end); # Only run this routine once.

297c354
< $found_end++;
---

$found_end = 1;

*******
Also:
*******
$ diff bigip.pm bigip12.pm
1c1
< package bigip;
---

package bigip12;

75c75
< $ENV{'TERM'} = "vt100-w";
---

$ENV{'TERM'} = "vt100";

*******
And:
*******
$ diff f5rancid f5rancid-v9
64c64,65
< $ENV{'TERM'} = "vt100";
---

#$ENV{'TERM'} = "vt100";
$ENV{'TERM'} = "vt100-w";

186a188,191

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("SHOWBASE","","","# $1community <removed>\n");
next;
}

190a196,199

if (/^(\s*)password crypt \S+/) {
ProcessHistory("SHOWBASE","","","# $1password crypt <removed>\n");
next;
}

225c234,236
<
---

if (/^(.*)\.password = / && $filter_pwds >= 1) {
ProcessHistory("SHOWDB","","","# $1.password = <removed>\n") && next;
}

269a281,286

if (/^(\s*)monitor state (up|down)$/) {
ProcessHistory("SHOWDB","","","# $1monitor state <removed>\n") && next;
}
if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("SHOWDB","","","# $1community <removed>\n") && next;
}

277c294
< if (/^(\s*)password / && $filter_pwds >= 1) {
---

if (/^(\s*)password (\w*)/ && $filter_pwds >= 1 && $2 ne "none") {

427a445,447

return (1) if (/BIGpipe:.*: syntax error/);
return (0) if ($found_end); # Only do this routine once

441a462,465

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("","","","# $1community <removed>\n") && next;
}

532a557

{'bigpipe list all' => 'WriteTerm'},

*******

Kind Regards,
Dan

Shaun,
Usually setting PATH correctly is enough, but I don't think that is your
All your devices work correctly except BIGIP 12. Even BIGIP 11 works
properly. BIGIP 12 fails from a cronjob but works correctly from the command
line as rancid user. This then excludes mistakes in router.db
Correct?
Nothing in cron or the environment strikes me as being unique to the BIGIP
12. For completeness, what is in the rancid user's environment in a regular
login shell?
If that doesn't reveal anything useful, I'd generate .raw files for both
device types and compare them. Then read those files directly into rancid;
at some point a difference in behaviour must show up

Post by Shaun Krok
Thanks Alan
Always have been running as *user* = rancid
Rancid
# Check every 30 mins
*/30 * * * * /usr/local/rancid/bin/rancid-run &
What do you suggest with regards setting up the cron environment for
rancid --- never had to prior BIGIP 12.0 and all other device types: Cisco,
Juniper, Riverbed , F5 working 100%
-----Original Message-----
Behalf Of Alan McKinnon
Sent: Monday, September 12, 2016 11:15 AM
Subject: Re: [rancid] F5 Rancid | BIGIP 12.0 | missed cmd(s)
- cron runs from a non-interactive non-login shell, meaning that the
environment is empty (the environment being nothing more than a handy
*user* convenience). Make sure your cron env is set up correctly and
don't run ./rancid-run, specify the full correct path
- which user's cron is this in? It must go in rancid's crontab, not
root. If this is the case with your site, make sure you chown all
relevant data files back to the rancid user as some of them will now be
owned by root

Post by Shaun Krok
Hi
I have tried to look into forums where other users of rancid experience
the same issue but could not find anything !
We have been running rancid successfully against F5 LTM clusters (HA) for
a few years now.
The issue with partitions and TMSH was resolved and we have had no issues
accept with the recent upgrade to BIGIP image : BIG-IP 12.0.0
The issue is below.
What we have noticed that if we use the cli command : ./rancid-run -r
lon-f5-pri GLN & the collection has no issues, diff is done, entered into
CVS and emails received !
The problem is when the general cron runs and we receive the below in the logs.
Using the NOPIPE I see the .raw file and have looked into the file and do
not see any issues.
The f5rancid script has been working as I said for years until version 12.0 of BIGIP.
It works against several other production sites but running version 11.6 of BIGIP
I realize the below is just the process saying there is an issue but if
anyone has suggestions would be much appreciated !
Version of rancid : ## rancid 3.2.99
***************************************
a snip of the crontab
# Check every 30 mins
*/30 * * * * /usr/local/rancid/bin/rancid-run &
**************************************
**************************************
A snip of the f5rancid script
# tmsh commands, BIGIP v11
@tmsh_commandtable = (
{'tmsh show /sys version' => 'ShowVersion'},
{'tmsh show /sys hardware' => 'ShowHardware'},
{'tmsh show /sys license' => 'ShowLicense'},
#{'cat /config/ZebOS.conf' => 'ShowZebOSconf'},
#{'lsof -i :179' => 'ShowZebOSsockets'},
{'tmsh show /net route static' => 'ShowRouteStatic'},
#{'ls --full-time --color=never /config/ssl/ssl.crt' => 'ShowSslCrt'},
#{'ls --full-time --color=never /config/ssl/ssl.key' => 'ShowSslKey'},
#{'tmsh -q list' => 'WriteTermTMSH'},
#{'tmsh -q -c /"cd /;list recursive"/' => 'WriteTermTMSH'},
{'./f5part' => 'WriteTermTMSH'},
**************************************
Trying to get all of the configs.
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys
version,tmsh show /sys hardware,tmsh show /sys license
=====================================
Getting missed routers: round 1.
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys
version,tmsh show /sys hardware,tmsh show /sys license
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
=====================================
Getting missed routers: round 2.
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys
version,tmsh show /sys hardware,tmsh show /sys license
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
=====================================
Getting missed routers: round 3.
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys
version,tmsh show /sys hardware,tmsh show /sys license
=====================================
Getting missed routers: round 4.
lon-f5-sec: missed cmd(s): tmsh show /net route static,tmsh show /sys
version,tmsh show /sys hardware,tmsh show /sys license
lon-f5-pri: missed cmd(s): tmsh show /net route static,tmsh show /sys hardware
regards
Shaun Krok
Network Team
Herzliya Business Park
Herzliya Pituach 46140
Israel
Telephone: +972 (0)732889406
Mobile: +972 (0)50 2424381
This email message and its attachments are for the sole use of the
intended recipient(s) and may not be shared with any other party. They may
contain confidential information of 888 Holdings plc or its direct and
indirect subsidiaries (together, the “888 Group”) and are to be regarded as
confidential information under any non-disclosure agreement. Any review,
use, disclosure or distribution by persons or entities other than the
intended recipient(s) is prohibited. Nothing in this message is capable of
or intended to create any legally binding obligation. The 888 Group will
only ever assume a legally binding obligation where recorded in a written
agreement duly executed by the authorized signatories of the relevant 888
Group company. The 888 Group accepts no liability for any personal views
expressed in this message. If you are not the intended recipient, please
contact the sender by return and destroy all copies of the original message
and its attachments. Thank you
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
This email message and its attachments are for the sole use of the
intended recipient(s) and may not be shared with any other party. They may
contain confidential information of 888 Holdings plc or its direct and
indirect subsidiaries (together, the “888 Group”) and are to be regarded as
confidential information under any non-disclosure agreement. Any review,
use, disclosure or distribution by persons or entities other than the
intended recipient(s) is prohibited. Nothing in this message is capable of
or intended to create any legally binding obligation. The 888 Group will
only ever assume a legally binding obligation where recorded in a written
agreement duly executed by the authorized signatories of the relevant 888
Group company. The 888 Group accepts no liability for any personal views
expressed in this message. If you are not the intended recipient, please
contact the sender by return and destroy all copies of the original message
and its attachments. Thank you

_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

heasley

2016-09-12 19:36:41 UTC

Post by Daniel Kerse
Hello Shaun et.al.,
Are you playing with the TERM environment variable at all?
1. the logs?
2. the resulting .raw files?
I'm also running version 3.2 (the latest available under EPEL) with
some F5s of various vintage.
- circa BIG-IP v9.x -> v10.x the tmsh shell was introduced in addition
to the pre-existing bigpipe.
- circa BIG-IP v10.x -> v11.x the old bigpipe shell was removed.
- (At least for BIG-IP v11.x and earlier) you must set TERM=vt100-w in
order to run some commands. This problem would only become apparent

Do you mean that it required that specific term type, or just that it
required that term be set to something other than dumb or network?

Post by Daniel Kerse
when running via CRON, When running rancid manually at the CLI
everything looks honky dory.. In my case the following commands would
"ls --full-time --color=never /config/ssl/ssl.crt"
"ls --full-time --color=never /config/ssl/ssl.key"
"tmsh show /net route static"
- Someone mentioned that you need to run "tmsh -q -c 'cd / \;list
recursive" to get everything out of the partitions.
- Was this circa v11.x? I forget. I need to do more research here
as I've almost got around that issue that had people setting up shell
scripts on their F5s to make it work... more on that later.
Now here's the interesting thing I've noticed on v12.x . Perhaps this
is widely known in the F5 community but I haven't read about it
anywhere.
- Somewhere around BIG-IP v11.x -> v12.x the vt100-w terminal started
misbehaving.
- If I have TERM set to vt100-w and enable debug I see this error
"Warning, can't fully initialize terminal, TERM is set to
"vt100-w", status (0)"

that implies that the type is unknown or there is no pty.

Post by Daniel Kerse
- Now I think again if you run via CLI it might appear to work OK
but not via CRON.
- TERM=vt100-w fails via CRON because it's broken
- TERM=network (or similar) fails via CRON because some commands
require it to stop the junk showing up mid-line.
- Either case works via CLI.
So you can end up with a rather fiddly rancid.types.conf file.
# F5s running 9.3.1 seem to lack tmsh support so we will use a
(slightly modified) version of f5rancid
bigip-v9;script;f5rancid-v9
bigip-v9;login;clogin
# F5s running 11.4.1 and 12.0.0 seem to lack bigpipe support.
# For some reason turning off debug flag causes problems for "tmsh -q
-c 'cd / \;list recursive'" I don't know why.
bigip-v10-11;script;rancid -t bigip-v10-11
bigip-v10-11;login;clogin
bigip-v10-11;module;bigip
bigip-v10-11;inloop;bigip::inloop
bigip-v10-11;command;bigip::ShowVersion;tmsh show /sys version
bigip-v10-11;command;bigip::ShowHardware;tmsh show /sys hardware
bigip-v10-11;command;bigip::ShowLicense;tmsh show /sys license
bigip-v10-11;command;bigip::ShowSslCrt;ls --full-time --color=never
/config/ssl/ssl.crt
bigip-v10-11;command;bigip::ShowSslKey;ls --full-time --color=never
/config/ssl/ssl.key
bigip-v10-11;command;bigip::ShowZebOSconf;cat /config/ZebOS.conf
bigip-v10-11;command;bigip::ShowZebOSsockets;lsof -i :179
bigip-v10-11;command;bigip::ShowRouteStatic;tmsh show /net route static
# This one seems to get confused
#bigip-v10-11;command;bigip::WriteTerm;tmsh -q -c 'cd / \;list recursive'
bigip-v10-11;command;bigip::WriteTerm;tmsh -q list
# F5s running 12.0.0 seem to lack support for vt100-w terminal type.
# "Warning, can't fully initialize terminal, TERM is set to "vt100-w",
status (0)"
# As a result ShowSslCrt, ShowSslKey and ShowRouteStatic don't work.
# For some reason turning off debug flag causes problems for "tmsh -q
-c 'cd / \;list recursive'" I don't know why.
bigip-v12;script;rancid -dt bigip-v12
bigip-v12;login;clogin
bigip-v12;module;bigip12
bigip-v12;inloop;bigip12::inloop
bigip-v12;command;bigip12::ShowVersion;tmsh show /sys version
bigip-v12;command;bigip12::ShowHardware;tmsh show /sys hardware
bigip-v12;command;bigip12::ShowLicense;tmsh show /sys license
#bigip-v12;command;bigip12::ShowSslCrt;ls --full-time --color=never
/config/ssl/ssl.crt
#bigip-v12;command;bigip12::ShowSslKey;ls --full-time --color=never
/config/ssl/ssl.key
bigip-v12;command;bigip12::ShowZebOSconf;cat /config/ZebOS.conf
bigip-v12;command;bigip12::ShowZebOSsockets;lsof -i :179
#bigip-v12;command;bigip12::ShowRouteStatic;tmsh show /net route static
#bigip-v12;command;bigip12::WriteTerm;tmsh -q -c 'cd / \;list recursive'
bigip-v12;command;bigip12::WriteTerm;tmsh -q list
# Huawei support not provided out of the box, added after the fact.
# Let me know if you want me to post this, it's a bit OT for this thread..
huawei;script;rancid -t huawei
huawei;login;hulogin
huawei;module;huawei
huawei;inloop;huawei::inloop
huawei;command;huawei::DisplayVersion;display version
huawei;command;huawei::DisplayPatchInfo;display patch-information
huawei;command;huawei::DisplayDevice;display esn
huawei;command;huawei::DisplayDevice;display device
huawei;command;huawei::DisplayDevice;display device manufacture-info
huawei;command;huawei::DisplayDevice;display device pic-status
huawei;command;huawei::DisplayElabel;display device elabel
huawei;command;huawei::DisplayElabel;display elabel
huawei;command;huawei::DisplayTransceiver;display interface transceiver
huawei;command;huawei::DisplayLicense;display license
huawei;command;huawei::WriteTerm;display current-configuration
1) To make "tmsh -q -c 'cd / \;list recursive'" do something useful
the following change to rancid.pm is required, otherwise the split is
too aggressive.
*******
< my($type, $directive, $value, $value2) = split('\;');
---

#my($type, $directive, $value, $value2) = split('\;');
my($type, $directive, $value, $value2) = split('\;', $_, 4);

## rancid 3.5 plus some customisations.

66c66,67
< use rancid 3.5;
---

#use rancid 3.5;
use rancid 3.2;

74c75
< $ENV{'TERM'} = "vt100";
---

$ENV{'TERM'} = "vt100-w";

183a185

s/^\ \ ([0-9]+)(\ +).*Air\ Outlet/ $1$2REMOVED Air Outlet/i;

184a187,188

s/^\ \ ([0-9]+)(\ +).*ADM1026/ $1$2REMOVED ADM1026/i;
s/^\ \ ([0-9]+)(\ +).*Main\ board(.*)$/ $1$2REMOVED Main board$3/i;

185a190

s/^\ \ ([0-9]+)(\ +)[0-9]+\ +[0-9]+\ +[0-9]+/ $1$2REMOVED REMOVED REMOVED/;

218a224,272

290a345,347

return (1) if (/Syntax Error: unexpected argument/);
return (0) if ($found_end); # Only run this routine once.

297c354
< $found_end++;
---

$found_end = 1;

*******
*******
$ diff bigip.pm bigip12.pm
1c1
< package bigip;
---

package bigip12;

75c75
< $ENV{'TERM'} = "vt100-w";
---

$ENV{'TERM'} = "vt100";

*******
*******
$ diff f5rancid f5rancid-v9
64c64,65
< $ENV{'TERM'} = "vt100";
---

#$ENV{'TERM'} = "vt100";
$ENV{'TERM'} = "vt100-w";

186a188,191

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("SHOWBASE","","","# $1community <removed>\n");
next;
}

190a196,199

if (/^(\s*)password crypt \S+/) {
ProcessHistory("SHOWBASE","","","# $1password crypt <removed>\n");
next;
}

225c234,236
<
---

if (/^(.*)\.password = / && $filter_pwds >= 1) {
ProcessHistory("SHOWDB","","","# $1.password = <removed>\n") && next;
}

269a281,286

277c294
< if (/^(\s*)password / && $filter_pwds >= 1) {
---

if (/^(\s*)password (\w*)/ && $filter_pwds >= 1 && $2 ne "none") {

427a445,447

return (1) if (/BIGpipe:.*: syntax error/);
return (0) if ($found_end); # Only do this routine once

441a462,465

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("","","","# $1community <removed>\n") && next;
}

532a557

{'bigpipe list all' => 'WriteTerm'},

*******

Daniel Kerse

2016-09-12 22:47:32 UTC

Do you mean that it required that specific term type, or just that it
required that term be set to something other than dumb or network?

That specific terminal type is required in some cases. I think if the
prompt plus the command gets longer than something like 80 characters you
start getting control characters showing up in the middle of that line
which causes a regex match to fail.

This has been the topic of previous discussion:
http://www.shrubbery.net/pipermail/rancid-discuss/2014-July/007767.html
http://www.shrubbery.net/pipermail/rancid-discuss/2015-February/008038.html
http://www.shrubbery.net/pipermail/rancid-discuss/2015-July/008603.html

I believe the "cd / ; list recursive" might required on 11.x (and
presumably later) with partitions.
http://www.shrubbery.net/pipermail/rancid-discuss/2014-August/007790.html
If I ever get this to fallback gracefully to "list" I'll let you know.

Post by Daniel Kerse
Now here's the interesting thing I've noticed on v12.x . Perhaps this
is widely known in the F5 community but I haven't read about it
anywhere.
- Somewhere around BIG-IP v11.x -> v12.x the vt100-w terminal started
misbehaving.
- If I have TERM set to vt100-w and enable debug I see this error
"Warning, can't fully initialize terminal, TERM is set to
"vt100-w", status (0)"

that implies that the type is unknown or there is no pty.

I tend to agree. Maybe this type is unknown to more recent versions of
BIG-IP despite working in older releases.

thread..

Post by Daniel Kerse
huawei;script;rancid -t huawei
huawei;login;hulogin
huawei;module;huawei
huawei;inloop;huawei::inloop
huawei;command;huawei::DisplayVersion;display version
huawei;command;huawei::DisplayPatchInfo;display patch-information
huawei;command;huawei::DisplayDevice;display esn
huawei;command;huawei::DisplayDevice;display device
huawei;command;huawei::DisplayDevice;display device manufacture-info
huawei;command;huawei::DisplayDevice;display device pic-status
huawei;command;huawei::DisplayElabel;display device elabel
huawei;command;huawei::DisplayElabel;display elabel
huawei;command;huawei::DisplayTransceiver;display interface transceiver
huawei;command;huawei::DisplayLicense;display license
huawei;command;huawei::WriteTerm;display current-configuration
1) To make "tmsh -q -c 'cd / \;list recursive'" do something useful
the following change to rancid.pm is required, otherwise the split is
too aggressive.
*******
< my($type, $directive, $value, $value2) = split('\;');
---

#my($type, $directive, $value, $value2) = split('\;');
my($type, $directive, $value, $value2) = split('\;', $_, 4);

## rancid 3.5 plus some customisations.

66c66,67
< use rancid 3.5;
---

#use rancid 3.5;
use rancid 3.2;

74c75
< $ENV{'TERM'} = "vt100";
---

$ENV{'TERM'} = "vt100-w";

183a185

s/^\ \ ([0-9]+)(\ +).*Air\ Outlet/ $1$2REMOVED Air Outlet/i;

184a187,188

s/^\ \ ([0-9]+)(\ +).*ADM1026/ $1$2REMOVED ADM1026/i;
s/^\ \ ([0-9]+)(\ +).*Main\ board(.*)$/ $1$2REMOVED Main

board$3/i;

Post by Daniel Kerse
185a190

s/^\ \ ([0-9]+)(\ +)[0-9]+\ +[0-9]+\ +[0-9]+/ $1$2REMOVED

REMOVED REMOVED/;

Post by Daniel Kerse
218a224,272

# This routine parses "ls --full-time --color=never

/config/ssl/ssl.crt"

sub ShowSslCrt {
my($line) = (0);
print STDERR " In ShowSslCrt: $_" if ($debug);
while (<$INPUT>) {
tr/\015//d;
# v9 software license does not have CR at EOF
s/^#-+($prompt.*)/$1/;
last if (/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if /^\s*\^\s*$/;
return(1) if /(Invalid input detected|Type help or )/;
return(-1) if (/command authorization failed/i);
if (!$line++) {

ProcessHistory("ShowSslCrt","","","#\n#/config/ssl/ssl.crt:\n");

}
ProcessHistory("ShowSslCrt","","","# $_") && next;
}
return(0);
}
# This routine parses "ls --full-time --color=never

/config/ssl/ssl.key"

sub ShowSslKey {
my($line) = (0);
print STDERR " In ShowSslKey: $_" if ($debug);
while (<$INPUT>) {
tr/\015//d;
# v9 software license does not have CR at EOF
s/^#-+($prompt.*)/$1/;
last if (/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if /^\s*\^\s*$/;
return(1) if /(Invalid input detected|Type help or )/;
return(-1) if (/command authorization failed/i);
if (!$line++) {

ProcessHistory("ShowSslKey","","","#\n#/config/ssl/ssl.key:\n");

}
ProcessHistory("ShowSslKey","","","# $_") && next;
}
return(0);
}

290a345,347

return (1) if (/Syntax Error: unexpected argument/);
return (0) if ($found_end); # Only run this routine once.

297c354
< $found_end++;
---

$found_end = 1;

*******
*******
$ diff bigip.pm bigip12.pm
1c1
< package bigip;
---

package bigip12;

75c75
< $ENV{'TERM'} = "vt100-w";
---

$ENV{'TERM'} = "vt100";

*******
*******
$ diff f5rancid f5rancid-v9
64c64,65
< $ENV{'TERM'} = "vt100";
---

#$ENV{'TERM'} = "vt100";
$ENV{'TERM'} = "vt100-w";

186a188,191

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("SHOWBASE","","","# $1community <removed>\n");
next;
}

190a196,199

if (/^(\s*)password crypt \S+/) {
ProcessHistory("SHOWBASE","","","# $1password crypt

<removed>\n");

next;
}

225c234,236
<
---

if (/^(.*)\.password = / && $filter_pwds >= 1) {
ProcessHistory("SHOWDB","","","# $1.password = <removed>\n")

&& next;

}

269a281,286

if (/^(\s*)monitor state (up|down)$/) {
ProcessHistory("SHOWDB","","","# $1monitor state

<removed>\n") && next;

}
if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("SHOWDB","","","# $1community <removed>\n")

&& next;

}

277c294
< if (/^(\s*)password / && $filter_pwds >= 1) {
---

if (/^(\s*)password (\w*)/ && $filter_pwds >= 1 && $2 ne "none")

{

Post by Daniel Kerse
427a445,447

return (1) if (/BIGpipe:.*: syntax error/);
return (0) if ($found_end); # Only do this routine

once

Post by Daniel Kerse
441a462,465

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("","","","# $1community <removed>\n") &&

next;

}

532a557

{'bigpipe list all' => 'WriteTerm'},

*******

Kind Regards,
Dan

--
Kind Regards,
Dan Kerse
+64 29 920 3745

Shaun Krok

2016-09-13 05:41:47 UTC

Thanks Daniel

We have been using TERM=vt100-w since moving into 11.4 and greater
In 11.6 rancid spits out a diff error :

+ #Warning, can't fully initialize terminal, TERM is set to "vt100-w", status (0) - But it still works

Since moving to BIGIP 12.0 same error above but now missed cmd(s)

I have shortened the prompt on the F5 because when the cluster is in SYNC we have no missed cmd(s) and a diff is created running on version 12.0
Waiting for the next cron run and will advise what happenes !

Old:
cli preference {
prompt { host user status current-folder config-sync-status }
New:
cli preference {
alias-path { /Common }
prompt { host user status config-sync-status }

From: Rancid-discuss [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of Daniel Kerse
Sent: Tuesday, September 13, 2016 1:48 AM
To: heasley <***@shrubbery.net>
Cc: rancid-***@shrubbery.net; Nati Danan <***@888holdings.com>
Subject: Re: [rancid] F5 Rancid | BIGIP 12.0 | missed cmd(s)

Hello Shaun et.al<http://et.al>.,
Are you playing with the TERM environment variable at all?
1. the logs?
2. the resulting .raw files?
I'm also running version 3.2 (the latest available under EPEL) with
some F5s of various vintage.
- circa BIG-IP v9.x -> v10.x the tmsh shell was introduced in addition
to the pre-existing bigpipe.
- circa BIG-IP v10.x -> v11.x the old bigpipe shell was removed.
- (At least for BIG-IP v11.x and earlier) you must set TERM=vt100-w in
order to run some commands. This problem would only become apparent

Do you mean that it required that specific term type, or just that it
required that term be set to something other than dumb or network?

That specific terminal type is required in some cases. I think if the prompt plus the command gets longer than something like 80 characters you start getting control characters showing up in the middle of that line which causes a regex match to fail.

This has been the topic of previous discussion:
http://www.shrubbery.net/pipermail/rancid-discuss/2014-July/007767.html
http://www.shrubbery.net/pipermail/rancid-discuss/2015-February/008038.html
http://www.shrubbery.net/pipermail/rancid-discuss/2015-July/008603.html

when running via CRON, When running rancid manually at the CLI
everything looks honky dory.. In my case the following commands would
"ls --full-time --color=never /config/ssl/ssl.crt"
"ls --full-time --color=never /config/ssl/ssl.key"
"tmsh show /net route static"
- Someone mentioned that you need to run "tmsh -q -c 'cd / \;list
recursive" to get everything out of the partitions.
- Was this circa v11.x? I forget. I need to do more research here
as I've almost got around that issue that had people setting up shell
scripts on their F5s to make it work... more on that later.

I believe the "cd / ; list recursive" might required on 11.x (and presumably later) with partitions.
http://www.shrubbery.net/pipermail/rancid-discuss/2014-August/007790.html
If I ever get this to fallback gracefully to "list" I'll let you know.

Now here's the interesting thing I've noticed on v12.x . Perhaps this
is widely known in the F5 community but I haven't read about it
anywhere.
- Somewhere around BIG-IP v11.x -> v12.x the vt100-w terminal started
misbehaving.
- If I have TERM set to vt100-w and enable debug I see this error
"Warning, can't fully initialize terminal, TERM is set to
"vt100-w", status (0)"

that implies that the type is unknown or there is no pty.

I tend to agree. Maybe this type is unknown to more recent versions of BIG-IP despite working in older releases.

- Now I think again if you run via CLI it might appear to work OK
but not via CRON.
- TERM=vt100-w fails via CRON because it's broken
- TERM=network (or similar) fails via CRON because some commands
require it to stop the junk showing up mid-line.
- Either case works via CLI.
So you can end up with a rather fiddly rancid.types.conf file.
# F5s running 9.3.1 seem to lack tmsh support so we will use a
(slightly modified) version of f5rancid
bigip-v9;script;f5rancid-v9
bigip-v9;login;clogin
# F5s running 11.4.1 and 12.0.0 seem to lack bigpipe support.
# For some reason turning off debug flag causes problems for "tmsh -q
-c 'cd / \;list recursive'" I don't know why.
bigip-v10-11;script;rancid -t bigip-v10-11
bigip-v10-11;login;clogin
bigip-v10-11;module;bigip
bigip-v10-11;inloop;bigip::inloop
bigip-v10-11;command;bigip::ShowVersion;tmsh show /sys version
bigip-v10-11;command;bigip::ShowHardware;tmsh show /sys hardware
bigip-v10-11;command;bigip::ShowLicense;tmsh show /sys license
bigip-v10-11;command;bigip::ShowSslCrt;ls --full-time --color=never
/config/ssl/ssl.crt
bigip-v10-11;command;bigip::ShowSslKey;ls --full-time --color=never
/config/ssl/ssl.key
bigip-v10-11;command;bigip::ShowZebOSconf;cat /config/ZebOS.conf
bigip-v10-11;command;bigip::ShowZebOSsockets;lsof -i :179
bigip-v10-11;command;bigip::ShowRouteStatic;tmsh show /net route static
# This one seems to get confused
#bigip-v10-11;command;bigip::WriteTerm;tmsh -q -c 'cd / \;list recursive'
bigip-v10-11;command;bigip::WriteTerm;tmsh -q list
# F5s running 12.0.0 seem to lack support for vt100-w terminal type.
# "Warning, can't fully initialize terminal, TERM is set to "vt100-w",
status (0)"
# As a result ShowSslCrt, ShowSslKey and ShowRouteStatic don't work.
# For some reason turning off debug flag causes problems for "tmsh -q
-c 'cd / \;list recursive'" I don't know why.
bigip-v12;script;rancid -dt bigip-v12
bigip-v12;login;clogin
bigip-v12;module;bigip12
bigip-v12;inloop;bigip12::inloop
bigip-v12;command;bigip12::ShowVersion;tmsh show /sys version
bigip-v12;command;bigip12::ShowHardware;tmsh show /sys hardware
bigip-v12;command;bigip12::ShowLicense;tmsh show /sys license
#bigip-v12;command;bigip12::ShowSslCrt;ls --full-time --color=never
/config/ssl/ssl.crt
#bigip-v12;command;bigip12::ShowSslKey;ls --full-time --color=never
/config/ssl/ssl.key
bigip-v12;command;bigip12::ShowZebOSconf;cat /config/ZebOS.conf
bigip-v12;command;bigip12::ShowZebOSsockets;lsof -i :179
#bigip-v12;command;bigip12::ShowRouteStatic;tmsh show /net route static
#bigip-v12;command;bigip12::WriteTerm;tmsh -q -c 'cd / \;list recursive'
bigip-v12;command;bigip12::WriteTerm;tmsh -q list
# Huawei support not provided out of the box, added after the fact.
# Let me know if you want me to post this, it's a bit OT for this thread..
huawei;script;rancid -t huawei
huawei;login;hulogin
huawei;module;huawei
huawei;inloop;huawei::inloop
huawei;command;huawei::DisplayVersion;display version
huawei;command;huawei::DisplayPatchInfo;display patch-information
huawei;command;huawei::DisplayDevice;display esn
huawei;command;huawei::DisplayDevice;display device
huawei;command;huawei::DisplayDevice;display device manufacture-info
huawei;command;huawei::DisplayDevice;display device pic-status
huawei;command;huawei::DisplayElabel;display device elabel
huawei;command;huawei::DisplayElabel;display elabel
huawei;command;huawei::DisplayTransceiver;display interface transceiver
huawei;command;huawei::DisplayLicense;display license
huawei;command;huawei::WriteTerm;display current-configuration
1) To make "tmsh -q -c 'cd / \;list recursive'" do something useful
the following change to rancid.pm<http://rancid.pm> is required, otherwise the split is
too aggressive.
*******
< my($type, $directive, $value, $value2) = split('\;');
---

#my($type, $directive, $value, $value2) = split('\;');
my($type, $directive, $value, $value2) = split('\;', $_, 4);

*******
2) Following changes made to bigip.pm<http://bigip.pm> (from rancid 3.5). There might
be a bug here still as uncommenting the ";tmsh -q -c 'cd / \;list
recursive'" line above seems to lead to unnecessary reattempts at
config collection - but only when debug is disabled for some reason.
*******
5c5
< ## rancid 3.5
---

## rancid 3.5 plus some customisations.

66c66,67
< use rancid 3.5;
---

#use rancid 3.5;
use rancid 3.2;

74c75
< $ENV{'TERM'} = "vt100";
---

$ENV{'TERM'} = "vt100-w";

183a185

s/^\ \ ([0-9]+)(\ +).*Air\ Outlet/ $1$2REMOVED Air Outlet/i;

184a187,188

s/^\ \ ([0-9]+)(\ +).*ADM1026/ $1$2REMOVED ADM1026/i;
s/^\ \ ([0-9]+)(\ +).*Main\ board(.*)$/ $1$2REMOVED Main board$3/i;

185a190

s/^\ \ ([0-9]+)(\ +)[0-9]+\ +[0-9]+\ +[0-9]+/ $1$2REMOVED REMOVED REMOVED/;

218a224,272

290a345,347

return (1) if (/Syntax Error: unexpected argument/);
return (0) if ($found_end); # Only run this routine once.

297c354
< $found_end++;
---

$found_end = 1;

*******
*******
$ diff bigip.pm<http://bigip.pm> bigip12.pm<http://bigip12.pm>
1c1
< package bigip;
---

package bigip12;

75c75
< $ENV{'TERM'} = "vt100-w";
---

$ENV{'TERM'} = "vt100";

*******
*******
$ diff f5rancid f5rancid-v9
64c64,65
< $ENV{'TERM'} = "vt100";
---

#$ENV{'TERM'} = "vt100";
$ENV{'TERM'} = "vt100-w";

186a188,191

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("SHOWBASE","","","# $1community <removed>\n");
next;
}

190a196,199

if (/^(\s*)password crypt \S+/) {
ProcessHistory("SHOWBASE","","","# $1password crypt <removed>\n");
next;
}

225c234,236
<
---

if (/^(.*)\.password = / && $filter_pwds >= 1) {
ProcessHistory("SHOWDB","","","# $1.password = <removed>\n") && next;
}

269a281,286

277c294
< if (/^(\s*)password / && $filter_pwds >= 1) {
---

if (/^(\s*)password (\w*)/ && $filter_pwds >= 1 && $2 ne "none") {

427a445,447

return (1) if (/BIGpipe:.*: syntax error/);
return (0) if ($found_end); # Only do this routine once

441a462,465

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("","","","# $1community <removed>\n") && next;
}

532a557

{'bigpipe list all' => 'WriteTerm'},

*******

Kind Regards,
Dan

--
Kind Regards,
Dan Kerse
+64 29 920 3745

This email message and its attachments are for the sole use of the intended recipient(s) and may not be shared with any other party. They may contain confidential information of 888 Holdings plc or its direct and indirect subsidiaries (together, the â888 Groupâ) and are to be regarded as confidential information under any non-disclosure agreement. Any review, use, disclosure or distribution by persons or entities other than the intended recipient(s) is prohibited. Nothing in this message is capable of or intended to create any legally binding obligation. The 888 Group will only ever assume a legally binding obligation where recorded in a written agreement duly executed by the authorized signatories of the relevant 888 Group company. The 888 Group accepts no liability for any personal views expressed in this message. If you are not the intended recipient, please contact the sender by return and destroy all copies of the original message and its attachments. Thank you

Shaun Krok

2016-09-13 10:14:37 UTC

Hi

So we have had success today on this issue !

As we knew the script was working when the BIG-IP cluster was in SYNC
We focused on the prompt:

1. We tried to shorten the TMSH prompt for the specific user --- this did not work âlist cli preference promptâ

cli preference {

prompt { host user status current-folder config-sync-status } >>> removed config-sync-status and current-folder

2. We then shorted the BASH prompt on both chassis in the cluster â everything is working 100% now even with changes-pending

Thanks to all

**** The way it is by default ********
if [ "$PS1" ]; then
if [ -z "$PROMPT_COMMAND" ]; then
case $TERM in
xterm*)
PS1='[$USER@$(getHostName):$(getPromptStatus)$(getAvcCount):$(getConfigSyncStatus)]\[\e]0;$(getHostName)\a\] \W \$ '
;;
screen)
PS1='[$USER@$(getHostName):$(getPromptStatus)$(getAvcCount):$(getConfigSyncStatus)] \W \$ '
;;
*)
PS1='[$USER@$(getHostName):$(getPromptStatus)$(getAvcCount):$(getConfigSyncStatus)] \W \$ '
;;
esac
fi

**** The way it is by now ********

if [ "$PS1" ]; then
if [ -z "$PROMPT_COMMAND" ]; then
case $TERM in
xterm*)
PS1='[$USER@$(getHostName)]:[$(getPromptStatus)]\[\e]0;$(getHostName)\a\] \W \$ '
;;
screen)
PS1='[$USER@$(getHostName)]:[$(getPromptStatus)]\[\e]0;$(getHostName)\a\] \W \$ '
;;
*)
PS1='[$USER@$(getHostName)]:[$(getPromptStatus)]\[\e]0;$(getHostName)\a\] \W \$ '
;;
esac
fi

Shaun Krok
Network Team

Herzliya Business Park
Herzliya Pituach 46140
Israel

Telephone: +972 (0)732889406
Mobile: +972 (0)50 2424381
email:***@888holdings.com

[Description: Description: Description: Description: 888]
[Description: 888]

From: Rancid-discuss [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of Shaun Krok
Sent: Tuesday, September 13, 2016 8:42 AM
To: Daniel Kerse <***@gmail.com>; heasley <***@shrubbery.net>
Cc: Nati Danan <***@888holdings.com>; rancid-***@shrubbery.net
Subject: Re: [rancid] F5 Rancid | BIGIP 12.0 | missed cmd(s)

Thanks Daniel

We have been using TERM=vt100-w since moving into 11.4 and greater
In 11.6 rancid spits out a diff error :

+ #Warning, can't fully initialize terminal, TERM is set to "vt100-w", status (0) - But it still works

Since moving to BIGIP 12.0 same error above but now missed cmd(s)

I have shortened the prompt on the F5 because when the cluster is in SYNC we have no missed cmd(s) and a diff is created running on version 12.0
Waiting for the next cron run and will advise what happenes !

Old:
cli preference {
prompt { host user status current-folder config-sync-status }
New:
cli preference {
alias-path { /Common }
prompt { host user status config-sync-status }

From: Rancid-discuss [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of Daniel Kerse
Sent: Tuesday, September 13, 2016 1:48 AM
To: heasley <***@shrubbery.net<mailto:***@shrubbery.net>>
Cc: rancid-***@shrubbery.net<mailto:rancid-***@shrubbery.net>; Nati Danan <***@888holdings.com<mailto:***@888holdings.com>>
Subject: Re: [rancid] F5 Rancid | BIGIP 12.0 | missed cmd(s)

I believe the "cd / ; list recursive" might required on 11.x (and presumably later) with partitions.
http://www.shrubbery.net/pipermail/rancid-discuss/2014-August/007790.html
If I ever get this to fallback gracefully to "list" I'll let you know.

that implies that the type is unknown or there is no pty.

I tend to agree. Maybe this type is unknown to more recent versions of BIG-IP despite working in older releases.

#my($type, $directive, $value, $value2) = split('\;');
my($type, $directive, $value, $value2) = split('\;', $_, 4);

## rancid 3.5 plus some customisations.

66c66,67
< use rancid 3.5;
---

#use rancid 3.5;
use rancid 3.2;

74c75
< $ENV{'TERM'} = "vt100";
---

$ENV{'TERM'} = "vt100-w";

183a185

s/^\ \ ([0-9]+)(\ +).*Air\ Outlet/ $1$2REMOVED Air Outlet/i;

184a187,188

s/^\ \ ([0-9]+)(\ +).*ADM1026/ $1$2REMOVED ADM1026/i;
s/^\ \ ([0-9]+)(\ +).*Main\ board(.*)$/ $1$2REMOVED Main board$3/i;

185a190

s/^\ \ ([0-9]+)(\ +)[0-9]+\ +[0-9]+\ +[0-9]+/ $1$2REMOVED REMOVED REMOVED/;

218a224,272

290a345,347

return (1) if (/Syntax Error: unexpected argument/);
return (0) if ($found_end); # Only run this routine once.

297c354
< $found_end++;
---

$found_end = 1;

*******
*******
$ diff bigip.pm<http://bigip.pm> bigip12.pm<http://bigip12.pm>
1c1
< package bigip;
---

package bigip12;

75c75
< $ENV{'TERM'} = "vt100-w";
---

$ENV{'TERM'} = "vt100";

*******
*******
$ diff f5rancid f5rancid-v9
64c64,65
< $ENV{'TERM'} = "vt100";
---

#$ENV{'TERM'} = "vt100";
$ENV{'TERM'} = "vt100-w";

186a188,191

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("SHOWBASE","","","# $1community <removed>\n");
next;
}

190a196,199

if (/^(\s*)password crypt \S+/) {
ProcessHistory("SHOWBASE","","","# $1password crypt <removed>\n");
next;
}

225c234,236
<
---

if (/^(.*)\.password = / && $filter_pwds >= 1) {
ProcessHistory("SHOWDB","","","# $1.password = <removed>\n") && next;
}

269a281,286

277c294
< if (/^(\s*)password / && $filter_pwds >= 1) {
---

if (/^(\s*)password (\w*)/ && $filter_pwds >= 1 && $2 ne "none") {

427a445,447

return (1) if (/BIGpipe:.*: syntax error/);
return (0) if ($found_end); # Only do this routine once

441a462,465

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("","","","# $1community <removed>\n") && next;
}

532a557

{'bigpipe list all' => 'WriteTerm'},

*******

Kind Regards,
Dan
--
Kind Regards,
Dan Kerse
+64 29 920 3745

This email message and its attachments are for the sole use of the intended recipient(s) and may not be shared with any other party. They may contain confidential information of 888 Holdings plc or its direct and indirect subsidiaries (together, the â888 Groupâ) and are to be regarded as confidential information under any non-disclosure agreement. Any review, use, disclosure or distribution by persons or entities other than the intended recipient(s) is prohibited. Nothing in this message is capable of or intended to create any legally binding obligation. The 888 Group will only ever assume a legally binding obligation where recorded in a written agreement duly executed by the authorized signatories of the relevant 888 Group company. The 888 Group accepts no liability for any personal views expressed in this message. If you are not the intended recipient, please contact the sender by return and destroy all copies of the original message and its attachments. Thank you

This email message and its attachments are for the sole use of the intended recipient(s) and may not be shared with any other party. They may contain confidential information of 888 Holdings plc or its direct and indirect subsidiaries (together, the â888 Groupâ) and are to be regarded as confidential information under any non-disclosure agreement. Any review, use, disclosure or distribution by persons or entities other than the intended recipient(s) is prohibited. Nothing in this message is capable of or intended to create any legally binding obligation. The 888 Group will only ever assume a legally binding obligation where recorded in a written agreement duly executed by the authorized signatories of the relevant 888 Group company. The 888 Group accepts no liability for any personal views expressed in this message. If you are not the intended recipient, please contact the sender by return and destroy all copies of the original message and its attachments. Thank you

Ambroise

2016-09-15 20:17:05 UTC

Hi,

I have the same issue in our company.
The solution is to add the command "stty cols 255" before to send any
command in expect script (the command should be passed in bash mode).

Ambroise

Post by Shaun Krok
Hi
So we have had success today on this issue !
As we knew the script was working when the BIG-IP cluster was in SYNC
1. We tried to shorten the TMSH prompt for the specific user ---
this did not work list cli preference prompt
cli preference {
prompt { host user status current-folder config-sync-status } >>>
removed config-sync-status and current-folder
2. We then shorted the BASH prompt on both chassis in the cluster
everything is working 100% now even with changes-pending
Thanks to all
**** The way it is by default ********
if [ "$PS1" ]; then
if [ -z "$PROMPT_COMMAND" ]; then
case $TERM in
xterm*)
\W \$ '
;;
screen)
;;
*)
;;
esac
fi
**** The way it is by now ********
if [ "$PS1" ]; then
if [ -z "$PROMPT_COMMAND" ]; then
case $TERM in
xterm*)
;;
screen)
;;
*)
;;
esac
fi
*Shaun Krok*
Network Team
Herzliya Business Park
Herzliya Pituach 46140
Israel
Telephone: +972 (0)732889406
Mobile: +972 (0)50 2424381
* *
*Description: Description: Description: Description: 888*
Description: 888
Behalf Of *Shaun Krok
*Sent:* Tuesday, September 13, 2016 8:42 AM
*Subject:* Re: [rancid] F5 Rancid | BIGIP 12.0 | missed cmd(s)
Thanks Daniel
We have been using TERM=vt100-w since moving into 11.4 and greater
+ #Warning, can't fully initialize terminal, TERM is set to "vt100-w",
status (0) - But it still works
Since moving to BIGIP 12.0 same error above but now missed cmd(s)
I have shortened the prompt on the F5 because when the cluster is in
SYNC we have no missed cmd(s) and a diff is created running on version 12.0
Waiting for the next cron run and will advise what happenes !
cli preference {
prompt { host user status current-folder config-sync-status }
cli preference {
alias-path { /Common }
prompt { host user status config-sync-status }
Behalf Of *Daniel Kerse
*Sent:* Tuesday, September 13, 2016 1:48 AM
*Subject:* Re: [rancid] F5 Rancid | BIGIP 12.0 | missed cmd(s)

Hello Shaun et.al <http://et.al>.,
Are you playing with the TERM environment variable at all?
1. the logs?
2. the resulting .raw files?
I'm also running version 3.2 (the latest available under EPEL) with
some F5s of various vintage.
- circa BIG-IP v9.x -> v10.x the tmsh shell was introduced in addition
to the pre-existing bigpipe.
- circa BIG-IP v10.x -> v11.x the old bigpipe shell was removed.
- (At least for BIG-IP v11.x and earlier) you must set TERM=vt100-w in
order to run some commands. This problem would only become apparent

Do you mean that it required that specific term type, or just that it
required that term be set to something other than dumb or network?
That specific terminal type is required in some cases. I think if the
prompt plus the command gets longer than something like 80 characters
you start getting control characters showing up in the middle of that
line which causes a regex match to fail.
http://www.shrubbery.net/pipermail/rancid-discuss/2014-July/007767.html
http://www.shrubbery.net/pipermail/rancid-discuss/2015-February/008038.html
http://www.shrubbery.net/pipermail/rancid-discuss/2015-July/008603.html

that implies that the type is unknown or there is no pty.
I tend to agree. Maybe this type is unknown to more recent versions of
BIG-IP despite working in older releases.

static

# This one seems to get confused
#bigip-v10-11;command;bigip::WriteTerm;tmsh -q -c 'cd / \;list

recursive'

bigip-v10-11;command;bigip::WriteTerm;tmsh -q list
# F5s running 12.0.0 seem to lack support for vt100-w terminal type.
# "Warning, can't fully initialize terminal, TERM is set to "vt100-w",
status (0)"
# As a result ShowSslCrt, ShowSslKey and ShowRouteStatic don't work.
# For some reason turning off debug flag causes problems for "tmsh -q
-c 'cd / \;list recursive'" I don't know why.
bigip-v12;script;rancid -dt bigip-v12
bigip-v12;login;clogin
bigip-v12;module;bigip12
bigip-v12;inloop;bigip12::inloop
bigip-v12;command;bigip12::ShowVersion;tmsh show /sys version
bigip-v12;command;bigip12::ShowHardware;tmsh show /sys hardware
bigip-v12;command;bigip12::ShowLicense;tmsh show /sys license
#bigip-v12;command;bigip12::ShowSslCrt;ls --full-time --color=never
/config/ssl/ssl.crt
#bigip-v12;command;bigip12::ShowSslKey;ls --full-time --color=never
/config/ssl/ssl.key
bigip-v12;command;bigip12::ShowZebOSconf;cat /config/ZebOS.conf
bigip-v12;command;bigip12::ShowZebOSsockets;lsof -i :179
#bigip-v12;command;bigip12::ShowRouteStatic;tmsh show /net route

static

#bigip-v12;command;bigip12::WriteTerm;tmsh -q -c 'cd / \;list

recursive'

bigip-v12;command;bigip12::WriteTerm;tmsh -q list
# Huawei support not provided out of the box, added after the fact.
# Let me know if you want me to post this, it's a bit OT for this

thread..

huawei;script;rancid -t huawei
huawei;login;hulogin
huawei;module;huawei
huawei;inloop;huawei::inloop
huawei;command;huawei::DisplayVersion;display version
huawei;command;huawei::DisplayPatchInfo;display patch-information
huawei;command;huawei::DisplayDevice;display esn
huawei;command;huawei::DisplayDevice;display device
huawei;command;huawei::DisplayDevice;display device manufacture-info
huawei;command;huawei::DisplayDevice;display device pic-status
huawei;command;huawei::DisplayElabel;display device elabel
huawei;command;huawei::DisplayElabel;display elabel
huawei;command;huawei::DisplayTransceiver;display interface

transceiver

huawei;command;huawei::DisplayLicense;display license
huawei;command;huawei::WriteTerm;display current-configuration
1) To make "tmsh -q -c 'cd / \;list recursive'" do something useful
the following change to rancid.pm <http://rancid.pm> is required,

otherwise the split is

too aggressive.
*******
< my($type, $directive, $value, $value2) = split('\;');
---

#my($type, $directive, $value, $value2) = split('\;');
my($type, $directive, $value, $value2) = split('\;',

$_, 4);

*******
2) Following changes made to bigip.pm <http://bigip.pm> (from

rancid 3.5). There might

be a bug here still as uncommenting the ";tmsh -q -c 'cd / \;list
recursive'" line above seems to lead to unnecessary reattempts at
config collection - but only when debug is disabled for some reason.
*******
5c5
< ## rancid 3.5
---

## rancid 3.5 plus some customisations.

66c66,67
< use rancid 3.5;
---

#use rancid 3.5;
use rancid 3.2;

74c75
< $ENV{'TERM'} = "vt100";
---

$ENV{'TERM'} = "vt100-w";

183a185

s/^\ \ ([0-9]+)(\ +).*Air\ Outlet/ $1$2REMOVED Air

Outlet/i;

184a187,188

s/^\ \ ([0-9]+)(\ +).*ADM1026/ $1$2REMOVED ADM1026/i;
s/^\ \ ([0-9]+)(\ +).*Main\ board(.*)$/ $1$2REMOVED

Main board$3/i;

185a190

s/^\ \ ([0-9]+)(\ +)[0-9]+\ +[0-9]+\ +[0-9]+/

$1$2REMOVED REMOVED REMOVED/;

218a224,272

# This routine parses "ls --full-time --color=never

/config/ssl/ssl.crt"

ProcessHistory("ShowSslCrt","","","#\n#/config/ssl/ssl.crt:\n");

}
ProcessHistory("ShowSslCrt","","","# $_") && next;
}
return(0);
}
# This routine parses "ls --full-time --color=never

/config/ssl/ssl.key"

ProcessHistory("ShowSslKey","","","#\n#/config/ssl/ssl.key:\n");

}
ProcessHistory("ShowSslKey","","","# $_") && next;
}
return(0);
}

290a345,347

return (1) if (/Syntax Error: unexpected argument/);
return (0) if ($found_end); # Only run this routine once.

297c354
< $found_end++;
---

$found_end = 1;

*******
*******
$ diff bigip.pm <http://bigip.pm> bigip12.pm <http://bigip12.pm>
1c1
< package bigip;
---

package bigip12;

75c75
< $ENV{'TERM'} = "vt100-w";
---

$ENV{'TERM'} = "vt100";

*******
*******
$ diff f5rancid f5rancid-v9
64c64,65
< $ENV{'TERM'} = "vt100";
---

#$ENV{'TERM'} = "vt100";
$ENV{'TERM'} = "vt100-w";

186a188,191

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("SHOWBASE","","","# $1community

<removed>\n");

next;
}

190a196,199

if (/^(\s*)password crypt \S+/) {
ProcessHistory("SHOWBASE","","","# $1password crypt

<removed>\n");

next;
}

225c234,236
<
---

if (/^(.*)\.password = / && $filter_pwds >= 1) {
ProcessHistory("SHOWDB","","","# $1.password =

<removed>\n") && next;

}

269a281,286

if (/^(\s*)monitor state (up|down)$/) {
ProcessHistory("SHOWDB","","","# $1monitor state

<removed>\n") && next;

}
if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("SHOWDB","","","# $1community

<removed>\n") && next;

}

277c294
< if (/^(\s*)password / && $filter_pwds >= 1) {
---

if (/^(\s*)password (\w*)/ && $filter_pwds >= 1 && $2 ne

"none") {

427a445,447

return (1) if (/BIGpipe:.*: syntax error/);
return (0) if ($found_end); # Only do this

routine once

441a462,465

if (/^(\s*)community \S+/ && $filter_commstr) {
ProcessHistory("","","","# $1community <removed>\n")

&& next;

}

532a557

{'bigpipe list all' => 'WriteTerm'},

*******

Kind Regards,
Dan
--
Kind Regards,
Dan Kerse
+64 29 920 3745
This email message and its attachments are for the sole use of the
intended recipient(s) and may not be shared with any other party. They
may contain confidential information of 888 Holdings plc or its direct
and indirect subsidiaries (together, the 888 Group) and are to be
regarded as confidential information under any non-disclosure agreement.
Any review, use, disclosure or distribution by persons or entities other
than the intended recipient(s) is prohibited. Nothing in this message is
capable of or intended to create any legally binding obligation. The 888
Group will only ever assume a legally binding obligation where recorded
in a written agreement duly executed by the authorized signatories of
the relevant 888 Group company. The 888 Group accepts no liability for
any personal views expressed in this message. If you are not the
intended recipient, please contact the sender by return and destroy all
copies of the original message and its attachments. Thank you
This email message and its attachments are for the sole use of the
intended recipient(s) and may not be shared with any other party. They
may contain confidential information of 888 Holdings plc or its direct
and indirect subsidiaries (together, the 888 Group) and are to be
regarded as confidential information under any non-disclosure agreement.
Any review, use, disclosure or distribution by persons or entities other
than the intended recipient(s) is prohibited. Nothing in this message is
capable of or intended to create any legally binding obligation. The 888
Group will only ever assume a legally binding obligation where recorded
in a written agreement duly executed by the authorized signatories of
the relevant 888 Group company. The 888 Group accepts no liability for
any personal views expressed in this message. If you are not the
intended recipient, please contact the sender by return and destroy all
copies of the original message and its attachments. Thank you
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

heasley

2016-09-13 21:57:23 UTC

Do you mean that it required that specific term type, or just that it
required that term be set to something other than dumb or network?

That specific terminal type is required in some cases. I think if the
prompt plus the command gets longer than something like 80 characters you
start getting control characters showing up in the middle of that line
which causes a regex match to fail.
http://www.shrubbery.net/pipermail/rancid-discuss/2014-July/007767.html
http://www.shrubbery.net/pipermail/rancid-discuss/2015-February/008038.html
http://www.shrubbery.net/pipermail/rancid-discuss/2015-July/008603.html

is there a terminal type that omits the screen manipulation junk (esc codes)
that is recognized by the f5? eg: dumb, network, ... and set the terminal
width to 132.

Shaun Krok

2016-09-13 05:36:30 UTC