[rancid] How to ignore certain output?

[Matt Almgren]

I've seen this question pass through this list a couple times, but I wasn't paying attention.

I have this output from our f5 that rancid sees and is checking in the changes. The monitor state up and down is usually alternating (QA environment), so I'd like to ignore that line:

Index: configs/sca-qa-lb001
===================================================================
- -- configs/sca-qa-lb001 (revision 7756)
@@ -398,13 +398,13 @@
# }
# 10.102.72.122:4241 {
# node type pool member t
- # monitor state down
+ # monitor state up
# enable
# inband inst disable
# }
# 10.102.72.122:4243 {
# node type pool member t
- # monitor state down
+ # monitor state up
# enable
# inband inst disable
# }
Index: configs/sca-qa-lb002
===================================================================

How and where do I configure rancid to ignore the lines that have "monitor state" in them?

Thanks, Matt


This message is being sent by Skyfire Labs, Inc. It is intended exclusively for the individuals and entities to which it is addressed. This communication, including any attachments, may contain information that is proprietary, privileged, confidential, or otherwise subject to restrictions on disclosure pursuant to applicable law. If you are not the named addressee, you are not authorized to read, print, retain copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by email and delete all copies of this message. This message is protected by applicable legal privileges and is confidential.

[Matt Almgren]

I should add, that I did modify f5rancid a while back based on a suggestion from this list, but that doesn't seem to be working:

sub WriteTermTMSH {
my($lines) = 0;
print STDERR " In WriteTerm: $_" if ($debug);

while (<INPUT>) {
tr/\015//d;
next if (/^\s*$/);

s/state down$/state up/i; # Ignore monitor down state, save the config as up.

# end of config - hopefully. f5 does not have a reliable end-of-config
# tag.
if (/^$prompt/) {
$found_end++;
last;
}
return(-1) if (/command authorization failed/i);

$lines++;

if (/(bind-pw|encrypted-password|user-password-encrypted|passphrase) / && $filter_pwds >= 1) {
ProcessHistory("ENABLE","","","# $1 <removed>\n");
next;
}

# catch anything that wasnt matched above.
ProcessHistory("","","","$_");
}

if ($lines < 3) {
printf(STDERR "ERROR: $host configuration appears truncated.\n");
$found_end = 0;
return(-1);
}

return(0);
}




________________________________________
From: Rancid-discuss [rancid-discuss-***@shrubbery.net] On Behalf Of Matt Almgren [***@skyfire.com]
Sent: Tuesday, March 03, 2015 8:33 PM
To: rancid-***@shrubbery.net
Subject: [rancid] How to ignore certain output?

I've seen this question pass through this list a couple times, but I wasn't paying attention.

I have this output from our f5 that rancid sees and is checking in the changes. The monitor state up and down is usually alternating (QA environment), so I'd like to ignore that line:

Index: configs/sca-qa-lb001
===================================================================
- -- configs/sca-qa-lb001 (revision 7756)
@@ -398,13 +398,13 @@
# }
# 10.102.72.122:4241 {
# node type pool member t
- # monitor state down
+ # monitor state up
# enable
# inband inst disable
# }
# 10.102.72.122:4243 {
# node type pool member t
- # monitor state down
+ # monitor state up
# enable
# inband inst disable
# }
Index: configs/sca-qa-lb002
===================================================================

How and where do I configure rancid to ignore the lines that have "monitor state" in them?

Thanks, Matt


This message is being sent by Skyfire Labs, Inc. It is intended exclusively for the individuals and entities to which it is addressed. This communication, including any attachments, may contain information that is proprietary, privileged, confidential, or otherwise subject to restrictions on disclosure pursuant to applicable law. If you are not the named addressee, you are not authorized to read, print, retain copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by email and delete all copies of this message. This message is protected by applicable legal privileges and is confidential.
_______________________________________________
Rancid-discuss mailing list
Rancid-***@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

This message is being sent by Skyfire Labs, Inc. It is intended exclusively for the individuals and entities to which it is addressed. This communication, including any attachments, may contain information that is proprietary, privileged, confidential, or otherwise subject to restrictions on disclosure pursuant to applicable law. If you are not the named addressee, you are not authorized to read, print, retain copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by email and delete all copies of this message. This message is protected by applicable legal privileges and is confidential.

[heasley]

Adjust for your script, but this should do it.

Index: bin/f5rancid.in
===================================================================
--- bin/f5rancid.in (revision 3038)
+++ bin/f5rancid.in (working copy)
@@ -502,6 +502,9 @@
/^ip nat (\S+) source static (\S+)/ &&
ProcessHistory("IP NAT $1","ipsort","$2","$_") && next;

+ # monitor state
+ /^\s+monitor state (\S+)/ && next;
+
# catch anything that wasnt matched above.
ProcessHistory("","","","$_");
}

[Alan McKinnon]

On Tue, 3 Mar 2015 20:33:07 -0800
Matt Almgren <***@skyfire.com> wrote:

> I've seen this question pass through this list a couple times, but I
> wasn't paying attention.
>
> I have this output from our f5 that rancid sees and is checking in
> the changes. The monitor state up and down is usually alternating
> (QA environment), so I'd like to ignore that line:
>
> Index: configs/sca-qa-lb001
> ===================================================================
> - -- configs/sca-qa-lb001 (revision 7756)
> @@ -398,13 +398,13 @@
> # }
> # 10.102.72.122:4241 {
> # node type pool member t
> - # monitor state down
> + # monitor state up
> # enable
> # inband inst disable
> # }
> # 10.102.72.122:4243 {
> # node type pool member t
> - # monitor state down
> + # monitor state up
> # enable
> # inband inst disable
> # }
> Index: configs/sca-qa-lb002
> ===================================================================
>
> How and where do I configure rancid to ignore the lines that have
> "monitor state" in them?

heasley gave the exact patch you need lower donw in the thread, but
I'll explain the general process, you need some basic understanding of
perl. This is for 2.3, I haven't gotten around to using 3.x yet, I
beleive the latest version is quite similar in this respect:

You have to edit the script that does the actual parsing of the output
from the device, and tell it there what to ignore.

In the case of Cisco IOS devices, that script is "rancid", for your
F5's it will be "f5rancid". In any event, look in rancid_fe to find
which script is called for which device type in router.db

Inside that script, you will usually find an array
"@command_table" [1], this maps commands run on the device to
subroutines in the script that deal with that command.

Now edit that subroutine to throw away the lines you are not interested
in. Taking heasley's example:

/^\s+monitor state (\S+)/ && next;

which means "if this line matches the regex, then skip it and go onto
the next one".

If you need something slightly more complex (eg the stuff you want to
ignore is the first line plus a bunch more), then you must write
slightly more complex perl. But it's all very easy and obvious and the
code has hundreds of examples you can follow.


[1] not all parser scripts do it this way, the author is free to wite
their code any way they feel like. "rancid" is the original and first,
so most new parsers follow the same template. If you are using a parser
that doesn't, then you must read the code and figure it out.


--
alan dot mckinnon at gmail dot com

[Matt Almgren]

Thanks Alan and Heasley!

-- Matt

________________________________________
From: Rancid-discuss [rancid-discuss-***@shrubbery.net] On Behalf Of Alan McKinnon [***@gmail.com]
Sent: Wednesday, March 04, 2015 5:52 AM
To: rancid-***@shrubbery.net
Subject: Re: [rancid] How to ignore certain output?

On Tue, 3 Mar 2015 20:33:07 -0800
Matt Almgren <***@skyfire.com> wrote:

> I've seen this question pass through this list a couple times, but I
> wasn't paying attention.
>
> I have this output from our f5 that rancid sees and is checking in
> the changes. The monitor state up and down is usually alternating
> (QA environment), so I'd like to ignore that line:
>
> Index: configs/sca-qa-lb001
> ===================================================================
> - -- configs/sca-qa-lb001 (revision 7756)
> @@ -398,13 +398,13 @@
> # }
> # 10.102.72.122:4241 {
> # node type pool member t
> - # monitor state down
> + # monitor state up
> # enable
> # inband inst disable
> # }
> # 10.102.72.122:4243 {
> # node type pool member t
> - # monitor state down
> + # monitor state up
> # enable
> # inband inst disable
> # }
> Index: configs/sca-qa-lb002
> ===================================================================
>
> How and where do I configure rancid to ignore the lines that have
> "monitor state" in them?

heasley gave the exact patch you need lower donw in the thread, but
I'll explain the general process, you need some basic understanding of
perl. This is for 2.3, I haven't gotten around to using 3.x yet, I
beleive the latest version is quite similar in this respect:

You have to edit the script that does the actual parsing of the output
from the device, and tell it there what to ignore.

In the case of Cisco IOS devices, that script is "rancid", for your
F5's it will be "f5rancid". In any event, look in rancid_fe to find
which script is called for which device type in router.db

Inside that script, you will usually find an array
"@command_table" [1], this maps commands run on the device to
subroutines in the script that deal with that command.

Now edit that subroutine to throw away the lines you are not interested
in. Taking heasley's example:

/^\s+monitor state (\S+)/ && next;

which means "if this line matches the regex, then skip it and go onto
the next one".

If you need something slightly more complex (eg the stuff you want to
ignore is the first line plus a bunch more), then you must write
slightly more complex perl. But it's all very easy and obvious and the
code has hundreds of examples you can follow.


[1] not all parser scripts do it this way, the author is free to wite
their code any way they feel like. "rancid" is the original and first,
so most new parsers follow the same template. If you are using a parser
that doesn't, then you must read the code and figure it out.


--
alan dot mckinnon at gmail dot com
_______________________________________________
Rancid-discuss mailing list
Rancid-***@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

This message is being sent by Skyfire Labs, Inc. It is intended exclusively for the individuals and entities to which it is addressed. This communication, including any attachments, may contain information that is proprietary, privileged, confidential, or otherwise subject to restrictions on disclosure pursuant to applicable law. If you are not the named addressee, you are not authorized to read, print, retain copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by email and delete all copies of this message. This message is protected by applicable legal privileges and is confidential.

[heasley]

Wed, Mar 04, 2015 at 03:52:12PM +0200, Alan McKinnon:
> heasley gave the exact patch you need lower donw in the thread, but
> I'll explain the general process, you need some basic understanding of
> perl. This is for 2.3, I haven't gotten around to using 3.x yet, I
> beleive the latest version is quite similar in this respect:

I encourage upgrading to 3.x, and i'll consider 3.2 production after I
attend to 3 outstanding issues/requests. 3.x makes it easier to make
localizations such as this and still be easy to make future upgrades.

you would copy the definition for a device type to a new name, create a
new module to handle your localization, and configure your new device
type to use it. eg rancid.types.conf:

crisco;script;rancid -t crisco
crisco;login;clogin
crisco;module;ios
crisco;module;ios_local
crisco;inloop;ios::inloop
crisco;command;ios::ShowVersion;show version
# skipit crisco;command;ios::ShowRedundancy;show redundancy secondary
crisco;command;ios_local::ShowIDprom;show idprom backplane
crisco;command;ios::ShowInstallActive;show install active
....

i believe the type configs could be more featureful and i've left some
comment about that in the manpage, but its handy for this need.