[rancid] Strange Jerancid/Rancid behavior

[Mack, David A (Dave)]

All,
Hello! I am in the process of bringing up a new Rancid server and have run into something with at least the jerancid script for Juniper ERXs. What I am seeing is related to password removal. I have filter passwords globally set to "no". However the ERXs have certain passwords that have their hashes calculated on the fly. So each time they are display the values shown will change. This means you get a new CVS every run. Jerancid has routines to handle two of these conditions for FTP and MPLS LDP passwords:

# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}

I am comparing the collected configuration files between my old Rancid install and my new one and here is what I am seeing:

Old Rancid:

!host "FTP1" 172.15.1.1 ftp <removed>
!host "FTP2" 172.15.1.2 ftp <removed>

New Rancid:

! <removed>
! <removed>

I looked at the code for jerancid and I could not find any significant changes that would cause this. When I jerancid from the command line with the debug flag I got:

csrest01(opseng)$ jerancid -d -l erx-ser-01
executing clogin -t 90 -c"show version;show redundancy;show boot;show environment all;dir;show hardware;show configuration" ERX-ser-01
executing clogin -t 90 -c"show version;show redundancy;show boot;show environment all;dir;show hardware;show configuration" ERX-ser-01
PROMPT MATCH: ERX-SER-01#
HIT COMMAND:ERX-SER-01#show version
In ShowVersion: ERX-SER-01#show version
HIT COMMAND:ERX-SER-01#show redundancy
In ShowRedundancy: ERX-SER-01#show redundancy
HIT COMMAND:ERX-SER-01#show boot
In ShowBoot: ERX-SER-01#show boot
HIT COMMAND:ERX-SER-01#show environment all
In ShowEnv: ERX-SER-01#show environment all
HIT COMMAND:ERX-SER-01#dir
In DirSlotN: ERX-SER-01#dir
HIT COMMAND:ERX-SER-01#show hardware
In ShowHardware: ERX-SER-01#show hardware
HIT COMMAND:ERX-SER-01#show configuration
In WriteTerm: ERX-SER-01#show configuration
Done : ERX-SER-01#exit


When I look at the .new file created, it does have full commands:

!host "FTP1" 172.15.1.1 ftp <removed>
!host "FTP2" 172.15.1.2 ftp <removed>

What gives? Any ideas how to troubleshoot and resolve this?

My setup:
Rancid 2.3.6
CVS-1.11.23
Perl 5.8.8
Solaris 10 x86

Thanks!
Dave

[john heasley]

Post by Mack, David A (Dave)
All,
# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
!host "FTP1" 172.15.1.1 ftp <removed>
!host "FTP2" 172.15.1.2 ftp <removed>
! <removed>
! <removed>

Does this fix it?

Index: jerancid.in
===================================================================
--- jerancid.in (revision 2320)
+++ jerancid.in (working copy)
@@ -444,14 +444,16 @@

# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}

[Mack, David A (Dave)]

John,
That fixed it. Can you tell me what the issue was?

Thanks!
Dave


___________________________________________________________________________
David A. Mack [phone] (W)
CCIE #6963 (Sec, SP OPS, SP and R&S) JNCIE-M #399 CISSP [phone] (C)
email: ***@verizon.com
___________________________________________________________________________
"We are now the knights who say... Ping!"

-----Original Message-----
From: john heasley [mailto:***@shrubbery.net]
Sent: Monday, August 22, 2011 6:32 PM
To: Mack, David A (Dave)
Cc: rancid-***@shrubbery.net
Subject: Re: [rancid] Strange Jerancid/Rancid behavior

Post by Mack, David A (Dave)
All,
# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
!host "FTP1" 172.15.1.1 ftp <removed>
!host "FTP2" 172.15.1.2 ftp <removed>
! <removed>
! <removed>

Does this fix it?

Index: jerancid.in
===================================================================
--- jerancid.in (revision 2320)
+++ jerancid.in (working copy)
@@ -444,14 +444,16 @@

# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}

[Mack, David A (Dave)]

Oh and btw the bgp neighbor passwords are also fluctuating. So I am going to use the same logic on them as well. You might want to include that with future updates.

Thanks!
Dave


-----Original Message-----
From: john heasley [mailto:***@shrubbery.net]
Sent: Monday, August 22, 2011 6:32 PM
To: Mack, David A (Dave)
Cc: rancid-***@shrubbery.net
Subject: Re: [rancid] Strange Jerancid/Rancid behavior

Post by Mack, David A (Dave)
All,
# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
!host "FTP1" 172.15.1.1 ftp <removed>
!host "FTP2" 172.15.1.2 ftp <removed>
! <removed>
! <removed>

Does this fix it?

Index: jerancid.in
===================================================================
--- jerancid.in (revision 2320)
+++ jerancid.in (working copy)
@@ -444,14 +444,16 @@

# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}

[Yudayev, Konstantin]

John,

Is there any way to make rancid ignore "fluctuating" password lines during diff rather than removing them from configs? Configs with removed passwords are not suitable for restore for example.

----
Konstantin Yudayev

Verizon - IP Systems Operations & Support
Office: [phone]
22001 Loudoun County Parkway, Ashburn, VA 20147
Email: ***@verizon.com



-----Original Message-----
From: rancid-discuss-***@shrubbery.net [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of john heasley
Sent: Monday, August 22, 2011 18:32
To: Mack, David A (Dave)
Cc: rancid-***@shrubbery.net
Subject: Re: [rancid] Strange Jerancid/Rancid behavior

Post by Mack, David A (Dave)
All,
# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
!host "FTP1" 172.15.1.1 ftp <removed>
!host "FTP2" 172.15.1.2 ftp <removed>
! <removed>
! <removed>

Does this fix it?

Index: jerancid.in
===================================================================
--- jerancid.in (revision 2320)
+++ jerancid.in (working copy)
@@ -444,14 +444,16 @@

# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}

[Mack, David A (Dave)]

All,
I agree with Konstantin. Would it be possible to pass a REGEX to diff so that it will ignore the offending password lines. Perhaps using the command line option below:

-I RE --ignore-matching-lines=RE Ignore changes whose lines all match RE.

Thanks!
Dave


-----Original Message-----
From: Yudayev, Konstantin
Sent: Tuesday, August 23, 2011 9:16 AM
To: john heasley; Mack, David A (Dave)
Cc: rancid-***@shrubbery.net; Yudayev, Konstantin
Subject: RE: [rancid] Strange Jerancid/Rancid behavior

John,

Is there any way to make rancid ignore "fluctuating" password lines during diff rather than removing them from configs? Configs with removed passwords are not suitable for restore for example.

----
Konstantin Yudayev

Verizon - IP Systems Operations & Support
Office: [phone]
22001 Loudoun County Parkway, Ashburn, VA 20147
Email: ***@verizon.com



-----Original Message-----
From: rancid-discuss-***@shrubbery.net [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of john heasley
Sent: Monday, August 22, 2011 18:32
To: Mack, David A (Dave)
Cc: rancid-***@shrubbery.net
Subject: Re: [rancid] Strange Jerancid/Rancid behavior

Post by Mack, David A (Dave)
All,
# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
ProcessHistory("","","","!$1 <removed>\n"); next;
}
}
!host "FTP1" 172.15.1.1 ftp <removed>
!host "FTP2" 172.15.1.2 ftp <removed>
! <removed>
! <removed>

Does this fix it?

Index: jerancid.in
===================================================================
--- jerancid.in (revision 2320)
+++ jerancid.in (working copy)
@@ -444,14 +444,16 @@

# ftp host encrypted password oscillates
if (/^(host \S+ \S+ ftp) /) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^host \S+ \S+ ftp \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}
# mpls ldp encrypted password oscillates
if (/^(mpls ldp neighbor \S+ password)/) {
+ my($prefix) = $1;
if ($filter_pwds >= 1 || /^mpls ldp neighbor \S+ password \d /) {
- ProcessHistory("","","","!$1 <removed>\n"); next;
+ ProcessHistory("","","","!$prefix <removed>\n"); next;
}
}