[rancid] Reverse RANCID

Discussion:

James Bensley

2015-02-11 12:02:29 UTC

Hi All,

I am think about writing a web interface that uses RANCID in the
background to make configuration changes on devices. Since RANCID has
a bunch of scripts for various device types my thinking is a
simple-ish web interface in which I can paste in some config and then
use RANCID to log into the device and input the config, also though I
can specify some commands and RANCID will run though them and capture
output which can be passed to Bash/PERL/Python scripts to interogate
the output and check that the BGP sessions have come back up or that
the number of routes in a VRF is still the same etc.

The goal is: Anything I do on the CLI when making changes to devices
can be automated.

I know I can push config using the RANCID CLI wrapper scripts but I'm
wondering if anyone has done this before to extend RANCID to also run
"show" style commands and interogated the output to make checks to
valid the success of the change, and also if anyone has made a web
interface already (other than the CVS types for RANCID's normal
purpose of backing up rather than pushing config) ?

Kind regards,
James.

Alan McKinnon

2015-02-11 15:31:58 UTC

Permalink

Post by James Bensley
Hi All,
I am think about writing a web interface that uses RANCID in the
background to make configuration changes on devices. Since RANCID has
a bunch of scripts for various device types my thinking is a
simple-ish web interface in which I can paste in some config and then
use RANCID to log into the device and input the config, also though I
can specify some commands and RANCID will run though them and capture
output which can be passed to Bash/PERL/Python scripts to interogate
the output and check that the BGP sessions have come back up or that
the number of routes in a VRF is still the same etc.
The goal is: Anything I do on the CLI when making changes to devices
can be automated.
I know I can push config using the RANCID CLI wrapper scripts but I'm
wondering if anyone has done this before to extend RANCID to also run
"show" style commands and interogated the output to make checks to
valid the success of the change, and also if anyone has made a web
interface already (other than the CVS types for RANCID's normal
purpose of backing up rather than pushing config) ?

It doesn't make sense to extend rancid in this way.

Consider rancid's purpose: it logs in, captures the config, diffs it and
stores the result. Then tells you what the diff is.

None of that involves in any way changing the device in question and it
is highly recommended that you lock down the rancid user to only the
specific commands listed in @commands.

There is one part of rancid that enables you to do config changes
however: clogin

Rather do something like this:
Get the changes you want to make from the user, apply them using clogin
and then write a framework that will do the double-checking you
describe. Rancid itself has no code you can leverage to do any of that.
It's best done in an entirely separate system, with the added benefit
that rancid will come along in an hour and record the fact of a change made.

All this depends however on your Risk department being OK with the idea.
I know mine would shoot me at the very thought :-)

--
Alan McKinnon
***@gmail.com

Aaron Dudek

2015-02-11 15:49:33 UTC

Permalink

Isn't this kind of the function Tail-F was proposing?

Post by Alan McKinnon

It doesn't make sense to extend rancid in this way.
Consider rancid's purpose: it logs in, captures the config, diffs it and
stores the result. Then tells you what the diff is.
None of that involves in any way changing the device in question and it
is highly recommended that you lock down the rancid user to only the
There is one part of rancid that enables you to do config changes
however: clogin
Get the changes you want to make from the user, apply them using clogin
and then write a framework that will do the double-checking you
describe. Rancid itself has no code you can leverage to do any of that.
It's best done in an entirely separate system, with the added benefit
that rancid will come along in an hour and record the fact of a change made.
All this depends however on your Risk department being OK with the idea.
I know mine would shoot me at the very thought :-)
--
Alan McKinnon
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

heasley

2015-02-12 00:39:09 UTC

Permalink

Post by Aaron Dudek
Isn't this kind of the function Tail-F was proposing?

sort of. the best way to introduce yourself to tail-f is to search for
presentations by Carl Moberg; he does a fantastic job in various videos
from NANOG, etc.

Tony Tauber

2015-02-12 00:58:38 UTC

Permalink

You might also want to see a presentation on automation
<https://www.nanog.org/meetings/abstract?id=2477> from NANOG63 just last
week.
This discusses using the NETCONF notion of a candidate config to do the
syntax validation.
Unfortunately the author indicated that in his experience only JunOS had
good support for this so far.

Something to aim for perhaps.

Tony

Post by heasley

Post by Aaron Dudek
Isn't this kind of the function Tail-F was proposing?

sort of. the best way to introduce yourself to tail-f is to search for
presentations by Carl Moberg; he does a fantastic job in various videos
from NANOG, etc.
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

heasley

2015-02-12 05:17:07 UTC

Permalink

Post by Tony Tauber
You might also want to see a presentation on automation
<https://www.nanog.org/meetings/abstract?id=2477> from NANOG63 just last
week.
This discusses using the NETCONF notion of a candidate config to do the
syntax validation.

junos and XR VMs exist for this too and theyre cheap. this isnt the forum
for it, but if anyone figures-out how to make the bloody things work on
ESXi, please school me. rumor is that the current XR VM, of a few weeks
ago, works on ESXi.

Hagen, Skye (skyeh@uidaho.edu)

2015-02-11 16:54:12 UTC

Permalink

This post might be inappropriate. Click to display it.

Russell, David

2015-02-11 23:59:46 UTC

Permalink

That sounds very impressive and useful. Have you shared any of these
scripts?

Regards,

David

Post by Hagen, Skye (***@uidaho.edu)
I have been asked to do something similar where I work. The problem that I
ran into was the verification process for certain kinds of jobs. For a
simple change, that only affected the device itself, and if there was a
problem, wouldn't cause a major outage, I could hack together some scripts
to use clogin and do the job. But, when identical changes had to be made
to several devices in coordination, no way. The number of ways things
could go wrong, and the varieties of backout procedures, it just got too
complex. And for something as potentially disruptive as making changes to
a routing protocol, I always wanted to be hands on.
On the other side of RANCID, you have a repository that contains a near
real-time copy of your device configurations. I have written a number of
auditing scripts that will determine all routed networks, and compare them
against our network management system to make sure all routed networks are
defined. I also use this list of routed networks to audit ACL's, to make
sure that we clean up related ACL's when we delete networks. I audit the
VLAN's to make sure they are all contiguous across all our switches. I
also have a configuration auditing system that will compare a
configuration file against a set of rules, and check for compliance.
As I learned from an auditor, there are two ways to approach controlling
something. Control it up front, or audit after the fact. In my case,
auditing after the fact was a lot easier and quicker.
Skye.

Post by Alan McKinnon

It doesn't make sense to extend rancid in this way.
Consider rancid's purpose: it logs in, captures the config, diffs it and
stores the result. Then tells you what the diff is.
None of that involves in any way changing the device in question and it
is highly recommended that you lock down the rancid user to only the
There is one part of rancid that enables you to do config changes
however: clogin
Get the changes you want to make from the user, apply them using clogin
and then write a framework that will do the double-checking you
describe. Rancid itself has no code you can leverage to do any of that.
It's best done in an entirely separate system, with the added benefit
that rancid will come along in an hour and record the fact of a change made.
All this depends however on your Risk department being OK with the idea.
I know mine would shoot me at the very thought :-)
--
Alan McKinnon
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

--
David R. Russell
CCIE #5751

*Infrastructure Planning & Engineering*
*Dow Jones Technology*
P.O. Box 300 | Princeton NJ 08543-0300
Direct: 609-520-4458 | Cell: 610-909-1129

*Email: *****@dowjones.com <***@dowjones.com>*

Alan McKinnon

2015-02-12 06:53:36 UTC

Permalink

This post might be inappropriate. Click to display it.

heasley

2015-02-11 17:16:26 UTC

Permalink

not quite the same, but rancid comes with a version of Ed Kern's looking
glass adapted to use rancid. you may gain some ideas from that.

James Andrewartha

2015-02-12 08:13:56 UTC

Permalink

Post by James Bensley
I know I can push config using the RANCID CLI wrapper scripts but I'm
wondering if anyone has done this before to extend RANCID to also run
"show" style commands and interogated the output to make checks to
valid the success of the change, and also if anyone has made a web
interface already (other than the CVS types for RANCID's normal
purpose of backing up rather than pushing config) ?

Notch/Mr CLI/PUNC/netmunge might be up your alley:
https://code.google.com/p/notch/

--
James Andrewartha
Network & Projects Engineer
Christ Church Grammar School
Claremont, Western Australia
Ph. (08) 9442 1757
Mob. 0424 160 877

James Bensley

2015-02-12 10:08:24 UTC

Permalink

On 12 February 2015 at 08:13, James Andrewartha

Post by James Andrewartha

https://code.google.com/p/notch/

That is a rather cool tool, and that is the kind of think I am thinking off.

What I am imagining in my head is something like a list of actions to
be execute that you can define along with an IP/hostname to perform
the action on and some login credentials for that specific action.

An example would be selecting the action "perform a CLI check" and
them supplied a command to show the number of routes received by a
neighbour. Giving the IP of RouterA. Then add this to the action queue
as the first action.

Next I would add an action to the action queue "upload some config" to
RouterB and supply RouterB's IP address and username/password.

Finally a third action to "perform a CLI check" is added to the action
queue again to check on RouterA it is still receiving the same number
of routes from RouterB. Actions could have pass/fail criteria too, if
the value is lt/eq/gt etc its a pass or faill, and sub-action could be
defined.

Set a start time of midnight and go to bed.

That is again a slightly simplified view of what I am proposing but it
doesn't seem like rocket science to me, nor to build in safegaurds?

Kind regards,
James.

James Bensley

2015-02-12 09:57:55 UTC

Permalink

Hi All,

In answer to some of the feedback I've had I perhaps should have been
more detailed in my explenation as people are thowing back reason not
really related to the technically of do this which is what I was
trying to query for;

- When I said I'd like to use RANCID, I mean I have no intention of
using it for backups we already have a system for that which we
prefer, I mean clogin specifically (as someone mentioned). RANCID
contains a bunch of scripts that allow you to execute commands on a
whole range of vendor devices, I'm talking about bastardising though
as the interaction layer with the devices so I don't have to write my
own per vendor/make/OS.

- Obviously RANCID can't check syntax of config it would push, when we
make changes on the network we write out the full concfig to be
applied, it is peer reviewed by another engineer, then submitted to a
change board to reviewel, only then would it go into the reverse
rancid tool so that is no more risk than a human finally copying and
pasting it in. If at any point an error is through back, the tool
would see that.

- We also have a full virtual mock-up of the core and a nearly full
hardware mock-up in the lab, so again, the syntax will be tested, that
could even be built into the tool that at the scheduled time of change
executing it runs it on the lab first.

- Making changes to something seen as "dangerous" like routing
protocols shouldn't be shied away from because of the potential
impact, you have to find ways to de-risk the change. Like someone else
mentions we have thousand and thousands of devices, this must be
automated.

- Someone mentioned security, We have plenty of that locking down
rancid access to a sepcific IP, in a specific VRF, and the user
account is of course in Tacacs so we can then limit the exact commands
it runs on a per-device basis, we can even limit the dates/times the
account is allowed to log in. All comands that it does run at logged
back to Tacac's so its fully auditable. No issues there.

I'm really just interested in writing a web interface in which you can
paste in some config, give a date/time, username/password maybe, and
hostname/IP, at that time it uploads the configs. Also one must be
able to configure checks to the run before and after the config upload
as I said, like grabing the number of routers in a table, or number of
routes received from a specific neighbour, or number of neighbours we
are exchanging routes with. Then I can define some pass/fail criteria
and my reverse RANCID would simply roll back the changes, I could have
pre-supplied the equivilent config to roll back.

This sounds like fairly strait forward stuff so I could be missing
something. I have seen systems like NETCONF but the support isn't wide
spread yet, there is more support on JunOS at present than Cisco (and
we have more Cisco and JunOS) and no one is really rolling much in the
way of applications for utilising it.

Kind regards,
James.