Discussion:
[rancid] DONE: Implementation of the (generic) linux configuration backup-versioning DEVELOPMENT VERSION
Diego Ercolani
2009-07-07 13:50:28 UTC
Permalink
Hello to all,
I finished modules for the archive of linux configuration.
It's a development version so please dear developer support it in the main
trunk


I wrote 2 modules:
llogin
lrancid


---------- LLOGIN --------------------


llogin is based on clogin 2.3.2 script, try to login via ssh and telnet. It
should login as root.

I did many modification to the clogin script, and I suppose a better check and
better diagnostic look.

The point of view I started from is that linux normally stores configurations
as files in the filesystem, every distribution use its own directory structure
and often, an entire backup of /etc directory is too redundant so I added a
new configuration parameter in the .cloginrc and a new "MetaCommand" that is
BackupFiles that is runned by llogin script using .cloginrc files to feed it.


The new parameter in the .cloginrc is "backupfile", so you need to add for
your linux machines at least a line in the .cloginrc file in the clogin
syntax:

add backupfile <glob host identifier> <path/filename> [path/filename] ...

if you want you can of course add new files in a new line; this is the reason
why I modified the clogin "add" procedure to feed correctly the
int_backupfiles global variable the llogin script is using.

I wrote a new procedure that is send_debug that, when called with a string as
parameter, prints a gren "Debug:" followed by the string, if debug is on.
(this of course is to increment visibility of check strings along the expect
debug stream.

--------- LRANCID ---------
lrancid is a modified version of rancid 2.3.2 that calls llogin with the
BackupFiles metacommand as command argument.
It have some hack to optimize the output of the llogin script, but is more or
less very similar to the rancid script

----------- rancid-fe -----------------
I of course modified the rancid-fe script to manage a new device category that
is "linux".
I'm attaching it to complete the contribution.

----------- sample .cloginrc ----------------------
CUT >>>>>>>>>>>>>>
add method linux.machine.domain ssh rlogin
add user linux.machine.domain root
add password linux.machine.domain thepassword thepassword
add autoenable linux.machine.domain 1
add backupfile linux.machine.domain /etc/issue
add backupfile linux.machine.domain /etc/network/interfaces
#add backupfile linux.machine.domain /etc/network/interfaces
<<<<<<<<<<<<<< CUT <<<<<<<<<<<<<<<<

Hope this would help many of you
Diego Ercolani
Sam Munzani
2009-07-07 14:20:16 UTC
Permalink
Diego,

Many companies don't allow root user for the SSH. So usually its login
as normal user then either "su -" or "sudo su -". I think this script
can be more useful if it considers non root logins.

My 2 cents.

Thanks,
Sam
Post by Diego Ercolani
Hello to all,
I finished modules for the archive of linux configuration.
It's a development version so please dear developer support it in the main
trunk
llogin
lrancid
---------- LLOGIN --------------------
llogin is based on clogin 2.3.2 script, try to login via ssh and telnet. It
should login as root.
I did many modification to the clogin script, and I suppose a better check and
better diagnostic look.
The point of view I started from is that linux normally stores configurations
as files in the filesystem, every distribution use its own directory structure
and often, an entire backup of /etc directory is too redundant so I added a
new configuration parameter in the .cloginrc and a new "MetaCommand" that is
BackupFiles that is runned by llogin script using .cloginrc files to feed it.
The new parameter in the .cloginrc is "backupfile", so you need to add for
your linux machines at least a line in the .cloginrc file in the clogin
add backupfile <glob host identifier> <path/filename> [path/filename] ...
if you want you can of course add new files in a new line; this is the reason
why I modified the clogin "add" procedure to feed correctly the
int_backupfiles global variable the llogin script is using.
I wrote a new procedure that is send_debug that, when called with a string as
parameter, prints a gren "Debug:" followed by the string, if debug is on.
(this of course is to increment visibility of check strings along the expect
debug stream.
--------- LRANCID ---------
lrancid is a modified version of rancid 2.3.2 that calls llogin with the
BackupFiles metacommand as command argument.
It have some hack to optimize the output of the llogin script, but is more or
less very similar to the rancid script
----------- rancid-fe -----------------
I of course modified the rancid-fe script to manage a new device category that
is "linux".
I'm attaching it to complete the contribution.
----------- sample .cloginrc ----------------------
CUT >>>>>>>>>>>>>>
add method linux.machine.domain ssh rlogin
add user linux.machine.domain root
add password linux.machine.domain thepassword thepassword
add autoenable linux.machine.domain 1
add backupfile linux.machine.domain /etc/issue
add backupfile linux.machine.domain /etc/network/interfaces
#add backupfile linux.machine.domain /etc/network/interfaces
<<<<<<<<<<<<<< CUT <<<<<<<<<<<<<<<<
Hope this would help many of you
Diego Ercolani
------------------------------------------------------------------------
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
Thomas Donnelly
2009-07-07 14:29:12 UTC
Permalink
Also with FreeBSD remote root logins are not permitted by default so I
would have to change that on every one of my servers.

-=Tom
Post by Sam Munzani
Diego,
Many companies don't allow root user for the SSH. So usually its login
as normal user then either "su -" or "sudo su -". I think this script
can be more useful if it considers non root logins.
My 2 cents.
Thanks,
Sam
Post by Diego Ercolani
Hello to all,
I finished modules for the archive of linux configuration.
It's a development version so please dear developer support it in the main
trunk
llogin
lrancid
---------- LLOGIN --------------------
llogin is based on clogin 2.3.2 script, try to login via ssh and telnet. It
should login as root.
I did many modification to the clogin script, and I suppose a better check and
better diagnostic look.
The point of view I started from is that linux normally stores configurations
as files in the filesystem, every distribution use its own directory structure
and often, an entire backup of /etc directory is too redundant so I added a
new configuration parameter in the .cloginrc and a new "MetaCommand" that is
BackupFiles that is runned by llogin script using .cloginrc files to feed it.
The new parameter in the .cloginrc is "backupfile", so you need to add for
your linux machines at least a line in the .cloginrc file in the clogin
add backupfile <glob host identifier> <path/filename> [path/filename] ...
if you want you can of course add new files in a new line; this is the reason
why I modified the clogin "add" procedure to feed correctly the
int_backupfiles global variable the llogin script is using.
I wrote a new procedure that is send_debug that, when called with a string as
parameter, prints a gren "Debug:" followed by the string, if debug is on.
(this of course is to increment visibility of check strings along the expect
debug stream.
--------- LRANCID ---------
lrancid is a modified version of rancid 2.3.2 that calls llogin with the
BackupFiles metacommand as command argument.
It have some hack to optimize the output of the llogin script, but is more or
less very similar to the rancid script
----------- rancid-fe -----------------
I of course modified the rancid-fe script to manage a new device category that
is "linux".
I'm attaching it to complete the contribution.
----------- sample .cloginrc ----------------------
CUT >>>>>>>>>>>>>>
add method linux.machine.domain ssh rlogin
add user linux.machine.domain root
add password linux.machine.domain thepassword thepassword
add autoenable linux.machine.domain 1
add backupfile linux.machine.domain /etc/issue
add backupfile linux.machine.domain /etc/network/interfaces
#add backupfile linux.machine.domain /etc/network/interfaces
<<<<<<<<<<<<<< CUT <<<<<<<<<<<<<<<<
Hope this would help many of you
Diego Ercolani
------------------------------------------------------------------------
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
------------------------------------------------------------------------
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
Diego Ercolani
2009-07-07 14:38:11 UTC
Permalink
....and of course.... it should work also with FreeBSD....
Diego
Post by Thomas Donnelly
Also with FreeBSD remote root logins are not permitted by default so I
would have to change that on every one of my servers.
-=Tom
Post by Sam Munzani
Diego,
Many companies don't allow root user for the SSH. So usually its login
as normal user then either "su -" or "sudo su -". I think this script
can be more useful if it considers non root logins.
My 2 cents.
Thanks,
Sam
Michael W. Lucas
2009-07-07 14:40:46 UTC
Permalink
FreeBSD's base config would be fairly easy: /etc/rc.conf.

The problem would be the configuration of all the secondary daemons:
ntpd, sendmail, etc.

To do this properly, you're probably better off using a real system
configuration tool. RANCID is absolutely awesome at what it does, but
trying to extend it to accomodate one of my highly customized FreeBSD
boxes would be extremely difficult.
Post by Diego Ercolani
....and of course.... it should work also with FreeBSD....
Diego
Post by Thomas Donnelly
Also with FreeBSD remote root logins are not permitted by default so I
would have to change that on every one of my servers.
-=Tom
Post by Sam Munzani
Diego,
Many companies don't allow root user for the SSH. So usually its login
as normal user then either "su -" or "sudo su -". I think this script
can be more useful if it considers non root logins.
My 2 cents.
Thanks,
Sam
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
--
Michael W. Lucas ***@BlackHelicopters.org
http://www.MichaelWLucas.com/
Latest book: Cisco Routers for the Desperate, 2nd Edition
http://www.CiscoRoutersForTheDesperate.com/
Thomas Donnelly
2009-07-07 14:44:38 UTC
Permalink
Maybe a way to pass the locations of the configs for each box
individually? The RANCID interface is nice and I wouldnt mind making a
list of each file to be backed up for each box, as most of it would be
copy paste. Also if you do everything from ports (like a good little
UNIX user) all your configs should be in /etc and /usr/local/etc,
backing up anything.conf in all those directories would cover 99% of the
applications out there.

-=Tom
Post by Michael W. Lucas
FreeBSD's base config would be fairly easy: /etc/rc.conf.
ntpd, sendmail, etc.
To do this properly, you're probably better off using a real system
configuration tool. RANCID is absolutely awesome at what it does, but
trying to extend it to accomodate one of my highly customized FreeBSD
boxes would be extremely difficult.
Post by Diego Ercolani
....and of course.... it should work also with FreeBSD....
Diego
Post by Thomas Donnelly
Also with FreeBSD remote root logins are not permitted by default so I
would have to change that on every one of my servers.
-=Tom
Post by Sam Munzani
Diego,
Many companies don't allow root user for the SSH. So usually its login
as normal user then either "su -" or "sudo su -". I think this script
can be more useful if it considers non root logins.
My 2 cents.
Thanks,
Sam
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
Ryan West
2009-07-07 14:51:10 UTC
Permalink
Backuppc does a really good job of this BTW. It uses RSYNC, performs basic file level de-duplication and will allow to restore to any version from their interface. It's free and pretty much designed for *NIX boxes already.

This is listed as their main feature:

A clever pooling scheme minimizes disk storage and disk I/O. Identical files across multiple backups of the same or different PCs are stored only once resulting in substantial savings in disk storage and disk I/O.

http://backuppc.sourceforge.net/info.html

-ryan


-----Original Message-----
From: rancid-discuss-***@shrubbery.net [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of Thomas Donnelly
Sent: Tuesday, July 07, 2009 10:45 AM
To: Michael W. Lucas
Cc: rancid-***@shrubbery.net
Subject: [rancid] Re: DONE: Implementation of the (generic) linux configuration backup-versioning DEVELOPMENT VERSION

Maybe a way to pass the locations of the configs for each box
individually? The RANCID interface is nice and I wouldnt mind making a
list of each file to be backed up for each box, as most of it would be
copy paste. Also if you do everything from ports (like a good little
UNIX user) all your configs should be in /etc and /usr/local/etc,
backing up anything.conf in all those directories would cover 99% of the
applications out there.

-=Tom
Post by Michael W. Lucas
FreeBSD's base config would be fairly easy: /etc/rc.conf.
ntpd, sendmail, etc.
To do this properly, you're probably better off using a real system
configuration tool. RANCID is absolutely awesome at what it does, but
trying to extend it to accomodate one of my highly customized FreeBSD
boxes would be extremely difficult.
Post by Diego Ercolani
....and of course.... it should work also with FreeBSD....
Diego
Post by Thomas Donnelly
Also with FreeBSD remote root logins are not permitted by default so I
would have to change that on every one of my servers.
-=Tom
Post by Sam Munzani
Diego,
Many companies don't allow root user for the SSH. So usually its login
as normal user then either "su -" or "sudo su -". I think this script
can be more useful if it considers non root logins.
My 2 cents.
Thanks,
Sam
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
Lance Vermilion
2009-07-07 16:45:04 UTC
Permalink
This sounds like a very nice tool but unless I missed something I don't see
that it uses SVN/CVS to show the different revisions. That is scary
considering that it might overwrite a good file with a bad file should
someone change something.
Post by Ryan West
Backuppc does a really good job of this BTW. It uses RSYNC, performs basic
file level de-duplication and will allow to restore to any version from
their interface. It's free and pretty much designed for *NIX boxes already.
A clever pooling scheme minimizes disk storage and disk I/O. Identical
files across multiple backups of the same or different PCs are stored only
once resulting in substantial savings in disk storage and disk I/O.
http://backuppc.sourceforge.net/info.html
-ryan
-----Original Message-----
Sent: Tuesday, July 07, 2009 10:45 AM
To: Michael W. Lucas
Subject: [rancid] Re: DONE: Implementation of the (generic) linux
configuration backup-versioning DEVELOPMENT VERSION
Maybe a way to pass the locations of the configs for each box
individually? The RANCID interface is nice and I wouldnt mind making a
list of each file to be backed up for each box, as most of it would be
copy paste. Also if you do everything from ports (like a good little
UNIX user) all your configs should be in /etc and /usr/local/etc,
backing up anything.conf in all those directories would cover 99% of the
applications out there.
-=Tom
Post by Michael W. Lucas
FreeBSD's base config would be fairly easy: /etc/rc.conf.
ntpd, sendmail, etc.
To do this properly, you're probably better off using a real system
configuration tool. RANCID is absolutely awesome at what it does, but
trying to extend it to accomodate one of my highly customized FreeBSD
boxes would be extremely difficult.
Post by Diego Ercolani
....and of course.... it should work also with FreeBSD....
Diego
Post by Thomas Donnelly
Also with FreeBSD remote root logins are not permitted by default so I
would have to change that on every one of my servers.
-=Tom
Post by Sam Munzani
Diego,
Many companies don't allow root user for the SSH. So usually its login
as normal user then either "su -" or "sudo su -". I think this script
can be more useful if it considers non root logins.
My 2 cents.
Thanks,
Sam
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
Diego Ercolani
2009-07-07 14:36:30 UTC
Permalink
I did it already..... but I haven't tested
it uses the login password for the user you specify in the "user" parameter
it uses the "enable" password as the root password (the second password in the
"password" parameter). Please verify and correct the script if something is
going wrong
Diego

In data martedì 07 luglio 2009 16:20:16, Sam Munzani ha scritto:
: > Diego,
Post by Sam Munzani
Many companies don't allow root user for the SSH. So usually its login
as normal user then either "su -" or "sudo su -". I think this script
can be more useful if it considers non root logins.
My 2 cents.
Thanks,
Sam
Post by Diego Ercolani
----------- sample .cloginrc ----------------------
[...]
CUT >>>>>>>>>>>>>>
add method linux.machine.domain ssh rlogin
add user linux.machine.domain root
add password linux.machine.domain thepassword thepassword
add autoenable linux.machine.domain 1
add backupfile linux.machine.domain /etc/issue
add backupfile linux.machine.domain /etc/network/interfaces
#add backupfile linux.machine.domain /etc/network/interfaces
<<<<<<<<<<<<<< CUT <<<<<<<<<<<<<<<<
Hope this would help many of you
Diego Ercolani
Loading...