Discussion:
[rancid] [PATCH 0/2] Better subversion support, multiple fixes for Extreme XOS
Zenon Mousmoulas
2011-11-28 13:36:43 UTC
Permalink
Hi,

my colleague Faidon Liambotis and I had been running rancid 2.3.2a8 on
Debian Lenny 5.0 (Linux 2.6.26, expect 5.43) since 2009. We use
rancid for configuration monitoring in GRNET (Greek Research and
Technology Network, the Greek NREN), among other things for various
Extreme Networks switches running XOS. We occasionally had to
introduce fixes and workarounds in clogin and xrancid, in order to
restore compatibility with XOS devices, usually after a XOS upgrade
would break something.
Recently we had to upgrade to Debian Squeeze 6.0, at which time we
decided to forward port our changes to rancid 2.3.6 and submit them
in hope they will be considered for inclusion in the rancid code base.
In retrospect it might be a good idea to split clogin for XOS devices
into a new script (xlogin?), however such an effort would have been
beyond the scope of what we afforded to do in this context.

We also submit a few changes to improve support for using subversion
repositories with rancid.

These patches apply independently against rancid 2.3.6.

Thanks in advance for your comments and feedback.

Best regards,
Zenon Mousmoulas
Zenon Mousmoulas
2011-11-28 13:36:45 UTC
Permalink
Improve subversion support:
- Support arbitrary subversion URLs as (pre-provisioned) repositories.
- Do not overwrite an existing local repository.
- Avoid "svn: Directory '<GROUP>' is out of date" message.
Included for completeness, copied from Debian patch
by Nicolas DEFFAYET <nicolas-***@deffayet.com>:
http://patch-tracker.debian.org/patch/series/dl/rancid/2.3.6-1/09_svn.dpatch

Signed-off-by: Zenon Mousmoulas <***@noc.grnet.gr>
---

diff -ru rancid-2.3.6/bin/control_rancid.in rancid-2.3.6-grnet/bin/control_rancid.in
--- rancid-2.3.6/bin/control_rancid.in 2011-02-16 01:16:59.000000000 +0200
+++ rancid-2.3.6-grnet/bin/control_rancid.in 2011-11-26 00:12:08.000000000 +0200
@@ -167,6 +167,12 @@
fi
fi

+# svn update to avoid 'Out of date' error
+if [ $RCSSYS = svn ]
+then
+ svn update
+fi
+
# do cvs update of router.db in case anyone has fiddled.
$RCSSYS update router.db > $TMP 2>&1
grep "^C" $TMP > /dev/null
diff -ru rancid-2.3.6/bin/rancid-cvs.in rancid-2.3.6-grnet/bin/rancid-cvs.in
--- rancid-2.3.6/bin/rancid-cvs.in 2011-02-16 01:16:59.000000000 +0200
+++ rancid-2.3.6-grnet/bin/rancid-cvs.in 2011-11-26 00:12:08.000000000 +0200
@@ -93,11 +93,20 @@
fi

# Top level CVS stuff
-if [ ! -d $CVSROOT ]; then
- if [ $RCSSYS = cvs ]; then
+if [ $RCSSYS = cvs ]; then
+ if [ ! -d $CVSROOT ]; then
cvs -d $CVSROOT init
+ fi
+else
+ if echo "$CVSROOT" | grep -q "://"; then
+ # do nothing because CVSROOT is some sort of a URL
+ # also assume the repository has already been provisioned
+ :
else
- svnadmin create $CVSROOT @SVN_FSTYPE@
+ if [ ! -d $CVSROOT ] && ! svn ls "file://$CVSROOT" >/dev/null 2>&1; then
+ svnadmin create $CVSROOT @SVN_FSTYPE@
+ fi
+ CVSROOT="file://$CVSROOT"
fi
fi

@@ -128,9 +137,11 @@
cd $BASEDIR
cvs checkout $GROUP
else
- svn import -m "$GROUP" . file:///$CVSROOT/$GROUP
+ svn import -m "$GROUP" . $CVSROOT/$GROUP
cd $BASEDIR
- svn checkout file:///$CVSROOT/$GROUP $GROUP
+ svn checkout $CVSROOT/$GROUP $GROUP
+ cd $DIR
+ svn update
fi
fi
cd $DIR
diff -ru rancid-2.3.6/etc/rancid.conf.sample.in rancid-2.3.6-grnet/etc/rancid.conf.sample.in
--- rancid-2.3.6/etc/rancid.conf.sample.in 2011-02-16 01:16:59.000000000 +0200
+++ rancid-2.3.6-grnet/etc/rancid.conf.sample.in 2011-11-26 00:12:08.000000000 +0200
@@ -21,6 +21,15 @@
BASEDIR=@localstatedir@; export BASEDIR
PATH=@bindir@:@ENV_PATH@; export PATH
# Location of the CVS/SVN repository. Be careful changing this.
+# If RCSSYS is svn, this can be:
+# - an (absolute) path (a subdirectory of BASEDIR by default).
+# - any URL that subversion understands, but beware that:
+# - no attempt will be made to create the repository when
+# running rancid-cvs.
+# - authentication credentials, if necessary, MUST be cached
+# (see SVN book, Ch. 3, Network Model, Caching credentials)
+# before non-interactive commands can run, e.g. by running
+# rancid-cvs after installation.
CVSROOT=$BASEDIR/CVS; export CVSROOT
# Location of log files produced by rancid-run(1).
LOGDIR=$BASEDIR/logs; export LOGDIR
john heasley
2011-11-29 22:28:38 UTC
Permalink
Post by Zenon Mousmoulas
- Support arbitrary subversion URLs as (pre-provisioned) repositories.
- Do not overwrite an existing local repository.
- Avoid "svn: Directory '<GROUP>' is out of date" message.
Included for completeness, copied from Debian patch
http://patch-tracker.debian.org/patch/series/dl/rancid/2.3.6-1/09_svn.dpatch
+++ rancid-2.3.6-grnet/bin/control_rancid.in 2011-11-26 00:12:08.000000000 +0200
@@ -167,6 +167,12 @@
fi
fi
+# svn update to avoid 'Out of date' error
+if [ $RCSSYS = svn ]
+then
+ svn update
+fi
why is this necessary? it should never be out of date, unless someone has
altered the repository, in which case you want to be alerted to that.
Zenon Mousmoulas
2011-11-30 18:10:28 UTC
Permalink
Post by john heasley
Post by Zenon Mousmoulas
- Support arbitrary subversion URLs as (pre-provisioned) repositories.
- Do not overwrite an existing local repository.
- Avoid "svn: Directory '<GROUP>' is out of date" message.
Included for completeness, copied from Debian patch
http://patch-tracker.debian.org/patch/series/dl/rancid/2.3.6-1/09_svn.dpatch
Post by john heasley
Post by Zenon Mousmoulas
+++ rancid-2.3.6-grnet/bin/control_rancid.in 2011-11-26
00:12:08.000000000 +0200
@@ -167,6 +167,12 @@
fi
fi
+# svn update to avoid 'Out of date' error
+if [ $RCSSYS = svn ]
+then
+ svn update
+fi
why is this necessary? it should never be out of date, unless someone has
altered the repository, in which case you want to be alerted to that.
We "inherited" this due to merging with the Debian patch I
mentioned earlier, while forward porting our changes to 2.3.6. Since
we initially tried to apply our changes on top of the Debian package
for 2.3.6 (rather than vanilla 2.3.6), we had a conflict with the
other hunk from this Debian patch:

--- rancid~/bin/rancid-cvs.in
+++ rancid/bin/rancid-cvs.in
@@ -131,6 +131,8 @@
svn import -m "$GROUP" . file:///$CVSROOT/$GROUP
cd $BASEDIR
svn checkout file:///$CVSROOT/$GROUP $GROUP
+ cd $DIR
+ svn update
fi
fi
cd $DIR

So rather than sending only this change (which is required and has
been tested) we opted for merging with the Debian patch.

I am not sure however when the change you noted would be necessary and
I certainly can not object to your reasoning against it. Therefore I
suppose we can drop this hunk.

I can send an updated patch if you are OK with the rest of the
changes.

Cheers,
Z.
john heasley
2011-11-30 21:59:34 UTC
Permalink
Post by Zenon Mousmoulas
Post by john heasley
Post by Zenon Mousmoulas
- Support arbitrary subversion URLs as (pre-provisioned) repositories.
- Do not overwrite an existing local repository.
- Avoid "svn: Directory '<GROUP>' is out of date" message.
Included for completeness, copied from Debian patch
http://patch-tracker.debian.org/patch/series/dl/rancid/2.3.6-1/09_svn.dpatch
Post by john heasley
Post by Zenon Mousmoulas
+++ rancid-2.3.6-grnet/bin/control_rancid.in 2011-11-26
00:12:08.000000000 +0200
@@ -167,6 +167,12 @@
fi
fi
+# svn update to avoid 'Out of date' error
+if [ $RCSSYS = svn ]
+then
+ svn update
+fi
why is this necessary? it should never be out of date, unless someone
has
Post by john heasley
altered the repository, in which case you want to be alerted to that.
We "inherited" this due to merging with the Debian patch I
mentioned earlier, while forward porting our changes to 2.3.6. Since
we initially tried to apply our changes on top of the Debian package
for 2.3.6 (rather than vanilla 2.3.6), we had a conflict with the
--- rancid~/bin/rancid-cvs.in
+++ rancid/bin/rancid-cvs.in
@@ -131,6 +131,8 @@
svn import -m "$GROUP" . file:///$CVSROOT/$GROUP
cd $BASEDIR
svn checkout file:///$CVSROOT/$GROUP $GROUP
+ cd $DIR
+ svn update
fi
fi
cd $DIR
So rather than sending only this change (which is required and has
been tested) we opted for merging with the Debian patch.
I am not sure however when the change you noted would be necessary and
I certainly can not object to your reasoning against it. Therefore I
suppose we can drop this hunk.
there are three cases that i am aware of - 1) someone changes the respository
from another working set, 2) host crashes leaving files mangled (rare), or
3) when svn:ignore property is altered. i dont understand why changing that
property on the directory requires an update, but it does and that was added
to 2.3.6.
Post by Zenon Mousmoulas
I can send an updated patch if you are OK with the rest of the
changes.
Cheers,
Z.
Zenon Mousmoulas
2011-12-01 01:41:02 UTC
Permalink
Post by john heasley
Post by Zenon Mousmoulas
Post by john heasley
Post by Zenon Mousmoulas
- Support arbitrary subversion URLs as (pre-provisioned)
repositories.
Post by john heasley
Post by Zenon Mousmoulas
Post by john heasley
Post by Zenon Mousmoulas
- Do not overwrite an existing local repository.
- Avoid "svn: Directory '<GROUP>' is out of date" message.
Included for completeness, copied from Debian patch
http://patch-tracker.debian.org/patch/series/dl/rancid/2.3.6-1/09_svn.dpatch
Post by john heasley
Post by Zenon Mousmoulas
Post by john heasley
Post by Zenon Mousmoulas
+++ rancid-2.3.6-grnet/bin/control_rancid.in 2011-11-26
00:12:08.000000000 +0200
@@ -167,6 +167,12 @@
fi
fi
+# svn update to avoid 'Out of date' error
+if [ $RCSSYS = svn ]
+then
+ svn update
+fi
why is this necessary? it should never be out of date, unless someone
has
Post by john heasley
altered the repository, in which case you want to be alerted to that.
We "inherited" this due to merging with the Debian patch I
mentioned earlier, while forward porting our changes to 2.3.6. Since
we initially tried to apply our changes on top of the Debian package
for 2.3.6 (rather than vanilla 2.3.6), we had a conflict with the
--- rancid~/bin/rancid-cvs.in
+++ rancid/bin/rancid-cvs.in
@@ -131,6 +131,8 @@
svn import -m "$GROUP" . file:///$CVSROOT/$GROUP
cd $BASEDIR
svn checkout file:///$CVSROOT/$GROUP $GROUP
+ cd $DIR
+ svn update
fi
fi
cd $DIR
So rather than sending only this change (which is required and has
been tested) we opted for merging with the Debian patch.
I am not sure however when the change you noted would be necessary and
I certainly can not object to your reasoning against it. Therefore I
suppose we can drop this hunk.
there are three cases that i am aware of - 1) someone changes the respository
from another working set, 2) host crashes leaving files mangled (rare), or
3) when svn:ignore property is altered. i dont understand why changing that
property on the directory requires an update, but it does and that was added
to 2.3.6.
Got it, OK.
Post by john heasley
Post by Zenon Mousmoulas
I can send an updated patch if you are OK with the rest of the
changes.
?
Zenon Mousmoulas
2011-11-28 13:36:45 UTC
Permalink
Improve support for Extreme Networks devices running XOS:
- Fix end of config detection, since there is no "# End of configuration file"
anymore, since XOS 12.2 or maybe earlier (xrancid).
- Handle more forms of syntax error messages (xrancid).
- Also parse "show switch" output to grab chassis type (xrancid).
- ANSI escape sequences seem to be used since XOS 12.3 or maybe earlier
to draw the pager prompt, skip them if found in order to help with
pager prompt detection (clogin).
- The pager can be disabled per session since XOS 12.3, try to do that
in order to avoid struggling with pager prompts and ANSI sequences
altogether (clogin).
- Eat all space that may be there after username/password prompts (clogin).
- XOS runs a large banner by default after login (which matches prompt
detection!), however output buffering is traditionally and consistently
broken on XOS devices, causing mismatches in prompt detection and
subsequent commands. Therefore we need to wait 1 second to eat all output
up to the first prompt, for things to work from there on. Unfortunately
the delay affects all devices, not just XOS.

Signed-off-by: Zenon Mousmoulas <***@noc.grnet.gr>
---

diff -ru rancid-2.3.6/bin/clogin.in rancid-2.3.6-grnet/bin/clogin.in
--- rancid-2.3.6/bin/clogin.in 2011-02-16 01:16:59.000000000 +0200
+++ rancid-2.3.6-grnet/bin/clogin.in 2011-11-26 00:15:17.000000000 +0200
@@ -610,11 +610,9 @@
global do_saveconfig in_proc platform
set in_proc 1

- # If the prompt is (enable), then we are on a switch and the
- # command is "set length 0"; otherwise its "terminal length 0".
- # skip if its an extreme (since the pager can not be disabled on a
- # per-vty basis).
if { [ string compare "extreme" "$platform" ] } {
+ # If the prompt is (enable), then we are on a switch and the
+ # command is "set length 0"; otherwise its "terminal length 0".
if [ regexp -- ".*> .*enable" "$prompt" ] {
send "set length 0\r"
# This is ugly, but reduces code duplication, allowing the
@@ -631,7 +629,12 @@
-re "\[\n\r]+" { exp_continue }
}
} else {
+ send "disable clipaging\r"
set reprompt $prompt
+ expect {
+ -re $reprompt {}
+ -re "\[\n\r]+" { exp_continue }
+ }
}

# this is the only way i see to get rid of more prompts in o/p..grrrrr
@@ -640,11 +643,14 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious, with a
- # global switch in the config.
+ # for the "More" prompt.
for {set i 0} {$i < $num_commands} { incr i} {
send -- "[subst -nocommands [lindex $commands $i]]\r"
expect {
+ -re "^\x1b(\\\[|\(|\))\[;?0-9]*\[0-9A-Za-z]" { # skip ANSI escape sequences
+ set seen_ansi 1
+ exp_continue
+ }
-re "\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"
}
@@ -658,12 +664,14 @@
-re "\[\n\r]+" { send_user -- "$expect_out(buffer)"
exp_continue
}
- -re "\[^\r\n]*Press <SPACE> to cont\[^\r\n]*" {
+ -re "\[^\r\n]*Press <SPACE> to cont\[^\r\n]*" {
send " "
+ if { ! $seen_ansi } {
# bloody ^[[2K after " "
- expect {
- -re "^\[^\r\n]*\r" {}
- }
+ expect {
+ -re "^\[^\r\n]*\r" {}
+ }
+ }
exp_continue
}
-re "^ *--More--\[^\n\r]*" {
@@ -808,19 +816,19 @@
# Figure out prompts
set u_prompt [find userprompt $router]
if { "$u_prompt" == "" } {
- set u_prompt "(Username|Login|login|user name|User):"
+ set u_prompt "(Username|Login|login|user name|User): *"
} else {
set u_prompt [join [lindex $u_prompt 0] ""]
}
set p_prompt [find passprompt $router]
if { "$p_prompt" == "" } {
- set p_prompt "(\[Pp]assword|passwd|Enter password for \[^ :]+):"
+ set p_prompt "(\[Pp]assword|passwd|Enter password for \[^ :]+): *"
} else {
set p_prompt [join [lindex $p_prompt 0] ""]
}
set e_prompt [find enableprompt $router]
if { "$e_prompt" == "" } {
- set e_prompt "\[Pp]assword:"
+ set e_prompt "\[Pp]assword: *"
} else {
set e_prompt [join [lindex $e_prompt 0] ""]
}
@@ -861,6 +869,18 @@
# if login failed or rsh was unsuccessful, move on to the next device
continue
}
+
+ # Get all (output after login) you can eat in 1s
+ # (hopefully up to and including the first prompt).
+ # This is mostly necessary to work around
+ # stoopid extreme output buffering.
+ expect {
+ -timeout 1
+ -re "\[\n\r]+" { exp_continue }
+ -re "\[^\n\r]+" { exp_continue -continue_timer }
+ timeout {}
+ }
+
# Figure out the prompt.
if { [regexp -- "(#| \\(enable\\))" $prompt_match junk] == 1 } {
set enable 0
diff -ru rancid-2.3.6/bin/xrancid.in rancid-2.3.6-grnet/bin/xrancid.in
--- rancid-2.3.6/bin/xrancid.in 2011-02-16 01:16:59.000000000 +0200
+++ rancid-2.3.6-grnet/bin/xrancid.in 2011-11-26 00:15:17.000000000 +0200
@@ -280,6 +280,8 @@
/^(boot |next reboot)/i && next;
/^(auto |qos mode|sys\S*:|temperature|time)/i && next;

+ /^system type\s*:\s*(.+)/i &&
+ ProcessHistory("COMMENTS","keysort","A0","#Chassis type: $1\n") && next;
/^power supply: (.*)/i &&
ProcessHistory("COMMENTS","keysort","C0","#$1") && next;
/^license/i && ProcessHistory("COMMENTS","keysort","D0","#Image: $_")
@@ -300,14 +302,17 @@
print STDERR " In WriteTerm: $_" if ($debug);
my($comment) = 1; # strip extra comments, esp to preserve chassis type

+ $found_end = 0;
while (<INPUT>) {
tr/\015//d;
last if(/^$prompt/);
next if(/^\s*$/);
return(0) if(/^syntax error at token /i);
+ return(0) if(/^%% Invalid input detected at /i);
+ return(0) if(/^%% Ambiguous command:/i);
# the pager can not be disabled per-session on the PIX
s/^<-+ More -+>\s*//;
- return(0) if ($found_end);
+ last if ($found_end);

s/^\s*$/#/;
next if (/full detail configuration/i);
@@ -389,12 +394,6 @@

# catch anything that wasnt match above.
ProcessHistory("COMMENTS","keysort","H0","$_");
- # end of config
- if (/^# End of configuration file/i) {
- printf STDERR " End WriteTerm: $_" if ($debug);
- $found_end = 1;
- return(0);
- }
}

if ($lines < 3) {
@@ -403,6 +402,7 @@
return(-1);
}
$found_end = 1;
+ printf STDERR " End WriteTerm: $_" if ($debug);

return(0);
}
@@ -487,19 +487,19 @@
ProcessHistory("COMMENTS","keysort","X0","#\n");
TOP: while(<INPUT>) {
tr/\015//d;
- # note: this match sucks rocks, but currently the extreme bits are
- # unreliable about echoing the 'exit\n' command. this match might really
- # be a bad idea, but instead rely upon WriteTerm's found_end?
- if (/$prompt\s?(quit|exit|Connection( to \S+)? closed)/ && $found_end) {
- $clean_run = 1;
- last;
- }
if (/^Error:/) {
print STDOUT ("$host clogin error: $_");
print STDERR ("$host clogin error: $_") if ($debug);
$clean_run = 0;
last;
}
+ # note: this match sucks rocks, but currently the extreme bits are
+ # unreliable about echoing the 'exit\n' command. this match might really
+ # be a bad idea, but instead rely upon WriteTerm's found_end?
+ if (/($prompt\s?(quit|exit)|Connection( to \S+)? closed)/ && $found_end) {
+ $clean_run = 1;
+ last;
+ }
while (/$prompt\s*($cmds_regexp)\s*$/) {
$cmd = $1;
if (!defined($prompt)) {
john heasley
2011-11-29 22:51:46 UTC
Permalink
Post by Zenon Mousmoulas
- The pager can be disabled per session since XOS 12.3, try to do that
in order to avoid struggling with pager prompts and ANSI sequences
altogether (clogin).
+++ rancid-2.3.6-grnet/bin/clogin.in 2011-11-26 00:15:17.000000000 +0200
@@ -610,11 +610,9 @@
global do_saveconfig in_proc platform
set in_proc 1
- # If the prompt is (enable), then we are on a switch and the
- # command is "set length 0"; otherwise its "terminal length 0".
- # skip if its an extreme (since the pager can not be disabled on a
- # per-vty basis).
if { [ string compare "extreme" "$platform" ] } {
+ # If the prompt is (enable), then we are on a switch and the
+ # command is "set length 0"; otherwise its "terminal length 0".
if [ regexp -- ".*> .*enable" "$prompt" ] {
send "set length 0\r"
# This is ugly, but reduces code duplication, allowing the
@@ -631,7 +629,12 @@
-re "\[\n\r]+" { exp_continue }
}
} else {
+ send "disable clipaging\r"
set reprompt $prompt
+ expect {
+ -re $reprompt {}
+ -re "\[\n\r]+" { exp_continue }
+ }
}
# this is the only way i see to get rid of more prompts in o/p..grrrrr
@@ -640,11 +643,14 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious, with a
- # global switch in the config.
+ # for the "More" prompt.
i think this what you really want:

Index: bin/clogin.in
===================================================================
--- bin/clogin.in (revision 2340)
+++ bin/clogin.in (working copy)
@@ -633,8 +633,8 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious, with a
- # global switch in the config.
+ # for the "More" prompt. the extreme is equally obnoxious in pre-12.3 XOS,
+ # with a global switch in the config.
for {set i 0} {$i < $num_commands} { incr i} {
send -- "[subst -nocommands [lindex $commands $i]]\r"
expect {
@@ -897,20 +897,25 @@
}
}
if { $do_command || $do_script } {
- # If the prompt is (enable), then we are on a switch and the
- # command is "set length 0"; otherwise its "terminal length 0".
- if [regexp -- ".*> .*enable" "$prompt"] {
- send "set length 0\r"
- expect -re $prompt {}
- send "set width 132\r"
- expect -re $prompt {}
- send "set logging session disable\r"
+ if { [ string compare "extreme" "$platform" ] } {
+ # If the prompt is (enable), then we are on a switch and the
+ # command is "set length 0"; otherwise its "terminal length 0".
+ if [regexp -- ".*> .*enable" "$prompt"] {
+ send "set length 0\r"
+ expect -re $prompt {}
+ send "set width 132\r"
+ expect -re $prompt {}
+ send "set logging session disable\r"
+ } else {
+ send "terminal length 0\r"
+ expect -re $prompt {}
+ send "terminal width 132\r"
+ }
+ expect -re $prompt {}
} else {
- send "terminal length 0\r"
- expect -re $prompt {}
- send "terminal width 132\r"
+ send "disable clipaging\r"
+ expect -re $prompt {}
}
- expect -re $prompt {}
}
if { $do_command } {
if {[run_commands $prompt $command]} {
Post by Zenon Mousmoulas
- set u_prompt "(Username|Login|login|user name|User):"
+ set u_prompt "(Username|Login|login|user name|User): *"
why is this necessary? if there happen to be spaces after the prompt, it
should be possible to ignore them, send the input and allow the next expect
to eat the spaces, if they exist.
Post by Zenon Mousmoulas
} else {
set u_prompt [join [lindex $u_prompt 0] ""]
}
set p_prompt [find passprompt $router]
if { "$p_prompt" == "" } {
- set p_prompt "(\[Pp]assword|passwd|Enter password for \[^ :]+):"
+ set p_prompt "(\[Pp]assword|passwd|Enter password for \[^ :]+): *"
} else {
set p_prompt [join [lindex $p_prompt 0] ""]
}
set e_prompt [find enableprompt $router]
if { "$e_prompt" == "" } {
- set e_prompt "\[Pp]assword:"
+ set e_prompt "\[Pp]assword: *"
} else {
set e_prompt [join [lindex $e_prompt 0] ""]
}
Zenon Mousmoulas
2011-11-30 18:12:43 UTC
Permalink
Post by john heasley
Post by Zenon Mousmoulas
- The pager can be disabled per session since XOS 12.3, try to do that
in order to avoid struggling with pager prompts and ANSI sequences
altogether (clogin).
+++ rancid-2.3.6-grnet/bin/clogin.in 2011-11-26 00:15:17.000000000 +0200
@@ -610,11 +610,9 @@
global do_saveconfig in_proc platform
set in_proc 1
- # If the prompt is (enable), then we are on a switch and the
- # command is "set length 0"; otherwise its "terminal length 0".
- # skip if its an extreme (since the pager can not be disabled on a
- # per-vty basis).
if { [ string compare "extreme" "$platform" ] } {
+ # If the prompt is (enable), then we are on a switch and the
+ # command is "set length 0"; otherwise its "terminal length 0".
if [ regexp -- ".*> .*enable" "$prompt" ] {
send "set length 0\r"
# This is ugly, but reduces code duplication, allowing the
@@ -631,7 +629,12 @@
-re "\[\n\r]+" { exp_continue }
}
} else {
+ send "disable clipaging\r"
set reprompt $prompt
+ expect {
+ -re $reprompt {}
+ -re "\[\n\r]+" { exp_continue }
+ }
}
# this is the only way i see to get rid of more prompts in o/p..grrrrr
@@ -640,11 +643,14 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious, with a
- # global switch in the config.
+ # for the "More" prompt.
The following diff seems to be against some version of clogin (other
than 2.3.6) I don't have access to (is there a public source code repo
for rancid, btw?) so I can not apply/try it and therefore I'm not sure
if it is what I really want.
Post by john heasley
Index: bin/clogin.in
===================================================================
--- bin/clogin.in (revision 2340)
+++ bin/clogin.in (working copy)
@@ -633,8 +633,8 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious, with a
- # global switch in the config.
+ # for the "More" prompt. the extreme is equally obnoxious in pre-12.3 XOS,
+ # with a global switch in the config.
Actually I am not sure this last statement is true: we had not found
such a "global switch" in pre-12.3 XOS, as far as we can remember.
Post by john heasley
for {set i 0} {$i < $num_commands} { incr i} {
send -- "[subst -nocommands [lindex $commands $i]]\r"
expect {
@@ -897,20 +897,25 @@
}
}
if { $do_command || $do_script } {
- # If the prompt is (enable), then we are on a switch and the
- # command is "set length 0"; otherwise its "terminal length 0".
- if [regexp -- ".*> .*enable" "$prompt"] {
- send "set length 0\r"
- expect -re $prompt {}
- send "set width 132\r"
- expect -re $prompt {}
- send "set logging session disable\r"
+ if { [ string compare "extreme" "$platform" ] } {
+ # If the prompt is (enable), then we are on a switch and the
+ # command is "set length 0"; otherwise its "terminal length 0".
+ if [regexp -- ".*> .*enable" "$prompt"] {
+ send "set length 0\r"
+ expect -re $prompt {}
+ send "set width 132\r"
+ expect -re $prompt {}
+ send "set logging session disable\r"
+ } else {
+ send "terminal length 0\r"
+ expect -re $prompt {}
+ send "terminal width 132\r"
+ }
+ expect -re $prompt {}
} else {
- send "terminal length 0\r"
- expect -re $prompt {}
- send "terminal width 132\r"
+ send "disable clipaging\r"
+ expect -re $prompt {}
}
- expect -re $prompt {}
}
if { $do_command } {
if {[run_commands $prompt $command]} {
Post by Zenon Mousmoulas
- set u_prompt "(Username|Login|login|user name|User):"
+ set u_prompt "(Username|Login|login|user name|User): *"
why is this necessary? if there happen to be spaces after the prompt, it
should be possible to ignore them, send the input and allow the next expect
to eat the spaces, if they exist.
This was part of the changes we ported from 2.3.2a8 to 2.3.6 and it
was used (still used today actually) in production at GRNET since
2009, so it was definitely necessary at some point; unfortunately neither
Faidon nor I can no longer remember a specific case where this would
be necessary. Your explanation is reasonable. I suppose we can drop this.
Post by john heasley
Post by Zenon Mousmoulas
} else {
set u_prompt [join [lindex $u_prompt 0] ""]
}
set p_prompt [find passprompt $router]
if { "$p_prompt" == "" } {
- set p_prompt "(\[Pp]assword|passwd|Enter password for \[^ :]+):"
+ set p_prompt "(\[Pp]assword|passwd|Enter password for \[^ :]+): *"
} else {
set p_prompt [join [lindex $p_prompt 0] ""]
}
set e_prompt [find enableprompt $router]
if { "$e_prompt" == "" } {
- set e_prompt "\[Pp]assword:"
+ set e_prompt "\[Pp]assword: *"
} else {
set e_prompt [join [lindex $e_prompt 0] ""]
}
john heasley
2011-11-30 21:51:10 UTC
Permalink
Post by Zenon Mousmoulas
Post by john heasley
Post by Zenon Mousmoulas
- The pager can be disabled per session since XOS 12.3, try to do that
in order to avoid struggling with pager prompts and ANSI sequences
altogether (clogin).
+++ rancid-2.3.6-grnet/bin/clogin.in 2011-11-26 00:15:17.000000000
+0200
Post by john heasley
Post by Zenon Mousmoulas
@@ -610,11 +610,9 @@
global do_saveconfig in_proc platform
set in_proc 1
- # If the prompt is (enable), then we are on a switch and the
- # command is "set length 0"; otherwise its "terminal length 0".
- # skip if its an extreme (since the pager can not be disabled on a
- # per-vty basis).
if { [ string compare "extreme" "$platform" ] } {
+ # If the prompt is (enable), then we are on a switch and the
+ # command is "set length 0"; otherwise its "terminal length
0".
Post by john heasley
Post by Zenon Mousmoulas
if [ regexp -- ".*> .*enable" "$prompt" ] {
send "set length 0\r"
# This is ugly, but reduces code duplication, allowing the
@@ -631,7 +629,12 @@
-re "\[\n\r]+" { exp_continue }
}
} else {
+ send "disable clipaging\r"
set reprompt $prompt
+ expect {
+ -re $reprompt {}
+ -re "\[\n\r]+" { exp_continue }
+ }
}
# this is the only way i see to get rid of more prompts in o/p..grrrrr
@@ -640,11 +643,14 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious, with a
- # global switch in the config.
+ # for the "More" prompt.
The following diff seems to be against some version of clogin (other
than 2.3.6) I don't have access to (is there a public source code repo
for rancid, btw?) so I can not apply/try it and therefore I'm not sure
if it is what I really want.
attached
Post by Zenon Mousmoulas
Post by john heasley
Index: bin/clogin.in
===================================================================
--- bin/clogin.in (revision 2340)
+++ bin/clogin.in (working copy)
@@ -633,8 +633,8 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious, with a
- # global switch in the config.
+ # for the "More" prompt. the extreme is equally obnoxious in pre-12.3 XOS,
+ # with a global switch in the config.
Actually I am not sure this last statement is true: we had not found
such a "global switch" in pre-12.3 XOS, as far as we can remember.
What came before XOS? has it always been called XOS? the old extreme boxes
had a global knob, IIRC.
Post by Zenon Mousmoulas
Post by john heasley
for {set i 0} {$i < $num_commands} { incr i} {
send -- "[subst -nocommands [lindex $commands $i]]\r"
expect {
@@ -897,20 +897,25 @@
}
}
if { $do_command || $do_script } {
- # If the prompt is (enable), then we are on a switch and the
- # command is "set length 0"; otherwise its "terminal length 0".
- if [regexp -- ".*> .*enable" "$prompt"] {
- send "set length 0\r"
- expect -re $prompt {}
- send "set width 132\r"
- expect -re $prompt {}
- send "set logging session disable\r"
+ if { [ string compare "extreme" "$platform" ] } {
+ # If the prompt is (enable), then we are on a switch and the
+ # command is "set length 0"; otherwise its "terminal length 0".
+ if [regexp -- ".*> .*enable" "$prompt"] {
+ send "set length 0\r"
+ expect -re $prompt {}
+ send "set width 132\r"
+ expect -re $prompt {}
+ send "set logging session disable\r"
+ } else {
+ send "terminal length 0\r"
+ expect -re $prompt {}
+ send "terminal width 132\r"
+ }
+ expect -re $prompt {}
} else {
- send "terminal length 0\r"
- expect -re $prompt {}
- send "terminal width 132\r"
+ send "disable clipaging\r"
+ expect -re $prompt {}
}
- expect -re $prompt {}
}
if { $do_command } {
if {[run_commands $prompt $command]} {
Post by Zenon Mousmoulas
- set u_prompt "(Username|Login|login|user name|User):"
+ set u_prompt "(Username|Login|login|user name|User): *"
why is this necessary? if there happen to be spaces after the prompt,
it
Post by john heasley
should be possible to ignore them, send the input and allow the next
expect
Post by john heasley
to eat the spaces, if they exist.
This was part of the changes we ported from 2.3.2a8 to 2.3.6 and it
was used (still used today actually) in production at GRNET since
2009, so it was definitely necessary at some point; unfortunately neither
Faidon nor I can no longer remember a specific case where this would
be necessary. Your explanation is reasonable. I suppose we can drop this.
Post by john heasley
Post by Zenon Mousmoulas
} else {
set u_prompt [join [lindex $u_prompt 0] ""]
}
set p_prompt [find passprompt $router]
if { "$p_prompt" == "" } {
- set p_prompt "(\[Pp]assword|passwd|Enter password for \[^ :]+):"
+ set p_prompt "(\[Pp]assword|passwd|Enter password for \[^ :]+): *"
} else {
set p_prompt [join [lindex $p_prompt 0] ""]
}
set e_prompt [find enableprompt $router]
if { "$e_prompt" == "" } {
- set e_prompt "\[Pp]assword:"
+ set e_prompt "\[Pp]assword: *"
} else {
set e_prompt [join [lindex $e_prompt 0] ""]
}
john heasley
2011-11-30 22:01:07 UTC
Permalink
Post by john heasley
Post by Zenon Mousmoulas
Post by Zenon Mousmoulas
- The pager can be disabled per session since XOS 12.3, try to do that
in order to avoid struggling with pager prompts and ANSI sequences
altogether (clogin).
+++ rancid-2.3.6-grnet/bin/clogin.in 2011-11-26 00:15:17.000000000
+0200
Post by Zenon Mousmoulas
@@ -610,11 +610,9 @@
global do_saveconfig in_proc platform
set in_proc 1
- # If the prompt is (enable), then we are on a switch and the
- # command is "set length 0"; otherwise its "terminal length 0".
- # skip if its an extreme (since the pager can not be disabled on a
- # per-vty basis).
if { [ string compare "extreme" "$platform" ] } {
+ # If the prompt is (enable), then we are on a switch and the
+ # command is "set length 0"; otherwise its "terminal length
0".
Post by Zenon Mousmoulas
if [ regexp -- ".*> .*enable" "$prompt" ] {
send "set length 0\r"
# This is ugly, but reduces code duplication, allowing the
@@ -631,7 +629,12 @@
-re "\[\n\r]+" { exp_continue }
}
} else {
+ send "disable clipaging\r"
set reprompt $prompt
+ expect {
+ -re $reprompt {}
+ -re "\[\n\r]+" { exp_continue }
+ }
}
# this is the only way i see to get rid of more prompts in o/p..grrrrr
@@ -640,11 +643,14 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious, with a
- # global switch in the config.
+ # for the "More" prompt.
The following diff seems to be against some version of clogin (other
than 2.3.6) I don't have access to (is there a public source code repo
for rancid, btw?) so I can not apply/try it and therefore I'm not sure
if it is what I really want.
attached
Sorry, forgot the attachment.
Zenon Mousmoulas
2011-12-01 01:44:32 UTC
Permalink
Post by john heasley
Post by Zenon Mousmoulas
Post by john heasley
Post by Zenon Mousmoulas
- The pager can be disabled per session since XOS 12.3, try to do that
in order to avoid struggling with pager prompts and ANSI sequences
altogether (clogin).
+++ rancid-2.3.6-grnet/bin/clogin.in 2011-11-26 00:15:17.000000000
+0200
Post by john heasley
Post by Zenon Mousmoulas
@@ -610,11 +610,9 @@
global do_saveconfig in_proc platform
set in_proc 1
- # If the prompt is (enable), then we are on a switch and the
- # command is "set length 0"; otherwise its "terminal length 0".
- # skip if its an extreme (since the pager can not be disabled
on
Post by john heasley
Post by Zenon Mousmoulas
Post by john heasley
Post by Zenon Mousmoulas
a
- # per-vty basis).
if { [ string compare "extreme" "$platform" ] } {
+ # If the prompt is (enable), then we are on a switch and the
+ # command is "set length 0"; otherwise its "terminal length
0".
Post by john heasley
Post by Zenon Mousmoulas
if [ regexp -- ".*> .*enable" "$prompt" ] {
send "set length 0\r"
# This is ugly, but reduces code duplication, allowing the
@@ -631,7 +629,12 @@
-re "\[\n\r]+" { exp_continue }
}
} else {
+ send "disable clipaging\r"
set reprompt $prompt
+ expect {
+ -re $reprompt {}
+ -re "\[\n\r]+" { exp_continue }
+ }
}
# this is the only way i see to get rid of more prompts in o/p..grrrrr
@@ -640,11 +643,14 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious,
with
Post by john heasley
Post by Zenon Mousmoulas
Post by john heasley
Post by Zenon Mousmoulas
a
- # global switch in the config.
+ # for the "More" prompt.
The following diff seems to be against some version of clogin (other
than 2.3.6) I don't have access to (is there a public source code repo
for rancid, btw?) so I can not apply/try it and therefore I'm not sure
if it is what I really want.
attached
I tried it. The part about "disable clipaging" looks good.


However there's still a problem in this version with buffered output from
the extreme. After login, when you send "\r", the following expect
catches the first prompt, not the one after "\r". This becomes evident
later on, when the expect statement in the loop within run_commands
catches
"disable clipaging\r\n\<prompt>" rather than the command echoed and
its' output. So you're always one step behind, until the end, where you
erroneously see a prompt after sending "quit\r" so you also
send -h "exit\r". This obviously messes up xrancid parsing as
well. See the attached clogin log for more evidence (with exp_internal
added after login).

The following snippet from our patch addresses this issue, admittedly
not very elegantly, but we saw no way to deal with this other than
waiting for the extreme to send everything up to the first
prompt. Actually, according to my tests, the same thing can also happen on
a
cisco if you configure a very very large exec banner. So this is
generally useful, though not as much/often as for extreme.

+ # Get all (output after login) you can eat in 1s
+ # (hopefully up to and including the first prompt).
+ # This is mostly necessary to work around
+ # stoopid extreme output buffering.
+ expect {
+ -timeout 1
+ -re "\[\n\r]+" { exp_continue }
+ -re "\[^\n\r]+" { exp_continue -continue_timer }
+ timeout {}
+ }
Post by john heasley
Post by Zenon Mousmoulas
Post by john heasley
Index: bin/clogin.in
===================================================================
--- bin/clogin.in (revision 2340)
+++ bin/clogin.in (working copy)
@@ -633,8 +633,8 @@
set commands [split $command \;]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
- # for the "More" prompt. the extreme is equally obnoxious, with a
- # global switch in the config.
+ # for the "More" prompt. the extreme is equally obnoxious in pre-12.3 XOS,
+ # with a global switch in the config.
Actually I am not sure this last statement is true: we had not found
such a "global switch" in pre-12.3 XOS, as far as we can remember.
What came before XOS? has it always been called XOS? the old extreme boxes
had a global knob, IIRC.
Dunno, ExtremeWare? Fortunately we did not get a chance to see
that. Anyway, we didn't see such a knob in any version of XOS 12.x.
Zenon Mousmoulas
2011-12-01 01:47:56 UTC
Permalink
Post by Zenon Mousmoulas
[...]
However there's still a problem in this version with buffered output from
the extreme. After login, when you send "\r", the following expect
catches the first prompt, not the one after "\r". This becomes evident
later on, when the expect statement in the loop within run_commands
catches
"disable clipaging\r\n\<prompt>" rather than the command echoed and
its' output. So you're always one step behind, until the end, where you
erroneously see a prompt after sending "quit\r" so you also
send -h "exit\r". This obviously messes up xrancid parsing as
well. See the attached clogin log for more evidence (with exp_internal
added after login).
Plus the attachment, sorry.
john heasley
2011-12-07 23:15:07 UTC
Permalink
Post by Zenon Mousmoulas
Post by Zenon Mousmoulas
[...]
However there's still a problem in this version with buffered output
from
Post by Zenon Mousmoulas
the extreme. After login, when you send "\r", the following expect
catches the first prompt, not the one after "\r". This becomes evident
later on, when the expect statement in the loop within run_commands
catches
"disable clipaging\r\n\<prompt>" rather than the command echoed and
its' output. So you're always one step behind, until the end, where you
erroneously see a prompt after sending "quit\r" so you also
send -h "exit\r". This obviously messes up xrancid parsing as
well. See the attached clogin log for more evidence (with exp_internal
added after login).
Plus the attachment, sorry.
some data missing there. could you send a complete login to me?
clogin -s 'show version' host

Loading...