[rancid] Rancid-missed cmds

Discussion:

Erica James

2013-09-16 13:11:27 UTC

Hello
I have a problem with rancid parsing script. Logs show missed cmd(s).
Trying to get all of the configs.
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: End of run not found
#
All routers sucessfully completed.
cvs diff: Diffing .
cvs diff: Diffing configs
cvs commit: Examining .
cvs commit: Examining configs

The script is for Huawei. The problem now seems to be that rancid is not collecting configs.
Anyone with an idea on what to do?

Below is the rancid-script

#! /usr/bin/perl
##
## $Id$
##
## rancid 2.3.8
##
## Copyright (C) 1997-2004 by Terrapin Communications, Inc.
## All rights reserved.
##
## This software may be freely copied, modified and redistributed
## without fee for non-commerical purposes provided that this license
## remains intact and unmodified with any RANCID distribution.
##
## There is no warranty or other guarantee of fitness of this software.
## It is provided solely "as is". The author(s) disclaim(s) all
## responsibility and liability with respect to this software's usage
## or its effect upon hardware, computer systems, other software, or
## anything else.
##
## Except where noted otherwise, rancid was written by and is maintained by
## Henry Kilmer, John Heasley, Andrew Partan, Pete Whiting, and Austin Schutz.
##
#
# hurancid - Interface to Huawei devices
#
# RANCID - Really Awesome New Cisco confIg Differ
#
# usage: rancid [-d] [-l] [-f filename | $host]
#
use Getopt::Std;
getopts('dfl');
$log = $opt_l;
$debug = $opt_d;
$file = $opt_f;
$host = $ARGV[0];
$clean_run = 0;
$found_end = 0;
$timeo = 90;                    # hulogin timeout in seconds

my(%filter_pwds);               # password filtering mode
# This routine is used to print out the router configuration
sub ProcessHistory {
    my($new_hist_tag,$new_command,$command_string,@string)=(@_);
if((($new_hist_tag ne $hist_tag) || ($new_command ne $command))
       && defined %history) {
        print eval "$command \%history";
        undef %history;
    }
    if (($new_hist_tag) && ($new_command) && ($command_string)) {
        if ($history{$command_string}) {
            $history{$command_string} = "mailto:$history%7B$command_string%***@string";
        } else {
            $history{$command_string} = "@string";
        }
    } elsif (($new_hist_tag) && ($new_command)) {
        $history{++$#history} = "@string";
    } else {
        print "@string";
    }
    $hist_tag = $new_hist_tag;
    $command = $new_command;
    1;
}

sub numerically { $a <=> $b; }

# This is a sort routing that will sort numerically on the
# keys of a hash as if it were a normal array.
sub keynsort {
    local(%lines)=@_;
    local($i) = 0;
    local(@sorted_lines);
    foreach $key (sort numerically keys(%lines)) {
        $sorted_lines[$i] = $lines{$key};
        $i++;
    }
    @sorted_lines;
}

# This is a sort routing that will sort on the
# keys of a hash as if it were a normal array.
sub keysort {
    local(%lines)=@_;
    local($i) = 0;
    local(@sorted_lines);
    foreach $key (sort keys(%lines)) {
        $sorted_lines[$i] = $lines{$key};
        $i++;
    }
    @sorted_lines;
}

# This is a sort routing that will sort on the
# values of a hash as if it were a normal array.
sub valsort{
    local(%lines)=@_;
    local($i) = 0;
    local(@sorted_lines);
    foreach $key (sort values %lines) {
        $sorted_lines[$i] = $key;
        $i++;
    }
    @sorted_lines;
}

# This is a numerical sort routing (ascending).
sub numsort {
    local(%lines)=@_;
    local($i) = 0;
    local(@sorted_lines);
    foreach $num (sort {$a <=> $b} keys %lines) {
        $sorted_lines[$i] = $lines{$num};
        $i++;
    }
    @sorted_lines;
}

# This is a sort routine that will sort on the
# ip address when the ip address is anywhere in
# the strings.
sub ipsort {
    local(%lines)=@_;
    local($i) = 0;
    local(@sorted_lines);
    foreach $addr (sort sortbyipaddr keys %lines) {
        $sorted_lines[$i] = $lines{$addr};
        $i++;
    }
    @sorted_lines;
}

# These two routines will sort based upon IP addresses
sub ipaddrval {
    my(@a) = ($_[0] =~ m#^(\d+)\.(\d+)\.(\d+)\.(\d+)$#);
    $a[3]+256*($a[2]+256*($a[1]+256*$a[0]));
}
sub sortbyipaddr {
    &ipaddrval($a) <=> &ipaddrval($b);
}

# This routine processes a "show configuration"
# This routine processes a "show configuration"
sub WriteTerm {
    print STDERR "    In WriteTerm: $_" if ($debug);
    #my($lineauto) = 0;

    while (<INPUT>) {
#print STDERR "History $_\n";

        tr/\015//d;
        last if(/^$prompt/);
       next if (/^(\s*|\s*$cmd\s*)$/);
        return(1) if (/^\s*\^\s*$/);
        return(1) if (/Line has invalid autocommand /);
        return(1) if (/(Invalid (input|command) detected|Type help or )/i);
        return(-1) if (/command authorization failed/i);
        # the pager can not be disabled per-session on the PIX
        if (/^(<-+ More -+>)/) {
            my($len) = length($1);
            s/^$1\s{$len}//;
        }

        /^/ && next;
        ProcessHistory("COMMENTS","keysort","B1","# $_") && next;
    }

        # end of config...is a comment.
        if (/^return/i) {
$found_end = 1;
            return(1);
        }

        return(0);
}

# This routine copies everything with precefing "#"
sub CopyComment {
    print STDERR "    In CopyComment: $_" if ($debug);

    while (<INPUT>) {
        tr/\015//d;
        last if (/^$prompt/);

        ProcessHistory("","","","# $_");
        # end of config
    }
    return(0);
}

# filter out "uptime" lines
sub FilterUptime {
    print STDERR "    In FilterUptime: $_" if ($debug);

    while (<INPUT>) {
        tr/\015//d;
        last if (/^$prompt/);
#nmeongeza
        next if (/^(\s*|\s*$cmd\s*)$/);
        return(-1) if (/command authorization failed/i);
        /^VERSION\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C1","\#\n# Version: $1\n")
&& next;
        /^PATCH\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C2","\# Patch: $1\n")
&& next;

        /^PRODUCT\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C3","\# Version: $1\n")
&& next;
}
return(0);
}

# dummy function
sub DoNothing {print STDOUT;}

# Main
@commandtable = (
        {'display version'                       => 'FilterUptime'},
        {'display patch-information'             => 'CopyComment'},
        {'display device'                        => 'CopyComment'},
        {'display device pic-status'             => 'CopyComment'},
        {'display current-configuration'         => 'WriteTerm'},
);

# Use an array to preserve the order of the commands and a hash for mapping
# commands to the subroutine and track commands that have been completed.
@commands = map(keys(%$_), @commandtable);
%commands = map(%$_, @commandtable);

$huaw_cmds=join(";",@commands);
$cmds_regexp=join("|",@commands);

open(OUTPUT,">$host.new") || die "Can't open $host.new for writing: $!\n";
select(OUTPUT);
# make OUTPUT unbuffered if debugging
if ($debug) { $| = 1; }

if ($file) {
    print STDERR "opening file $host\n" if ($debug);
    print STDOUT "opening file $host\n" if ($log);
    open(INPUT,"<$host") || die "open failed for $host: $!\n";
} else {
    print STDERR "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n" if ($debug);
    print STDOUT "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n" if ($debug);
    if (defined($ENV{NOPIPE}) && $ENV{NOPIPE} =~ /^YES/i) {
        system "hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null > $host.raw" || die "hulogin failed for $host: $!\n";
        open(INPUT, "< $host.raw") || die "hulogin failed for $host: $!\n";
    } else {
        open(INPUT,"hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null |") || die "hulogin failed for $host: $!\n";
    }
}

# determine password filtering mode
if ($ENV{"FILTER_PWDS"} =~ /no/i) {
        $filter_pwds = 0;
} elsif ($ENV{"FILTER_PWDS"} =~ /all/i) {
        $filter_pwds = 2;
} else {
        $filter_pwds = 1;
}

ProcessHistory("","","","#RANCID-CONTENT-TYPE: Huawei\n#\n");
ProcessHistory("COMMENTS","keysort","B0","#\n");
ProcessHistory("COMMENTS","keysort","F0","#\n");
ProcessHistory("COMMENTS","keysort","G0","#\n");
TOP: while(<INPUT>) {
    tr/\015//d;

#print STDERR ("CMD: $_\n");

    if (/\>\s?quit.*$/) {
        $clean_run=1;
        last;
    }

    if (/^Error:/) {
        print STDOUT ("$host hulogin error: $_");
        print STDERR ("$host hulogin error: $_") if ($debug);
        $clean_run=0;
        last;
}

    while (/\>\007*\s*($cmds_regexp)\s*$/) {
        $cmd = $1;

        if (!defined($prompt)) {
            $prompt = ($_ =~ /^([^>]+\>)/)[0];
            $prompt =~ s/([][}{)(file://])///$1/g;
            print STDERR ("PROMPT MATCH: $prompt\n") if ($debug);
        }
        print STDERR ("HIT COMMAND:$_") if ($debug);
        if (! defined($commands{$cmd})) {
            print STDERR "$host: found unexpected command - \"$cmd\"\n";
            $clean_run = 0;
            last TOP;
        }
        $rval = &{$commands{$cmd}};
        delete($commands{$cmd});
        if ($rval == -1) {
            $clean_run = 0;
            last TOP;
        }
    }
}
print STDOUT "Done $logincmd: $_\n" if ($log);
# Flush History
ProcessHistory("","","","");
# Cleanup
close(INPUT);
close(OUTPUT);

if (defined($ENV{NOPIPE})) {
    #unlink("$host.raw") if (! $debug);
}
# check for completeness
if (scalar(%commands) || !$clean_run || !$found_end) {
    if (scalar(%commands)) {
        printf(STDOUT "$host: missed cmd(s): %s\n", join(',', keys(%commands)));
        printf(STDERR "$host: missed cmd(s): %s\n", join(',', keys(%commands))) if ($debug);
    }
    if (!$clean_run || !$found_end) {
        print STDOUT "$host: End of run not found\n";
        print STDERR "$host: End of run not found\n" if ($debug);
        system("/usr/bin/tail -1 $host.new");
    }
    #unlink "$host.new" if (! $debug);
}

Alan McKinnon

2013-09-16 13:33:32 UTC

Permalink

The approach to fixing this is always the same, at this point nothing
rancid is outputting helps debug in the slightest - all it says is that
something went wrong, but we already know that.

Do the following:

1. telnet to device and check you can auth manually
2. "hulogin <device>" and check it logs in and enables properly, and
that the prompt etc on the screen is correct
3. run "hulogin -t 90 -c"display current-configuration,display version"
<device>" and check that the commands do actually run correctly
4. run "hurancid -d <device>" - this dumps a *complete* log in the
current directory containing more info than the regular log you quoted
and often reveals the problem (but not always).

But first a few prior steps need to be verified:

- the command in #3 I pasted from your log output, and that has comma
separators. Your device might need semicolons, I have no idea as I don;t
know that vendors stuff
- the relevant line in router.db must contain the appropriate type
string in field 3
- you will have had to edit rancid-fe and add a line for huawei to a
perl hash. Without this, rancid-run doesn't know what parser script to
call and it cannot work.
- the device you have must be supported by the script you are using;
just because Huawei made it is no guarantee it works similarly to
anything else they might have made.

If all of that checks out, then we need to look into the running perl
itself.

I believe we've been over most of this before, I don't recall getting a
definitive answer. You really do need to go through all these steps as
given, nothing else is going to get to the root of your problem.

Post by Erica James
Hello
I have a problem with rancid parsing script. Logs show missed cmd(s).
Trying to get all of the configs.
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: End of run not found
#
All routers sucessfully completed.
cvs diff: Diffing .
cvs diff: Diffing configs
cvs commit: Examining .
cvs commit: Examining configs
The script is for Huawei. The problem now seems to be that rancid is not
collecting configs.
Anyone with an idea on what to do?
Below is the rancid-script
#! /usr/bin/perl
##
## $Id$
##
## rancid 2.3.8
##
## Copyright (C) 1997-2004 by Terrapin Communications, Inc.
## All rights reserved.
##
## This software may be freely copied, modified and redistributed
## without fee for non-commerical purposes provided that this license
## remains intact and unmodified with any RANCID distribution.
##
## There is no warranty or other guarantee of fitness of this software.
## It is provided solely "as is". The author(s) disclaim(s) all
## responsibility and liability with respect to this software's usage
## or its effect upon hardware, computer systems, other software, or
## anything else.
##
## Except where noted otherwise, rancid was written by and is maintained by
## Henry Kilmer, John Heasley, Andrew Partan, Pete Whiting, and Austin Schutz.
##
#
# hurancid - Interface to Huawei devices
#
# RANCID - Really Awesome New Cisco confIg Differ
#
# usage: rancid [-d] [-l] [-f filename | $host]
#
use Getopt::Std;
getopts('dfl');
$log = $opt_l;
$debug = $opt_d;
$file = $opt_f;
$host = $ARGV[0];
$clean_run = 0;
$found_end = 0;
$timeo = 90; # hulogin timeout in seconds
my(%filter_pwds); # password filtering mode
# This routine is used to print out the router configuration
sub ProcessHistory {
if((($new_hist_tag ne $hist_tag) || ($new_command ne $command))
&& defined %history) {
print eval "$command \%history";
undef %history;
}
if (($new_hist_tag) && ($new_command) && ($command_string)) {
if ($history{$command_string}) {
$history{$command_string} =
} else {
}
} elsif (($new_hist_tag) && ($new_command)) {
} else {
}
$hist_tag = $new_hist_tag;
$command = $new_command;
1;
}
sub numerically { $a <=> $b; }
# This is a sort routing that will sort numerically on the
# keys of a hash as if it were a normal array.
sub keynsort {
local($i) = 0;
foreach $key (sort numerically keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
@sorted_lines;
}
# This is a sort routing that will sort on the
# keys of a hash as if it were a normal array.
sub keysort {
local($i) = 0;
foreach $key (sort keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
@sorted_lines;
}
# This is a sort routing that will sort on the
# values of a hash as if it were a normal array.
sub valsort{
local($i) = 0;
foreach $key (sort values %lines) {
$sorted_lines[$i] = $key;
$i++;
}
@sorted_lines;
}
# This is a numerical sort routing (ascending).
sub numsort {
local($i) = 0;
foreach $num (sort {$a <=> $b} keys %lines) {
$sorted_lines[$i] = $lines{$num};
$i++;
}
@sorted_lines;
}
# This is a sort routine that will sort on the
# ip address when the ip address is anywhere in
# the strings.
sub ipsort {
local($i) = 0;
foreach $addr (sort sortbyipaddr keys %lines) {
$sorted_lines[$i] = $lines{$addr};
$i++;
}
@sorted_lines;
}
# These two routines will sort based upon IP addresses
sub ipaddrval {
$a[3]+256*($a[2]+256*($a[1]+256*$a[0]));
}
sub sortbyipaddr {
&ipaddrval($a) <=> &ipaddrval($b);
}
# This routine processes a "show configuration"
# This routine processes a "show configuration"
sub WriteTerm {
print STDERR " In WriteTerm: $_" if ($debug);
#my($lineauto) = 0;
while (<INPUT>) {
#print STDERR "History $_\n";
tr/\015//d;
last if(/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if (/^\s*\^\s*$/);
return(1) if (/Line has invalid autocommand /);
return(1) if (/(Invalid (input|command) detected|Type help or )/i);
return(-1) if (/command authorization failed/i);
# the pager can not be disabled per-session on the PIX
if (/^(<-+ More -+>)/) {
my($len) = length($1);
s/^$1\s{$len}//;
}
/^/ && next;
ProcessHistory("COMMENTS","keysort","B1","# $_") && next;
}
# end of config...is a comment.
if (/^return/i) {
$found_end = 1;
return(1);
}
return(0);
}
# This routine copies everything with precefing "#"
sub CopyComment {
print STDERR " In CopyComment: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
ProcessHistory("","","","# $_");
# end of config
}
return(0);
}
# filter out "uptime" lines
sub FilterUptime {
print STDERR " In FilterUptime: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
#nmeongeza
next if (/^(\s*|\s*$cmd\s*)$/);
return(-1) if (/command authorization failed/i);
/^VERSION\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C1","\#\n# Version: $1\n")
&& next;
/^PATCH\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C2","\# Patch: $1\n")
&& next;
/^PRODUCT\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C3","\# Version: $1\n")
&& next;
}
return(0);
}
# dummy function
sub DoNothing {print STDOUT;}
# Main
@commandtable = (
{'display version' => 'FilterUptime'},
{'display patch-information' => 'CopyComment'},
{'display device' => 'CopyComment'},
{'display device pic-status' => 'CopyComment'},
{'display current-configuration' => 'WriteTerm'},
);
# Use an array to preserve the order of the commands and a hash for mapping
# commands to the subroutine and track commands that have been completed.
@commands = map(keys(%$_), @commandtable);
open(OUTPUT,">$host.new") || die "Can't open $host.new for writing: $!\n";
select(OUTPUT);
# make OUTPUT unbuffered if debugging
if ($debug) { $| = 1; }
if ($file) {
print STDERR "opening file $host\n" if ($debug);
print STDOUT "opening file $host\n" if ($log);
open(INPUT,"<$host") || die "open failed for $host: $!\n";
} else {
print STDERR "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
print STDOUT "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
if (defined($ENV{NOPIPE}) && $ENV{NOPIPE} =~ /^YES/i) {
system "hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null >
$host.raw" || die "hulogin failed for $host: $!\n";
open(INPUT, "< $host.raw") || die "hulogin failed for $host: $!\n";
} else {
open(INPUT,"hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null
|") || die "hulogin failed for $host: $!\n";
}
}
# determine password filtering mode
if ($ENV{"FILTER_PWDS"} =~ /no/i) {
$filter_pwds = 0;
} elsif ($ENV{"FILTER_PWDS"} =~ /all/i) {
$filter_pwds = 2;
} else {
$filter_pwds = 1;
}
ProcessHistory("","","","#RANCID-CONTENT-TYPE: Huawei\n#\n");
ProcessHistory("COMMENTS","keysort","B0","#\n");
ProcessHistory("COMMENTS","keysort","F0","#\n");
ProcessHistory("COMMENTS","keysort","G0","#\n");
TOP: while(<INPUT>) {
tr/\015//d;
#print STDERR ("CMD: $_\n");
if (/\>\s?quit.*$/) {
$clean_run=1;
last;
}
if (/^Error:/) {
print STDOUT ("$host hulogin error: $_");
print STDERR ("$host hulogin error: $_") if ($debug);
$clean_run=0;
last;
}
while (/\>\007*\s*($cmds_regexp)\s*$/) {
$cmd = $1;
if (!defined($prompt)) {
$prompt = ($_ =~ /^([^>]+\>)/)[0];
$prompt =~ s/([][}{)(file://])///$1/g;
print STDERR ("PROMPT MATCH: $prompt\n") if ($debug);
}
print STDERR ("HIT COMMAND:$_") if ($debug);
if (! defined($commands{$cmd})) {
print STDERR "$host: found unexpected command - \"$cmd\"\n";
$clean_run = 0;
last TOP;
}
$rval = &{$commands{$cmd}};
delete($commands{$cmd});
if ($rval == -1) {
$clean_run = 0;
last TOP;
}
}
}
print STDOUT "Done $logincmd: $_\n" if ($log);
# Flush History
ProcessHistory("","","","");
# Cleanup
close(INPUT);
close(OUTPUT);
if (defined($ENV{NOPIPE})) {
#unlink("$host.raw") if (! $debug);
}
# check for completeness
if (scalar(%commands) || !$clean_run || !$found_end) {
if (scalar(%commands)) {
printf(STDOUT "$host: missed cmd(s): %s\n", join(',',
keys(%commands)));
printf(STDERR "$host: missed cmd(s): %s\n", join(',',
keys(%commands))) if ($debug);
}
if (!$clean_run || !$found_end) {
print STDOUT "$host: End of run not found\n";
print STDERR "$host: End of run not found\n" if ($debug);
system("/usr/bin/tail -1 $host.new");
}
#unlink "$host.new" if (! $debug);
}

--
Alan McKinnon
***@gmail.com

Erica James

2013-09-16 14:32:51 UTC

Permalink

Alan,

Thank you . Please see the below,

1. Can telnet to device and authenticate manually

2. /home/eserica/rancid/bin/hulogin -d 10.3.4.5 logs in successfully, and enables properly. Prompt is correct as well

3. /home/eserica/rancid/bin/hulogin -t 90 -c"display current-configuration ; display version" 10.3.4.5 runs cleanly. commands are run neatly. automatic login and logout4.[***@netflow logs]$ /home/eserica/rancid/bin/h3crancid -d 10.3.4.5
executing hulogin -t 90 -c"display version ; display current-configuration" 10.3.4.5
executing hulogin -t 90 -c"display version ; display current-configuration" 10.3.4.5
sh: hulogin: command not found
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: End of run not found
10.3.4.5: End of run not found

The router.db
10.3.4.5:huawei:up

The rancid-fe
    'huawei'            => 'h3crancid',

Don't understand why am getting such an output  from /home/eserica/rancid/bin/h3crancid -d 10.3.4.5
Any comments on the above?

Still the same logs.

Eserica

________________________________
From: Alan McKinnon <***@gmail.com>
To: rancid shrubbery <rancid-***@shrubbery.net>
Sent: Monday, September 16, 2013 4:33 PM
Subject: Re: [rancid] Rancid-missed cmds

The approach to fixing this is always the same, at this point nothing
rancid is outputting helps debug in the slightest - all it says is that
something went wrong, but we already know that.

Do the following:

1. telnet to device and check you can auth manually
2. "hulogin <device>" and check it logs in and enables properly, and
that the prompt etc on the screen is correct
3. run "hulogin -t 90 -c"display current-configuration,display version"
<device>" and check that the commands do actually run correctly
4. run "hurancid -d <device>" - this dumps a *complete* log in the
current directory containing more info than the regular log you quoted
and often reveals the problem (but not always).

But first a few prior steps need to be verified:

- the command in #3 I pasted from your log output, and that has comma
separators. Your device might need semicolons, I have no idea as I don;t
know that vendors stuff
- the relevant line in router.db must contain the appropriate type
string in field 3
- you will have had to edit rancid-fe and add a line for huawei to a
perl hash. Without this, rancid-run doesn't know what parser script to
call and it cannot work.
- the device you have must be supported by the script you are using;
just because Huawei made it is no guarantee it works similarly to
anything else they might have made.

If all of that checks out, then we need to look into the running perl
itself.

I believe we've been over most of this before, I don't recall getting a
definitive answer. You really do need to go through all these steps as
given, nothing else is going to get to the root of your problem.

Post by Erica James
Hello
I have a problem with rancid parsing script. Logs show missed cmd(s).
Trying to get all of the configs.
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: End of run not found
#
All routers sucessfully completed.
cvs diff: Diffing .
cvs diff: Diffing configs
cvs commit: Examining .
cvs commit: Examining configs

The script is for Huawei. The problem now seems to be that rancid is not
collecting configs.
Anyone with an idea on what to do?

Below is the rancid-script

#! /usr/bin/perl
##
## $Id$
##
## rancid 2.3.8
##
## Copyright (C) 1997-2004 by Terrapin Communications, Inc.
## All rights reserved.
##
## This software may be freely copied, modified and redistributed
## without fee for non-commerical purposes provided that this license
## remains intact and unmodified with any RANCID distribution.
##
## There is no warranty or other guarantee of fitness of this software.
## It is provided solely "as is". The author(s) disclaim(s) all
## responsibility and liability with respect to this software's usage
## or its effect upon hardware, computer systems, other software, or
## anything else.
##
## Except where noted otherwise, rancid was written by and is maintained by
## Henry Kilmer, John Heasley, Andrew Partan, Pete Whiting, and Austin Schutz.
##
#
# hurancid - Interface to Huawei devices
#
# RANCID - Really Awesome New Cisco confIg Differ
#
# usage: rancid [-d] [-l] [-f filename | $host]
#
use Getopt::Std;
getopts('dfl');
$log = $opt_l;
$debug = $opt_d;
$file = $opt_f;
$host = $ARGV[0];
$clean_run = 0;
$found_end = 0;
$timeo = 90; # hulogin timeout in seconds
my(%filter_pwds); # password filtering mode
# This routine is used to print out the router configuration
sub ProcessHistory {
if((($new_hist_tag ne $hist_tag) || ($new_command ne $command))
&& defined %history) {
print eval "$command \%history";
undef %history;
}
if (($new_hist_tag) && ($new_command) && ($command_string)) {
if ($history{$command_string}) {
$history{$command_string} =
} else {
}
} elsif (($new_hist_tag) && ($new_command)) {
} else {
}
$hist_tag = $new_hist_tag;
$command = $new_command;
1;
}
sub numerically { $a <=> $b; }
# This is a sort routing that will sort numerically on the
# keys of a hash as if it were a normal array.
sub keynsort {
local($i) = 0;
foreach $key (sort numerically keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
}
# This is a sort routing that will sort on the
# keys of a hash as if it were a normal array.
sub keysort {
local($i) = 0;
foreach $key (sort keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
}
# This is a sort routing that will sort on the
# values of a hash as if it were a normal array.
sub valsort{
local($i) = 0;
foreach $key (sort values %lines) {
$sorted_lines[$i] = $key;
$i++;
}
}
# This is a numerical sort routing (ascending).
sub numsort {
local($i) = 0;
foreach $num (sort {$a <=> $b} keys %lines) {
$sorted_lines[$i] = $lines{$num};
$i++;
}
}
# This is a sort routine that will sort on the
# ip address when the ip address is anywhere in
# the strings.
sub ipsort {
local($i) = 0;
foreach $addr (sort sortbyipaddr keys %lines) {
$sorted_lines[$i] = $lines{$addr};
$i++;
}
}
# These two routines will sort based upon IP addresses
sub ipaddrval {
$a[3]+256*($a[2]+256*($a[1]+256*$a[0]));
}
sub sortbyipaddr {
&ipaddrval($a) <=> &ipaddrval($b);
}

# This routine processes a "show configuration"
# This routine processes a "show configuration"
sub WriteTerm {
print STDERR " In WriteTerm: $_" if ($debug);
#my($lineauto) = 0;
while (<INPUT>) {
#print STDERR "History $_\n";

tr/\015//d;
last if(/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if (/^\s*\^\s*$/);
return(1) if (/Line has invalid autocommand /);
return(1) if (/(Invalid (input|command) detected|Type help or )/i);
return(-1) if (/command authorization failed/i);
# the pager can not be disabled per-session on the PIX
if (/^(<-+ More -+>)/) {
my($len) = length($1);
s/^$1\s{$len}//;
}

/^/ && next;
ProcessHistory("COMMENTS","keysort","B1","# $_") && next;
}

# end of config...is a comment.
if (/^return/i) {
$found_end = 1;
return(1);
}

return(0);
}
# This routine copies everything with precefing "#"
sub CopyComment {
print STDERR " In CopyComment: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
ProcessHistory("","","","# $_");
# end of config
}
return(0);
}
# filter out "uptime" lines
sub FilterUptime {
print STDERR " In FilterUptime: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
#nmeongeza
next if (/^(\s*|\s*$cmd\s*)$/);
return(-1) if (/command authorization failed/i);
/^VERSION\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C1","\#\n# Version: $1\n")
&& next;
/^PATCH\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C2","\# Patch: $1\n")
&& next;
/^PRODUCT\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C3","\# Version: $1\n")
&& next;
}
return(0);
}

# dummy function
sub DoNothing {print STDOUT;}
# Main
@commandtable = (
{'display version' => 'FilterUptime'},
{'display patch-information' => 'CopyComment'},
{'display device' => 'CopyComment'},
{'display device pic-status' => 'CopyComment'},
{'display current-configuration' => 'WriteTerm'},
);
# Use an array to preserve the order of the commands and a hash for mapping
# commands to the subroutine and track commands that have been completed.
@commands = map(keys(%$_), @commandtable);
open(OUTPUT,">$host.new") || die "Can't open $host.new for writing: $!\n";
select(OUTPUT);
# make OUTPUT unbuffered if debugging
if ($debug) { $| = 1; }
if ($file) {
print STDERR "opening file $host\n" if ($debug);
print STDOUT "opening file $host\n" if ($log);
open(INPUT,"<$host") || die "open failed for $host: $!\n";
} else {
print STDERR "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
print STDOUT "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
if (defined($ENV{NOPIPE}) && $ENV{NOPIPE} =~ /^YES/i) {
system "hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null >
$host.raw" || die "hulogin failed for $host: $!\n";
open(INPUT, "< $host.raw") || die "hulogin failed for $host: $!\n";
} else {
open(INPUT,"hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null
|") || die "hulogin failed for $host: $!\n";
}
}
# determine password filtering mode
if ($ENV{"FILTER_PWDS"} =~ /no/i) {
$filter_pwds = 0;
} elsif ($ENV{"FILTER_PWDS"} =~ /all/i) {
$filter_pwds = 2;
} else {
$filter_pwds = 1;
}
ProcessHistory("","","","#RANCID-CONTENT-TYPE: Huawei\n#\n");
ProcessHistory("COMMENTS","keysort","B0","#\n");
ProcessHistory("COMMENTS","keysort","F0","#\n");
ProcessHistory("COMMENTS","keysort","G0","#\n");
TOP: while(<INPUT>) {
tr/\015//d;
#print STDERR ("CMD: $_\n");

if (/\>\s?quit.*$/) {
$clean_run=1;
last;
}

if (/^Error:/) {
print STDOUT ("$host hulogin error: $_");
print STDERR ("$host hulogin error: $_") if ($debug);
$clean_run=0;
last;
}

while (/\>\007*\s*($cmds_regexp)\s*$/) {
$cmd = $1;
if (!defined($prompt)) {
$prompt = ($_ =~ /^([^>]+\>)/)[0];
$prompt =~ s/([][}{)(file://])///$1/g;
print STDERR ("PROMPT MATCH: $prompt\n") if ($debug);
}
print STDERR ("HIT COMMAND:$_") if ($debug);
if (! defined($commands{$cmd})) {
print STDERR "$host: found unexpected command - \"$cmd\"\n";
$clean_run = 0;
last TOP;
}
$rval = &{$commands{$cmd}};
delete($commands{$cmd});
if ($rval == -1) {
$clean_run = 0;
last TOP;
}
}
}
print STDOUT "Done $logincmd: $_\n" if ($log);
# Flush History
ProcessHistory("","","","");
# Cleanup
close(INPUT);
close(OUTPUT);

if (defined($ENV{NOPIPE})) {
#unlink("$host.raw") if (! $debug);
}
# check for completeness
if (scalar(%commands) || !$clean_run || !$found_end) {
if (scalar(%commands)) {
printf(STDOUT "$host: missed cmd(s): %s\n", join(',',
keys(%commands)));
printf(STDERR "$host: missed cmd(s): %s\n", join(',',
keys(%commands))) if ($debug);
}
if (!$clean_run || !$found_end) {
print STDOUT "$host: End of run not found\n";
print STDERR "$host: End of run not found\n" if ($debug);
system("/usr/bin/tail -1 $host.new");
}
#unlink "$host.new" if (! $debug);
}

--
Alan McKinnon
***@gmail.com

Jethro R Binks

2013-09-16 14:57:27 UTC

Permalink

Post by Erica James
sh: hulogin: command not found

h3crancid can't find it. Try replacing with the full path to hulogin in
h3crandid.

Jethro.

Post by Erica James
Alan,

Thank you . Please see the below,

1. Can telnet to device and authenticate manually

2. /home/eserica/rancid/bin/hulogin -d 10.3.4.5 logs in successfully, and enables properly. Prompt is correct as well

executing hulogin -t 90 -c"display version ; display current-configuration" 10.3.4.5
executing hulogin -t 90 -c"display version ; display current-configuration" 10.3.4.5
sh: hulogin: command not found
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: End of run not found
10.3.4.5: End of run not found
The router.db
10.3.4.5:huawei:up

The rancid-fe
    'huawei'            => 'h3crancid',

Don't understand why am getting such an output  from /home/eserica/rancid/bin/h3crancid -d 10.3.4.5
Any comments on the above?

Still the same logs.

Eserica

________________________________
Sent: Monday, September 16, 2013 4:33 PM
Subject: Re: [rancid] Rancid-missed cmds
The approach to fixing this is always the same, at this point nothing
rancid is outputting helps debug in the slightest - all it says is that
something went wrong, but we already know that.
1. telnet to device and check you can auth manually
2. "hulogin <device>" and check it logs in and enables properly, and
that the prompt etc on the screen is correct
3. run "hulogin -t 90 -c"display current-configuration,display version"
<device>" and check that the commands do actually run correctly
4. run "hurancid -d <device>" - this dumps a *complete* log in the
current directory containing more info than the regular log you quoted
and often reveals the problem (but not always).
- the command in #3 I pasted from your log output, and that has comma
separators. Your device might need semicolons, I have no idea as I don;t
know that vendors stuff
- the relevant line in router.db must contain the appropriate type
string in field 3
- you will have had to edit rancid-fe and add a line for huawei to a
perl hash. Without this, rancid-run doesn't know what parser script to
call and it cannot work.
- the device you have must be supported by the script you are using;
just because Huawei made it is no guarantee it works similarly to
anything else they might have made.
If all of that checks out, then we need to look into the running perl
itself.
I believe we've been over most of this before, I don't recall getting a
definitive answer. You really do need to go through all these steps as
given, nothing else is going to get to the root of your problem.

Post by Erica James
Hello
I have a problem with rancid parsing script. Logs show missed cmd(s).
Trying to get all of the configs.
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: End of run not found
#
All routers sucessfully completed.
cvs diff: Diffing .
cvs diff: Diffing configs
cvs commit: Examining .
cvs commit: Examining configs

The script is for Huawei. The problem now seems to be that rancid is not
collecting configs.
Anyone with an idea on what to do?

Below is the rancid-script

#! /usr/bin/perl
##
## $Id$
##
## rancid 2.3.8
##
## Copyright (C) 1997-2004 by Terrapin Communications, Inc.
## All rights reserved.
##
## This software may be freely copied, modified and redistributed
## without fee for non-commerical purposes provided that this license
## remains intact and unmodified with any RANCID distribution.
##
## There is no warranty or other guarantee of fitness of this software.
## It is provided solely "as is". The author(s) disclaim(s) all
## responsibility and liability with respect to this software's usage
## or its effect upon hardware, computer systems, other software, or
## anything else.
##
## Except where noted otherwise, rancid was written by and is maintained by
## Henry Kilmer, John Heasley, Andrew Partan, Pete Whiting, and Austin Schutz.
##
#
# hurancid - Interface to Huawei devices
#
# RANCID - Really Awesome New Cisco confIg Differ
#
# usage: rancid [-d] [-l] [-f filename | $host]
#
use Getopt::Std;
getopts('dfl');
$log = $opt_l;
$debug = $opt_d;
$file = $opt_f;
$host = $ARGV[0];
$clean_run = 0;
$found_end = 0;
$timeo = 90; # hulogin timeout in seconds
my(%filter_pwds); # password filtering mode
# This routine is used to print out the router configuration
sub ProcessHistory {
if((($new_hist_tag ne $hist_tag) || ($new_command ne $command))
&& defined %history) {
print eval "$command \%history";
undef %history;
}
if (($new_hist_tag) && ($new_command) && ($command_string)) {
if ($history{$command_string}) {
$history{$command_string} =
} else {
}
} elsif (($new_hist_tag) && ($new_command)) {
} else {
}
$hist_tag = $new_hist_tag;
$command = $new_command;
1;
}
sub numerically { $a <=> $b; }
# This is a sort routing that will sort numerically on the
# keys of a hash as if it were a normal array.
sub keynsort {
local($i) = 0;
foreach $key (sort numerically keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
}
# This is a sort routing that will sort on the
# keys of a hash as if it were a normal array.
sub keysort {
local($i) = 0;
foreach $key (sort keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
}
# This is a sort routing that will sort on the
# values of a hash as if it were a normal array.
sub valsort{
local($i) = 0;
foreach $key (sort values %lines) {
$sorted_lines[$i] = $key;
$i++;
}
}
# This is a numerical sort routing (ascending).
sub numsort {
local($i) = 0;
foreach $num (sort {$a <=> $b} keys %lines) {
$sorted_lines[$i] = $lines{$num};
$i++;
}
}
# This is a sort routine that will sort on the
# ip address when the ip address is anywhere in
# the strings.
sub ipsort {
local($i) = 0;
foreach $addr (sort sortbyipaddr keys %lines) {
$sorted_lines[$i] = $lines{$addr};
$i++;
}
}
# These two routines will sort based upon IP addresses
sub ipaddrval {
$a[3]+256*($a[2]+256*($a[1]+256*$a[0]));
}
sub sortbyipaddr {
&ipaddrval($a) <=> &ipaddrval($b);
}

# This routine processes a "show configuration"
# This routine processes a "show configuration"
sub WriteTerm {
print STDERR " In WriteTerm: $_" if ($debug);
#my($lineauto) = 0;
while (<INPUT>) {
#print STDERR "History $_\n";

tr/\015//d;
last if(/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if (/^\s*\^\s*$/);
return(1) if (/Line has invalid autocommand /);
return(1) if (/(Invalid (input|command) detected|Type help or )/i);
return(-1) if (/command authorization failed/i);
# the pager can not be disabled per-session on the PIX
if (/^(<-+ More -+>)/) {
my($len) = length($1);
s/^$1\s{$len}//;
}

/^/ && next;
ProcessHistory("COMMENTS","keysort","B1","# $_") && next;
}

# end of config...is a comment.
if (/^return/i) {
$found_end = 1;
return(1);
}

return(0);
}
# This routine copies everything with precefing "#"
sub CopyComment {
print STDERR " In CopyComment: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
ProcessHistory("","","","# $_");
# end of config
}
return(0);
}
# filter out "uptime" lines
sub FilterUptime {
print STDERR " In FilterUptime: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
#nmeongeza
next if (/^(\s*|\s*$cmd\s*)$/);
return(-1) if (/command authorization failed/i);
/^VERSION\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C1","\#\n# Version: $1\n")
&& next;
/^PATCH\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C2","\# Patch: $1\n")
&& next;
/^PRODUCT\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C3","\# Version: $1\n")
&& next;
}
return(0);
}

# dummy function
sub DoNothing {print STDOUT;}
# Main
@commandtable = (
{'display version' => 'FilterUptime'},
{'display patch-information' => 'CopyComment'},
{'display device' => 'CopyComment'},
{'display device pic-status' => 'CopyComment'},
{'display current-configuration' => 'WriteTerm'},
);
# Use an array to preserve the order of the commands and a hash for mapping
# commands to the subroutine and track commands that have been completed.
@commands = map(keys(%$_), @commandtable);
open(OUTPUT,">$host.new") || die "Can't open $host.new for writing: $!\n";
select(OUTPUT);
# make OUTPUT unbuffered if debugging
if ($debug) { $| = 1; }
if ($file) {
print STDERR "opening file $host\n" if ($debug);
print STDOUT "opening file $host\n" if ($log);
open(INPUT,"<$host") || die "open failed for $host: $!\n";
} else {
print STDERR "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
print STDOUT "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
if (defined($ENV{NOPIPE}) && $ENV{NOPIPE} =~ /^YES/i) {
system "hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null >
$host.raw" || die "hulogin failed for $host: $!\n";
open(INPUT, "< $host.raw") || die "hulogin failed for $host: $!\n";
} else {
open(INPUT,"hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null
|") || die "hulogin failed for $host: $!\n";
}
}
# determine password filtering mode
if ($ENV{"FILTER_PWDS"} =~ /no/i) {
$filter_pwds = 0;
} elsif ($ENV{"FILTER_PWDS"} =~ /all/i) {
$filter_pwds = 2;
} else {
$filter_pwds = 1;
}
ProcessHistory("","","","#RANCID-CONTENT-TYPE: Huawei\n#\n");
ProcessHistory("COMMENTS","keysort","B0","#\n");
ProcessHistory("COMMENTS","keysort","F0","#\n");
ProcessHistory("COMMENTS","keysort","G0","#\n");
TOP: while(<INPUT>) {
tr/\015//d;
#print STDERR ("CMD: $_\n");

if (/\>\s?quit.*$/) {
$clean_run=1;
last;
}

if (/^Error:/) {
print STDOUT ("$host hulogin error: $_");
print STDERR ("$host hulogin error: $_") if ($debug);
$clean_run=0;
last;
}

while (/\>\007*\s*($cmds_regexp)\s*$/) {
$cmd = $1;
if (!defined($prompt)) {
$prompt = ($_ =~ /^([^>]+\>)/)[0];
$prompt =~ s/([][}{)(file://])///$1/g;
print STDERR ("PROMPT MATCH: $prompt\n") if ($debug);
}
print STDERR ("HIT COMMAND:$_") if ($debug);
if (! defined($commands{$cmd})) {
print STDERR "$host: found unexpected command - \"$cmd\"\n";
$clean_run = 0;
last TOP;
}
$rval = &{$commands{$cmd}};
delete($commands{$cmd});
if ($rval == -1) {
$clean_run = 0;
last TOP;
}
}
}
print STDOUT "Done $logincmd: $_\n" if ($log);
# Flush History
ProcessHistory("","","","");
# Cleanup
close(INPUT);
close(OUTPUT);

if (defined($ENV{NOPIPE})) {
#unlink("$host.raw") if (! $debug);
}
# check for completeness
if (scalar(%commands) || !$clean_run || !$found_end) {
if (scalar(%commands)) {
printf(STDOUT "$host: missed cmd(s): %s\n", join(',',
keys(%commands)));
printf(STDERR "$host: missed cmd(s): %s\n", join(',',
keys(%commands))) if ($debug);
}
if (!$clean_run || !$found_end) {
print STDOUT "$host: End of run not found\n";
print STDERR "$host: End of run not found\n" if ($debug);
system("/usr/bin/tail -1 $host.new");
}
#unlink "$host.new" if (! $debug);
}

--
Alan McKinnon
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

. . . . . . . . . . . . . . . . . . . . . . . . .
Jethro R Binks, Network Manager,
Information Services Directorate, University Of Strathclyde, Glasgow, UK

The University of Strathclyde is a charitable body, registered in
Scotland, number SC015263.

Roy

2013-09-16 14:53:39 UTC

Permalink

<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">"sh: hulogin: command not found"<br>
<br>
To run in debug mode, you need to add "/home/eserica/rancid/bin"
to your path<br>
<br>
On 9/16/2013 7:32 AM, Erica James wrote:<br>
</div>
<blockquote
cite="mid:***@web160806.mail.bf1.yahoo.com"
type="cite">
<div style="color:#000; background-color:#fff; font-family:times
new roman, new york, times, serif;font-size:12pt">
<div><span>Alan,</span></div>
<div><span><var id="yui-ie-cursor"></var></span> </div>
<div><span>Thank you . Please see the below,</span></div>
<div><span></span> </div>
<div><span>1. Can telnet to device and authenticate manually</span></div>
<div><span></span> </div>
<div><span>2. /home/eserica/rancid/bin/hulogin -d 10.3.4.5 logs
in successfully, and enables properly. Prompt is correct as
well</span></div>
<div><span></span> </div>
<div><span>3. /home/eserica/rancid/bin/hulogin -t 90 -c"display
current-configuration ; display version" 10.3.4.5 runs
cleanly. commands are run neatly. automatic login and logout</span></div>
<span>
<div>4.[***@netflow logs]$
/home/eserica/rancid/bin/h3crancid -d 10.3.4.5<br>
executing hulogin -t 90 -c"display version ; display
current-configuration" 10.3.4.5<br>
executing hulogin -t 90 -c"display version ; display
current-configuration" 10.3.4.5<br>
sh: hulogin: command not found<br>
10.3.4.5: missed cmd(s): display
current-configuration,display version<br>
10.3.4.5: missed cmd(s): display
current-configuration,display version<br>
10.3.4.5: End of run not found<br>
10.3.4.5: End of run not found<br>
</div>
<div>The router.db</div>
<div>10.3.4.5:huawei:up</div>
<div> </div>
<div>The rancid-fe <br>
    'huawei'            => 'h3crancid',<br>
  </div>
<div>Don't understand why am getting such an output  from
/home/eserica/rancid/bin/h3crancid -d 10.3.4.5</div>
<div>Any comments on the above?</div>
<div> </div>
<div>Still the same logs.</div>
<div> </div>
<div>Eserica</div>
<div> </div>
</span><br>
</div>
</blockquote>
<br>
</body>
</html>

Alan McKinnon

2013-09-16 19:37:06 UTC

Permalink

^^^^^^^^^^^^^^^^^^^^^^^^^^

You almost certainly do not have "/home/eserica/rancid/bin/" in rancid's
PATH. It is set in $RANCID_DIR/etc/rancid.conf.

As to why this happened, it is very likely an installation failure.

Did you install all of rancid to /home/eserica/rancid, or is that just a
convenient place to put rancid scripts you are testing?

The rancid build system works almost perfectly every time. It uses
autotools and is designed to install the entire package to
/usr/local/rancid/ by default and for the scripts to be run by user
"rancid". As with all Unix apps it is sensitive to ownership and
permission errors

The errors you are getting indicate a faulty install, possibly trying to
do itmanually.

Post by Erica James
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: End of run not found
10.3.4.5: End of run not found
The router.db
10.3.4.5:huawei:up
The rancid-fe
'huawei' => 'h3crancid',
Don't understand why am getting such an output from
/home/eserica/rancid/bin/h3crancid -d 10.3.4.5
Any comments on the above?
Still the same logs.
Eserica
*Sent:* Monday, September 16, 2013 4:33 PM
*Subject:* Re: [rancid] Rancid-missed cmds
The approach to fixing this is always the same, at this point nothing
rancid is outputting helps debug in the slightest - all it says is that
something went wrong, but we already know that.
1. telnet to device and check you can auth manually
2. "hulogin <device>" and check it logs in and enables properly, and
that the prompt etc on the screen is correct
3. run "hulogin -t 90 -c"display current-configuration,display version"
<device>" and check that the commands do actually run correctly
4. run "hurancid -d <device>" - this dumps a *complete* log in the
current directory containing more info than the regular log you quoted
and often reveals the problem (but not always).
- the command in #3 I pasted from your log output, and that has comma
separators. Your device might need semicolons, I have no idea as I don;t
know that vendors stuff
- the relevant line in router.db must contain the appropriate type
string in field 3
- you will have had to edit rancid-fe and add a line for huawei to a
perl hash. Without this, rancid-run doesn't know what parser script to
call and it cannot work.
- the device you have must be supported by the script you are using;
just because Huawei made it is no guarantee it works similarly to
anything else they might have made.
If all of that checks out, then we need to look into the running perl
itself.
I believe we've been over most of this before, I don't recall getting a
definitive answer. You really do need to go through all these steps as
given, nothing else is going to get to the root of your problem.

maintained by

Post by Erica James
## Henry Kilmer, John Heasley, Andrew Partan, Pete Whiting, and Austin Schutz.
##
#
# hurancid - Interface to Huawei devices
#
# RANCID - Really Awesome New Cisco confIg Differ
#
# usage: rancid [-d] [-l] [-f filename | $host]
#
use Getopt::Std;
getopts('dfl');
$log = $opt_l;
$debug = $opt_d;
$file = $opt_f;
$host = $ARGV[0];
$clean_run = 0;
$found_end = 0;
$timeo = 90; # hulogin timeout in seconds
my(%filter_pwds); # password filtering mode
# This routine is used to print out the router configuration
sub ProcessHistory {
if((($new_hist_tag ne $hist_tag) || ($new_command ne $command))
&& defined %history) {
print eval "$command \%history";
undef %history;
}
if (($new_hist_tag) && ($new_command) && ($command_string)) {
if ($history{$command_string}) {
$history{$command_string} =
} else {
}
} elsif (($new_hist_tag) && ($new_command)) {
} else {
}
$hist_tag = $new_hist_tag;
$command = $new_command;
1;
}
sub numerically { $a <=> $b; }
# This is a sort routing that will sort numerically on the
# keys of a hash as if it were a normal array.
sub keynsort {
local($i) = 0;
foreach $key (sort numerically keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
@sorted_lines;
}
# This is a sort routing that will sort on the
# keys of a hash as if it were a normal array.
sub keysort {
local($i) = 0;
foreach $key (sort keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
@sorted_lines;
}
# This is a sort routing that will sort on the
# values of a hash as if it were a normal array.
sub valsort{
local($i) = 0;
foreach $key (sort values %lines) {
$sorted_lines[$i] = $key;
$i++;
}
@sorted_lines;
}
# This is a numerical sort routing (ascending).
sub numsort {
local($i) = 0;
foreach $num (sort {$a <=> $b} keys %lines) {
$sorted_lines[$i] = $lines{$num};
$i++;
}
@sorted_lines;
}
# This is a sort routine that will sort on the
# ip address when the ip address is anywhere in
# the strings.
sub ipsort {
local($i) = 0;
foreach $addr (sort sortbyipaddr keys %lines) {
$sorted_lines[$i] = $lines{$addr};
$i++;
}
@sorted_lines;
}
# These two routines will sort based upon IP addresses
sub ipaddrval {
$a[3]+256*($a[2]+256*($a[1]+256*$a[0]));
}
sub sortbyipaddr {
&ipaddrval($a) <=> &ipaddrval($b);
}
# This routine processes a "show configuration"
# This routine processes a "show configuration"
sub WriteTerm {
print STDERR " In WriteTerm: $_" if ($debug);
#my($lineauto) = 0;
while (<INPUT>) {
#print STDERR "History $_\n";
tr/\015//d;
last if(/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if (/^\s*\^\s*$/);
return(1) if (/Line has invalid autocommand /);
return(1) if (/(Invalid (input|command) detected|Type help or )/i);
return(-1) if (/command authorization failed/i);
# the pager can not be disabled per-session on the PIX
if (/^(<-+ More -+>)/) {
my($len) = length($1);
s/^$1\s{$len}//;
}
/^/ && next;
ProcessHistory("COMMENTS","keysort","B1","# $_") && next;
}
# end of config...is a comment.
if (/^return/i) {
$found_end = 1;
return(1);
}
return(0);
}
# This routine copies everything with precefing "#"
sub CopyComment {
print STDERR " In CopyComment: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
ProcessHistory("","","","# $_");
# end of config
}
return(0);
}
# filter out "uptime" lines
sub FilterUptime {
print STDERR " In FilterUptime: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
#nmeongeza
next if (/^(\s*|\s*$cmd\s*)$/);
return(-1) if (/command authorization failed/i);
/^VERSION\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C1","\#\n# Version: $1\n")
&& next;
/^PATCH\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C2","\# Patch: $1\n")
&& next;
/^PRODUCT\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C3","\# Version: $1\n")
&& next;
}
return(0);
}
# dummy function
sub DoNothing {print STDOUT;}
# Main
@commandtable = (
{'display version' => 'FilterUptime'},
{'display patch-information' => 'CopyComment'},
{'display device' => 'CopyComment'},
{'display device pic-status' => 'CopyComment'},
{'display current-configuration' => 'WriteTerm'},
);
# Use an array to preserve the order of the commands and a hash for

mapping

Post by Erica James
# commands to the subroutine and track commands that have been completed.
@commands = map(keys(%$_), @commandtable);
open(OUTPUT,">$host.new") || die "Can't open $host.new for writing: $!\n";
select(OUTPUT);
# make OUTPUT unbuffered if debugging
if ($debug) { $| = 1; }
if ($file) {
print STDERR "opening file $host\n" if ($debug);
print STDOUT "opening file $host\n" if ($log);
open(INPUT,"<$host") || die "open failed for $host: $!\n";
} else {
print STDERR "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
print STDOUT "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
if (defined($ENV{NOPIPE}) && $ENV{NOPIPE} =~ /^YES/i) {
system "hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null >
$host.raw" || die "hulogin failed for $host: $!\n";
open(INPUT, "< $host.raw") || die "hulogin failed for $host: $!\n";
} else {
open(INPUT,"hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null
|") || die "hulogin failed for $host: $!\n";
}
}
# determine password filtering mode
if ($ENV{"FILTER_PWDS"} =~ /no/i) {
$filter_pwds = 0;
} elsif ($ENV{"FILTER_PWDS"} =~ /all/i) {
$filter_pwds = 2;
} else {
$filter_pwds = 1;
}
ProcessHistory("","","","#RANCID-CONTENT-TYPE: Huawei\n#\n");
ProcessHistory("COMMENTS","keysort","B0","#\n");
ProcessHistory("COMMENTS","keysort","F0","#\n");
ProcessHistory("COMMENTS","keysort","G0","#\n");
TOP: while(<INPUT>) {
tr/\015//d;
#print STDERR ("CMD: $_\n");
if (/\>\s?quit.*$/) {
$clean_run=1;
last;
}
if (/^Error:/) {
print STDOUT ("$host hulogin error: $_");
print STDERR ("$host hulogin error: $_") if ($debug);
$clean_run=0;
last;
}
while (/\>\007*\s*($cmds_regexp)\s*$/) {
$cmd = $1;
if (!defined($prompt)) {
$prompt = ($_ =~ /^([^>]+\>)/)[0];
$prompt =~ s/([][}{)(file://])///$1/g;
print STDERR ("PROMPT MATCH: $prompt\n") if ($debug);
}
print STDERR ("HIT COMMAND:$_") if ($debug);
if (! defined($commands{$cmd})) {
print STDERR "$host: found unexpected command - \"$cmd\"\n";
$clean_run = 0;
last TOP;
}
$rval = &{$commands{$cmd}};
delete($commands{$cmd});
if ($rval == -1) {
$clean_run = 0;
last TOP;
}
}
}
print STDOUT "Done $logincmd: $_\n" if ($log);
# Flush History
ProcessHistory("","","","");
# Cleanup
close(INPUT);
close(OUTPUT);
if (defined($ENV{NOPIPE})) {
#unlink("$host.raw") if (! $debug);
}
# check for completeness
if (scalar(%commands) || !$clean_run || !$found_end) {
if (scalar(%commands)) {
printf(STDOUT "$host: missed cmd(s): %s\n", join(',',
keys(%commands)));
printf(STDERR "$host: missed cmd(s): %s\n", join(',',
keys(%commands))) if ($debug);
}
if (!$clean_run || !$found_end) {
print STDOUT "$host: End of run not found\n";
print STDERR "$host: End of run not found\n" if ($debug);
system("/usr/bin/tail -1 $host.new");
}
#unlink "$host.new" if (! $debug);
}

--
Alan McKinnon
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

--
Alan McKinnon
***@gmail.com

Erica James

2013-09-17 06:08:03 UTC

Permalink

Hello.
I have checked in rancid's PATH.
this is a portion of the rancid.conf file. BASEDIR=/home/eserica/rancid/var/rancid; export BASEDIR
PATH=/home/eserica/rancid/bin:/usr/bin:.:/bin:/usr/kerberos/bin:/usr/local/bin:/usr/bin; export PATH

________________________________
From: Alan McKinnon <***@gmail.com>
To: Erica James <***@yahoo.com>
Cc: rancid shrubbery <rancid-***@shrubbery.net>
Sent: Monday, September 16, 2013 10:37 PM
Subject: Re: [rancid] Rancid-missed cmds

Post by Erica James
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: missed cmd(s): display current-configuration,display version
10.3.4.5: End of run not found
10.3.4.5: End of run not found
The router.db
10.3.4.5:huawei:up

The rancid-fe
'huawei' => 'h3crancid',

Don't understand why am getting such an output from
/home/eserica/rancid/bin/h3crancid -d 10.3.4.5
Any comments on the above?

Still the same logs.

Eserica

*Sent:* Monday, September 16, 2013 4:33 PM
*Subject:* Re: [rancid] Rancid-missed cmds
The approach to fixing this is always the same, at this point nothing
rancid is outputting helps debug in the slightest - all it says is that
something went wrong, but we already know that.
1. telnet to device and check you can auth manually
2. "hulogin <device>" and check it logs in and enables properly, and
that the prompt etc on the screen is correct
3. run "hulogin -t 90 -c"display current-configuration,display version"
<device>" and check that the commands do actually run correctly
4. run "hurancid -d <device>" - this dumps a *complete* log in the
current directory containing more info than the regular log you quoted
and often reveals the problem (but not always).
- the command in #3 I pasted from your log output, and that has comma
separators. Your device might need semicolons, I have no idea as I don;t
know that vendors stuff
- the relevant line in router.db must contain the appropriate type
string in field 3
- you will have had to edit rancid-fe and add a line for huawei to a
perl hash. Without this, rancid-run doesn't know what parser script to
call and it cannot work.
- the device you have must be supported by the script you are using;
just because Huawei made it is no guarantee it works similarly to
anything else they might have made.
If all of that checks out, then we need to look into the running perl
itself.
I believe we've been over most of this before, I don't recall getting a
definitive answer. You really do need to go through all these steps as
given, nothing else is going to get to the root of your problem.

maintained by

Post by Erica James
## Henry Kilmer, John Heasley, Andrew Partan, Pete Whiting, and Austin Schutz.
##
#
# hurancid - Interface to Huawei devices
#
# RANCID - Really Awesome New Cisco confIg Differ
#
# usage: rancid [-d] [-l] [-f filename | $host]
#
use Getopt::Std;
getopts('dfl');
$log = $opt_l;
$debug = $opt_d;
$file = $opt_f;
$host = $ARGV[0];
$clean_run = 0;
$found_end = 0;
$timeo = 90; # hulogin timeout in seconds
my(%filter_pwds); # password filtering mode
# This routine is used to print out the router configuration
sub ProcessHistory {
if((($new_hist_tag ne $hist_tag) || ($new_command ne $command))
&& defined %history) {
print eval "$command \%history";
undef %history;
}
if (($new_hist_tag) && ($new_command) && ($command_string)) {
if ($history{$command_string}) {
$history{$command_string} =
} else {
}
} elsif (($new_hist_tag) && ($new_command)) {
} else {
}
$hist_tag = $new_hist_tag;
$command = $new_command;
1;
}
sub numerically { $a <=> $b; }
# This is a sort routing that will sort numerically on the
# keys of a hash as if it were a normal array.
sub keynsort {
local($i) = 0;
foreach $key (sort numerically keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
}
# This is a sort routing that will sort on the
# keys of a hash as if it were a normal array.
sub keysort {
local($i) = 0;
foreach $key (sort keys(%lines)) {
$sorted_lines[$i] = $lines{$key};
$i++;
}
}
# This is a sort routing that will sort on the
# values of a hash as if it were a normal array.
sub valsort{
local($i) = 0;
foreach $key (sort values %lines) {
$sorted_lines[$i] = $key;
$i++;
}
}
# This is a numerical sort routing (ascending).
sub numsort {
local($i) = 0;
foreach $num (sort {$a <=> $b} keys %lines) {
$sorted_lines[$i] = $lines{$num};
$i++;
}
}
# This is a sort routine that will sort on the
# ip address when the ip address is anywhere in
# the strings.
sub ipsort {
local($i) = 0;
foreach $addr (sort sortbyipaddr keys %lines) {
$sorted_lines[$i] = $lines{$addr};
$i++;
}
}
# These two routines will sort based upon IP addresses
sub ipaddrval {
$a[3]+256*($a[2]+256*($a[1]+256*$a[0]));
}
sub sortbyipaddr {
&ipaddrval($a) <=> &ipaddrval($b);
}
# This routine processes a "show configuration"
# This routine processes a "show configuration"
sub WriteTerm {
print STDERR " In WriteTerm: $_" if ($debug);
#my($lineauto) = 0;
while (<INPUT>) {
#print STDERR "History $_\n";

tr/\015//d;
last if(/^$prompt/);
next if (/^(\s*|\s*$cmd\s*)$/);
return(1) if (/^\s*\^\s*$/);
return(1) if (/Line has invalid autocommand /);
return(1) if (/(Invalid (input|command) detected|Type help or )/i);
return(-1) if (/command authorization failed/i);
# the pager can not be disabled per-session on the PIX
if (/^(<-+ More -+>)/) {
my($len) = length($1);
s/^$1\s{$len}//;
}

/^/ && next;
ProcessHistory("COMMENTS","keysort","B1","# $_") && next;
}
# end of config...is a comment.
if (/^return/i) {
$found_end = 1;
return(1);
}

return(0);
}
# This routine copies everything with precefing "#"
sub CopyComment {
print STDERR " In CopyComment: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
ProcessHistory("","","","# $_");
# end of config
}
return(0);
}
# filter out "uptime" lines
sub FilterUptime {
print STDERR " In FilterUptime: $_" if ($debug);
while (<INPUT>) {
tr/\015//d;
last if (/^$prompt/);
#nmeongeza
next if (/^(\s*|\s*$cmd\s*)$/);
return(-1) if (/command authorization failed/i);
/^VERSION\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C1","\#\n# Version: $1\n")
&& next;
/^PATCH\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C2","\# Patch: $1\n")
&& next;
/^PRODUCT\=>(\s+.*)$/ &&
ProcessHistory("COMMENTS","keysort","C3","\# Version: $1\n")
&& next;
}
return(0);
}
# dummy function
sub DoNothing {print STDOUT;}
# Main
@commandtable = (
{'display version' => 'FilterUptime'},
{'display patch-information' => 'CopyComment'},
{'display device' => 'CopyComment'},
{'display device pic-status' => 'CopyComment'},
{'display current-configuration' => 'WriteTerm'},
);
# Use an array to preserve the order of the commands and a hash for

mapping

Post by Erica James
# commands to the subroutine and track commands that have been completed.
@commands = map(keys(%$_), @commandtable);
open(OUTPUT,">$host.new") || die "Can't open $host.new for writing: $!\n";
select(OUTPUT);
# make OUTPUT unbuffered if debugging
if ($debug) { $| = 1; }
if ($file) {
print STDERR "opening file $host\n" if ($debug);
print STDOUT "opening file $host\n" if ($log);
open(INPUT,"<$host") || die "open failed for $host: $!\n";
} else {
print STDERR "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
print STDOUT "executing hulogin -t $timeo -c\"$huaw_cmds\" $host\n"
if ($debug);
if (defined($ENV{NOPIPE}) && $ENV{NOPIPE} =~ /^YES/i) {
system "hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null >
$host.raw" || die "hulogin failed for $host: $!\n";
open(INPUT, "< $host.raw") || die "hulogin failed for $host: $!\n";
} else {
open(INPUT,"hulogin -t $timeo -c \"$huaw_cmds\" $host </dev/null
|") || die "hulogin failed for $host: $!\n";
}
}
# determine password filtering mode
if ($ENV{"FILTER_PWDS"} =~ /no/i) {
$filter_pwds = 0;
} elsif ($ENV{"FILTER_PWDS"} =~ /all/i) {
$filter_pwds = 2;
} else {
$filter_pwds = 1;
}
ProcessHistory("","","","#RANCID-CONTENT-TYPE: Huawei\n#\n");
ProcessHistory("COMMENTS","keysort","B0","#\n");
ProcessHistory("COMMENTS","keysort","F0","#\n");
ProcessHistory("COMMENTS","keysort","G0","#\n");
TOP: while(<INPUT>) {
tr/\015//d;
#print STDERR ("CMD: $_\n");

if (/\>\s?quit.*$/) {
$clean_run=1;
last;
}
if (/^Error:/) {
print STDOUT ("$host hulogin error: $_");
print STDERR ("$host hulogin error: $_") if ($debug);
$clean_run=0;
last;
}

while (/\>\007*\s*($cmds_regexp)\s*$/) {
$cmd = $1;
if (!defined($prompt)) {
$prompt = ($_ =~ /^([^>]+\>)/)[0];
$prompt =~ s/([][}{)(file://])///$1/g;
print STDERR ("PROMPT MATCH: $prompt\n") if ($debug);
}
print STDERR ("HIT COMMAND:$_") if ($debug);
if (! defined($commands{$cmd})) {
print STDERR "$host: found unexpected command - \"$cmd\"\n";
$clean_run = 0;
last TOP;
}
$rval = &{$commands{$cmd}};
delete($commands{$cmd});
if ($rval == -1) {
$clean_run = 0;
last TOP;
}
}
}
print STDOUT "Done $logincmd: $_\n" if ($log);
# Flush History
ProcessHistory("","","","");
# Cleanup
close(INPUT);
close(OUTPUT);
if (defined($ENV{NOPIPE})) {
#unlink("$host.raw") if (! $debug);
}
# check for completeness
if (scalar(%commands) || !$clean_run || !$found_end) {
if (scalar(%commands)) {
printf(STDOUT "$host: missed cmd(s): %s\n", join(',',
keys(%commands)));
printf(STDERR "$host: missed cmd(s): %s\n", join(',',
keys(%commands))) if ($debug);
}
if (!$clean_run || !$found_end) {
print STDOUT "$host: End of run not found\n";
print STDERR "$host: End of run not found\n" if ($debug);
system("/usr/bin/tail -1 $host.new");
}
#unlink "$host.new" if (! $debug);
}

--
Alan McKinnon
_______________________________________________
Rancid-discuss mailing list
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

--
Alan McKinnon
***@gmail.com

Alan McKinnon

2013-09-17 08:19:40 UTC

Permalink

Ah, I see it now. The PATH is set in rancid.conf correctly so it is
available when run from cron.

You now need to add it to *your* path so it's available when you run the
command on the cli. Add it in your .bashrc or .profile as appropriate,
then log out, log in and run the test commands again. If rancid also
runs as a different user yu should add the same to that users
environment too.

These steps won't affect how rancid runs on a cron schedule, it will
only affect cli commands so our debugging efforts work

Post by Erica James
Hello.
I have checked in rancid's PATH.
this is a portion of the rancid.conf file.
BASEDIR=/home/eserica/rancid/var/rancid; export BASEDIR
PATH=/home/eserica/rancid/bin:/usr/bin:.:/bin:/usr/kerberos/bin:/usr/local/bin:/usr/bin;
export PATH
*Sent:* Monday, September 16, 2013 10:37 PM
*Subject:* Re: [rancid] Rancid-missed cmds

/home/eserica/rancid/bin/h3crancid -d 10.3.4.5

Post by Erica James
executing hulogin -t 90 -c"display version ; display
current-configuration" 10.3.4.5
executing hulogin -t 90 -c"display version ; display
current-configuration" 10.3.4.5
sh: hulogin: command not found

^^^^^^^^^^^^^^^^^^^^^^^^^^
You almost certainly do not have "/home/eserica/rancid/bin/" in rancid's
PATH. It is set in $RANCID_DIR/etc/rancid.conf.
As to why this happened, it is very likely an installation failure.
Did you install all of rancid to /home/eserica/rancid, or is that just a
convenient place to put rancid scripts you are testing?
The rancid build system works almost perfectly every time. It uses
autotools and is designed to install the entire package to
/usr/local/rancid/ by default and for the scripts to be run by user
"rancid". As with all Unix apps it is sensitive to ownership and
permission errors
The errors you are getting indicate a faulty install, possibly trying to
do itmanually.

maintained by

)/i);