[rancid] Fortigate additional tweaks and device filters

[Chris Wopat]

Hi Heasley and folks,

Sept 2017 i sent a note in with some proposed tweaks to a Fortigate. to
filter out some additional chattiness, see:

http://www.shrubbery.net/pipermail/rancid-discuss/2017-September/009871.html
http://www.shrubbery.net/pipermail/rancid-discuss/2017-June/009643.html

A few people chimed in seeming to be OK with the propsed changes, which are
to filter these things:

next if (/^\s*IPS-ETDB: .*/);
next if (/^\s*APP-DB: .*/);
next if (/^\s*IPS Malicious URL Database: .*/);
next if (/^\s*Botnet DB: .*/);

Mentioning this as 3.8 came out and i didn't notice any of these included.

We have an additional fortigate tweak we make every time we update too,
which to change from 'show full-configuration' to just 'show' in
@commandtable. 'full-configuration' shows default config, just like the
cisco 'full' command. It's really not necessary IMO.

Cheers,
Chris

[heasley]

Fri, Jul 27, 2018 at 08:02:28AM -0500, Chris Wopat:
> Hi Heasley and folks,
>
> Sept 2017 i sent a note in with some proposed tweaks to a Fortigate. to
> filter out some additional chattiness, see:
>
> http://www.shrubbery.net/pipermail/rancid-discuss/2017-September/009871.html
> http://www.shrubbery.net/pipermail/rancid-discuss/2017-June/009643.html
>
> A few people chimed in seeming to be OK with the propsed changes, which are
> to filter these things:
>
> next if (/^\s*IPS-ETDB: .*/);
> next if (/^\s*APP-DB: .*/);
> next if (/^\s*IPS Malicious URL Database: .*/);
> next if (/^\s*Botnet DB: .*/);
>
> Mentioning this as 3.8 came out and i didn't notice any of these included.
>
> We have an additional fortigate tweak we make every time we update too,
> which to change from 'show full-configuration' to just 'show' in
> @commandtable. 'full-configuration' shows default config, just like the
> cisco 'full' command. It's really not necessary IMO.

This is from:
r2258 | heas | 2010-10-11 20:49:05 +0000 (Mon, 11 Oct 2010) | 3 lines

fnrancid: update recent fortinet software - Diego Ercolani
Cleaned-up a little by me.

afaict, the justification for full-configuration was so that VDOMs would
be included in the output. perhaps this behavior has changed since this
change?? I have none of these devices.

[Doug Hughes]

On 7/31/2018 5:14 PM, heasley wrote:
> Fri, Jul 27, 2018 at 08:02:28AM -0500, Chris Wopat:
>> Hi Heasley and folks,
>>
>> Sept 2017 i sent a note in with some proposed tweaks to a Fortigate. to
>> filter out some additional chattiness, see:
>>
>> http://www.shrubbery.net/pipermail/rancid-discuss/2017-September/009871.html
>> http://www.shrubbery.net/pipermail/rancid-discuss/2017-June/009643.html
>>
>> A few people chimed in seeming to be OK with the propsed changes, which are
>> to filter these things:
>>
>> next if (/^\s*IPS-ETDB: .*/);
>> next if (/^\s*APP-DB: .*/);
>> next if (/^\s*IPS Malicious URL Database: .*/);
>> next if (/^\s*Botnet DB: .*/);
>>
>> Mentioning this as 3.8 came out and i didn't notice any of these included.
>>
>> We have an additional fortigate tweak we make every time we update too,
>> which to change from 'show full-configuration' to just 'show' in
>> @commandtable. 'full-configuration' shows default config, just like the
>> cisco 'full' command. It's really not necessary IMO.
> This is from:
> r2258 | heas | 2010-10-11 20:49:05 +0000 (Mon, 11 Oct 2010) | 3 lines
>
> fnrancid: update recent fortinet software - Diego Ercolani
> Cleaned-up a little by me.
>
> afaict, the justification for full-configuration was so that VDOMs would
> be included in the output. perhaps this behavior has changed since this
> change?? I have none of these devices.

I think you are right.. I have a vague recollection of this as well.

--
Doug Hughes
Keystone NAP
Fairless Hills, PA
1.844.KEYBLOCK (439.2562)

[Nick Nauwelaerts]

This post might be inappropriate. Click to display it.

[heasley]

Wed, Aug 01, 2018 at 08:37:03AM +0000, Nick Nauwelaerts:
> hm,
> i actually like to have those versions in the output. if something breaks my first reaction tends to be: "what changed?", and rancid is usually the first place i check.
>
> would it be an option to control this with FILTER_OSC , even though its not quite it's intended application?

Could be; what are they? version stamp of what exactly?

> thx
>
> // nick
>
>
> From: Rancid-discuss [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of Doug Hughes
> Sent: Tuesday, July 31, 2018 23:18
> To: rancid-***@shrubbery.net
> Subject: Re: [rancid] Fortigate additional tweaks and device filters
>
>
>
>
> On 7/31/2018 5:14 PM, heasley wrote:
>
> Fri, Jul 27, 2018 at 08:02:28AM -0500, Chris Wopat:
>
> Hi Heasley and folks,
>
>
>
> Sept 2017 i sent a note in with some proposed tweaks to a Fortigate. to
>
> filter out some additional chattiness, see:
>
>
>
> http://www.shrubbery.net/pipermail/rancid-discuss/2017-September/009871.html
>
> http://www.shrubbery.net/pipermail/rancid-discuss/2017-June/009643.html
>
>
>
> A few people chimed in seeming to be OK with the propsed changes, which are
>
> to filter these things:
>
>
>
> next if (/^\s*IPS-ETDB: .*/);
>
> next if (/^\s*APP-DB: .*/);
>
> next if (/^\s*IPS Malicious URL Database: .*/);
>
> next if (/^\s*Botnet DB: .*/);
>
>
>
> Mentioning this as 3.8 came out and i didn't notice any of these included.
>
>
>
> We have an additional fortigate tweak we make every time we update too,
>
> which to change from 'show full-configuration' to just 'show' in
>
> @commandtable. 'full-configuration' shows default config, just like the
>
> cisco 'full' command. It's really not necessary IMO.
>
>
>
> This is from:
>
> r2258 | heas | 2010-10-11 20:49:05 +0000 (Mon, 11 Oct 2010) | 3 lines
>
>
>
> fnrancid: update recent fortinet software - Diego Ercolani
>
> Cleaned-up a little by me.
>
>
>
> afaict, the justification for full-configuration was so that VDOMs would
>
> be included in the output. perhaps this behavior has changed since this
>
> change?? I have none of these devices.
>
> I think you are right.. I have a vague recollection of this as well.
> --
> Doug Hughes
> Keystone NAP
> Fairless Hills, PA
> 1.844.KEYBLOCK (439.2562)
>
> [http://www.keystonenap.com/wp-content/themes/keystoneNAP/images/keystone-nap-logo.png]
>
>
>
> ________________________________
>
> Volg Aquafin op Facebook<https://www.facebook.com/AquafinNV> | Twitter<https://twitter.com/aquafinnv> | YouTube<http://www.youtube.com/channel/UCk_4P5BJ-MtEEDCkCsR_KqQ?feature=mhee> | LinkedIN<http://www.linkedin.com/company/aquafin/products>
>
> In het kader van de uitoefening van onze taken verzamelen we bij Aquafin persoonsgegevens. Hoe we omgaan met deze gegevens en wat de rechten van de betrokkenen zijn, kan je nalezen in onze privacy policy<https://www.aquafin.be/nl-be/privacy-policy>.
>
> [https://www.aquafin.be/sites/aquafin/files/styles/paragraph_with_caption/public/2018-06/email_banner_web.jpg]<https://www.aquafin.be/>
> P Denk aan het milieu. Druk deze mail niet onnodig af.

> _______________________________________________
> Rancid-discuss mailing list
> Rancid-***@shrubbery.net
> http://www.shrubbery.net/mailman/listinfo/rancid-discuss

[Nick Nauwelaerts]

This post might be inappropriate. Click to display it.

[Chris Wopat]

> Wed, Aug 01, 2018 at 08:37:03AM +0000, Nick Nauwelaerts:
>> hm,
>> i actually like to have those versions in the output. if something breaks my first reaction tends to be: "what changed?", and rancid is usually the first place i check.
>>
>> would it be an option to control this with FILTER_OSC , even though its not quite it's intended application?
> Could be; what are they? version stamp of what exactly?
>


My additions to filter are based on the fact that there's already a
block of these being filtered, this is just 'more of the same' chatty
stuff that changes daily.

I'd say go one way or another- add more similar filters (my suggestion)
or do none or have a toggle-able option. FILTER_OSC sounds more like
it's for security stuff, so that doesn't seem like the best fit to me.

Has a new FILTER_CRUFT type of option been discussed in the past? Unsure
if this fits the category of any other previously discussed things.

--Chris

[heasley]

Thu, Aug 02, 2018 at 09:25:30AM -0500, Chris Wopat:
> > Wed, Aug 01, 2018 at 08:37:03AM +0000, Nick Nauwelaerts:
> >> hm,
> >> i actually like to have those versions in the output. if something breaks my first reaction tends to be: "what changed?", and rancid is usually the first place i check.
> >>
> >> would it be an option to control this with FILTER_OSC , even though its not quite it's intended application?
> > Could be; what are they? version stamp of what exactly?
> >
>
>
> My additions to filter are based on the fact that there's already a
> block of these being filtered, this is just 'more of the same' chatty
> stuff that changes daily.
>
> I'd say go one way or another- add more similar filters (my suggestion)
> or do none or have a toggle-able option. FILTER_OSC sounds more like
> it's for security stuff, so that doesn't seem like the best fit to me.
>
> Has a new FILTER_CRUFT type of option been discussed in the past? Unsure
> if this fits the category of any other previously discussed things.

it was intended for stuff that oscillated but is still desirable (by some).
so, seems to fit the application, perhaps for the other similar filters.
again, i dont know the platform, so I need input.

[Nick Nauwelaerts]

i guess the fortinet module could use some polishing. it does a great job for getting a complete running config backup. but other information could certainly be welcome to.

perhaps i'll have a look at converting it to a library later on, then you can just comment out the modules you have no interest in. but that will have to wait until i get aerohive hiveos polished a bit.

// nick


-----Original Message-----
From: Rancid-discuss [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of heasley
Sent: Friday, August 3, 2018 00:16
To: Chris Wopat <***@falz.net>
Cc: rancid-***@shrubbery.net
Subject: Re: [rancid] Fortigate additional tweaks and device filters

Thu, Aug 02, 2018 at 09:25:30AM -0500, Chris Wopat:
> > Wed, Aug 01, 2018 at 08:37:03AM +0000, Nick Nauwelaerts:
> >> hm,
> >> i actually like to have those versions in the output. if something breaks my first reaction tends to be: "what changed?", and rancid is usually the first place i check.
> >>
> >> would it be an option to control this with FILTER_OSC , even though its not quite it's intended application?
> > Could be; what are they? version stamp of what exactly?
> >
>
>
> My additions to filter are based on the fact that there's already a
> block of these being filtered, this is just 'more of the same' chatty
> stuff that changes daily.
>
> I'd say go one way or another- add more similar filters (my suggestion)
> or do none or have a toggle-able option. FILTER_OSC sounds more like
> it's for security stuff, so that doesn't seem like the best fit to me.
>
> Has a new FILTER_CRUFT type of option been discussed in the past? Unsure
> if this fits the category of any other previously discussed things.

it was intended for stuff that oscillated but is still desirable (by some).
so, seems to fit the application, perhaps for the other similar filters.
again, i dont know the platform, so I need input.

_______________________________________________
Rancid-discuss mailing list
Rancid-***@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

________________________________

Volg Aquafin op Facebook<https://www.facebook.com/AquafinNV> | Twitter<https://twitter.com/aquafinnv> | YouTube<http://www.youtube.com/channel/UCk_4P5BJ-MtEEDCkCsR_KqQ?feature=mhee> | LinkedIN<http://www.linkedin.com/company/aquafin/products>

In het kader van de uitoefening van onze taken verzamelen we bij Aquafin persoonsgegevens. Hoe we omgaan met deze gegevens en wat de rechten van de betrokkenen zijn, kan je nalezen in onze privacy policy<https://www.aquafin.be/nl-be/privacy-policy>.

[https://www.aquafin.be/sites/aquafin/files/styles/paragraph_with_caption/public/2018-06/email_banner_web.jpg]<https://www.aquafin.be/>
P Denk aan het milieu. Druk deze mail niet onnodig af.

[heasley]

Fri, Aug 03, 2018 at 03:34:05PM +0000, Nick Nauwelaerts:
> i guess the fortinet module could use some polishing. it does a great job for getting a complete running config backup. but other information could certainly be welcome to.
>
> perhaps i'll have a look at converting it to a library later on, then you can just comment out the modules you have no interest in. but that will have to wait until i get aerohive hiveos polished a bit.

i'll convert it, but someone needs to commit to testing it for me, since i
have none of these devices.

> // nick
>
>
> -----Original Message-----
> From: Rancid-discuss [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of heasley
> Sent: Friday, August 3, 2018 00:16
> To: Chris Wopat <***@falz.net>
> Cc: rancid-***@shrubbery.net
> Subject: Re: [rancid] Fortigate additional tweaks and device filters
>
> Thu, Aug 02, 2018 at 09:25:30AM -0500, Chris Wopat:
> > > Wed, Aug 01, 2018 at 08:37:03AM +0000, Nick Nauwelaerts:
> > >> hm,
> > >> i actually like to have those versions in the output. if something breaks my first reaction tends to be: "what changed?", and rancid is usually the first place i check.
> > >>
> > >> would it be an option to control this with FILTER_OSC , even though its not quite it's intended application?
> > > Could be; what are they? version stamp of what exactly?
> > >
> >
> >
> > My additions to filter are based on the fact that there's already a
> > block of these being filtered, this is just 'more of the same' chatty
> > stuff that changes daily.
> >
> > I'd say go one way or another- add more similar filters (my suggestion)
> > or do none or have a toggle-able option. FILTER_OSC sounds more like
> > it's for security stuff, so that doesn't seem like the best fit to me.
> >
> > Has a new FILTER_CRUFT type of option been discussed in the past? Unsure
> > if this fits the category of any other previously discussed things.
>
> it was intended for stuff that oscillated but is still desirable (by some).
> so, seems to fit the application, perhaps for the other similar filters.
> again, i dont know the platform, so I need input.
>
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-***@shrubbery.net
> http://www.shrubbery.net/mailman/listinfo/rancid-discuss
>
> ________________________________
>
> Volg Aquafin op Facebook<https://www.facebook.com/AquafinNV> | Twitter<https://twitter.com/aquafinnv> | YouTube<http://www.youtube.com/channel/UCk_4P5BJ-MtEEDCkCsR_KqQ?feature=mhee> | LinkedIN<http://www.linkedin.com/company/aquafin/products>
>
> In het kader van de uitoefening van onze taken verzamelen we bij Aquafin persoonsgegevens. Hoe we omgaan met deze gegevens en wat de rechten van de betrokkenen zijn, kan je nalezen in onze privacy policy<https://www.aquafin.be/nl-be/privacy-policy>.
>
> [https://www.aquafin.be/sites/aquafin/files/styles/paragraph_with_caption/public/2018-06/email_banner_web.jpg]<https://www.aquafin.be/>
> P Denk aan het milieu. Druk deze mail niet onnodig af.

[Nick Nauwelaerts]

i wouldnt mind testing at all, atm running a 800c cluster on 5.2, and expecting a 500e cluster with 5.6 by the end of the month.

// nick

> On 03 Aug 2018, at 22:08, heasley <***@shrubbery.net> wrote:
>
> Fri, Aug 03, 2018 at 03:34:05PM +0000, Nick Nauwelaerts:
>> i guess the fortinet module could use some polishing. it does a great job for getting a complete running config backup. but other information could certainly be welcome to.
>>
>> perhaps i'll have a look at converting it to a library later on, then you can just comment out the modules you have no interest in. but that will have to wait until i get aerohive hiveos polished a bit.
>
> i'll convert it, but someone needs to commit to testing it for me, since i
> have none of these devices.
>
>> // nick
>>
>>
>> -----Original Message-----
>> From: Rancid-discuss [mailto:rancid-discuss-***@shrubbery.net] On Behalf Of heasley
>> Sent: Friday, August 3, 2018 00:16
>> To: Chris Wopat <***@falz.net>
>> Cc: rancid-***@shrubbery.net
>> Subject: Re: [rancid] Fortigate additional tweaks and device filters
>>
>> Thu, Aug 02, 2018 at 09:25:30AM -0500, Chris Wopat:
>>>> Wed, Aug 01, 2018 at 08:37:03AM +0000, Nick Nauwelaerts:
>>>>> hm,
>>>>> i actually like to have those versions in the output. if something breaks my first reaction tends to be: "what changed?", and rancid is usually the first place i check.
>>>>>
>>>>> would it be an option to control this with FILTER_OSC , even though its not quite it's intended application?
>>>> Could be; what are they? version stamp of what exactly?
>>>
>>>
>>> My additions to filter are based on the fact that there's already a
>>> block of these being filtered, this is just 'more of the same' chatty
>>> stuff that changes daily.
>>>
>>> I'd say go one way or another- add more similar filters (my suggestion)
>>> or do none or have a toggle-able option. FILTER_OSC sounds more like
>>> it's for security stuff, so that doesn't seem like the best fit to me.
>>>
>>> Has a new FILTER_CRUFT type of option been discussed in the past? Unsure
>>> if this fits the category of any other previously discussed things.
>>
>> it was intended for stuff that oscillated but is still desirable (by some).
>> so, seems to fit the application, perhaps for the other similar filters.
>> again, i dont know the platform, so I need input.
>>
>> _______________________________________________
>> Rancid-discuss mailing list
>> Rancid-***@shrubbery.net
>> http://www.shrubbery.net/mailman/listinfo/rancid-discuss
>>
>> ________________________________
>>
>> Volg Aquafin op Facebook<https://www.facebook.com/AquafinNV> | Twitter<https://twitter.com/aquafinnv> | YouTube<http://www.youtube.com/channel/UCk_4P5BJ-MtEEDCkCsR_KqQ?feature=mhee> | LinkedIN<http://www.linkedin.com/company/aquafin/products>
>>
>> In het kader van de uitoefening van onze taken verzamelen we bij Aquafin persoonsgegevens. Hoe we omgaan met deze gegevens en wat de rechten van de betrokkenen zijn, kan je nalezen in onze privacy policy<https://www.aquafin.be/nl-be/privacy-policy>.
>>
>> [https://www.aquafin.be/sites/aquafin/files/styles/paragraph_with_caption/public/2018-06/email_banner_web.jpg]<https://www.aquafin.be/>
>> P Denk aan het milieu. Druk deze mail niet onnodig af.

[Chris Wopat]

On Fri, Aug 3, 2018 at 3:08 PM, heasley <***@shrubbery.net> wrote:

> Fri, Aug 03, 2018 at 03:34:05PM +0000, Nick Nauwelaerts:
> > i guess the fortinet module could use some polishing. it does a great
> job for getting a complete running config backup. but other information
> could certainly be welcome to.
> >
> > perhaps i'll have a look at converting it to a library later on, then
> you can just comment out the modules you have no interest in. but that will
> have to wait until i get aerohive hiveos polished a bit.
>
> i'll convert it, but someone needs to commit to testing it for me, since i
> have none of these devices.
>


We can test as well. we have a small variety - 1000d, 600d, 100d, 500e, all
of which are running 5.6.something.

[Chris Wopat]

On Tue, Jul 31, 2018 at 4:14 PM, heasley <***@shrubbery.net> wrote:
>
> This is from:
> r2258 | heas | 2010-10-11 20:49:05 +0000 (Mon, 11 Oct 2010) | 3 lines
>
> fnrancid: update recent fortinet software - Diego Ercolani
> Cleaned-up a little by me.
>
> afaict, the justification for full-configuration was so that VDOMs would
> be included in the output. perhaps this behavior has changed since this
> change?? I have none of these devices.
>

I had previously never used a vdom, but i just created one with:

config system global
set vdom-admin enable
config vdom
edit test-vdom
config system settings
set status enable

.. then let it run with just 'show' and it certainly shows it (its much
more than this, it created a cert and and a bunch of other stuff), This is
FortiOS 5.6.3.