eliminate RAT from build process

[Dan Kirkwood <da...@apache.org>]

Hi, all..     I think weasel is mature enough that it covers our license
usage well enough.  RAT is run as a separate job from the rpm builds and
runs on the source tarball;  weasel runs during the build process on the
source cloned from github:    they have pretty much the same function,  but
use different files to indicate files that should not be checked.

To eliminate this duplication of effort,  I'm proposing eliminating RAT
from the build process which would remove the "-rat" jobs from the list in
jenkins:

https://builds.apache.org/view/S-Z/view/TrafficControl

Any objections?

thanks..   Dan

Re: eliminate RAT from build process

[Chris Lemmons <al...@gmail.com>]

Rat does do a few things that weasel doesn't:
 - It has a lower false positive license rate; it is much less likely
to falsely identify a project as having a particular license and
instead produce an Unknown.
 - It has better support for checking to ensure that a file has the
appropriate Apache header.
 - At one point, all the rat reports were parsed by a global ASF
project to create a leaderboard between projects. I can't seem to find
it now, though.

I'm not really concerned about the first item. It's slightly annoying
to have it falsely identify headers based on very short keywords, but
it's easy to override in the .dependency_license file. And since some
of our files have no more license information than a single "//
License: MIT or BSD" in the file, it's how we avoid false negatives,
which are worse.

The real value at the moment comes from the second point, though. It's
possible to wind up with files that are detected as Apache, but don't
bear the ASF header. We wouldn't really want that to wind up in a
release. This would only really happen if the file itself had
something that looked kinda like apache licensing inside it, though.
It's been on my to-do list to add this to weasel, though.

Also, rat will offer to add headers to your files for you. This is
usually a bad idea, though, unless you're wholesale converting a
codebase for the first time. This doesn't really bear on the CI use
case, though.

If the ASF is still collecting rat reports, it might be worth checking
to see if it would be useful to teach weasel how to spit out whatever
rat has as its output.

There's the run-down. That being said, I'm +1 on going just weasel.
The cost of maintaining the ignores exceeds the value they are
producing. Saving a few cycles on the build servers is a useful goal
as well.
On Mon, Nov 26, 2018 at 4:53 PM Dave Neuman <ne...@apache.org> wrote:
>
> Is Rat providing license checking that is not covered by Weasel?  What are
> we losing but no longer using Rat?
>
>
> On Mon, Nov 26, 2018 at 2:47 PM Dan Kirkwood <da...@apache.org> wrote:
>
> > Hi, all..     I think weasel is mature enough that it covers our license
> > usage well enough.  RAT is run as a separate job from the rpm builds and
> > runs on the source tarball;  weasel runs during the build process on the
> > source cloned from github:    they have pretty much the same function,  but
> > use different files to indicate files that should not be checked.
> >
> > To eliminate this duplication of effort,  I'm proposing eliminating RAT
> > from the build process which would remove the "-rat" jobs from the list in
> > jenkins:
> >
> > https://builds.apache.org/view/S-Z/view/TrafficControl
> >
> > Any objections?
> >
> > thanks..   Dan
> >

Re: eliminate RAT from build process

[Dave Neuman <ne...@apache.org>]

Is Rat providing license checking that is not covered by Weasel?  What are
we losing but no longer using Rat?


On Mon, Nov 26, 2018 at 2:47 PM Dan Kirkwood <da...@apache.org> wrote:

> Hi, all..     I think weasel is mature enough that it covers our license
> usage well enough.  RAT is run as a separate job from the rpm builds and
> runs on the source tarball;  weasel runs during the build process on the
> source cloned from github:    they have pretty much the same function,  but
> use different files to indicate files that should not be checked.
>
> To eliminate this duplication of effort,  I'm proposing eliminating RAT
> from the build process which would remove the "-rat" jobs from the list in
> jenkins:
>
> https://builds.apache.org/view/S-Z/view/TrafficControl
>
> Any objections?
>
> thanks..   Dan
>