You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@creadur.apache.org by Gavin <ga...@16degrees.com.au> on 2009/10/15 15:47:52 UTC
[SOLVED] - RE: RAT with BuildBot at Apache
> -----Original Message-----
> From: Gavin [mailto:gavin@16degrees.com.au]
> Sent: Thursday, 15 October 2009 11:12 PM
> To: rat-dev@incubator.apache.org
> Subject: RE: RAT with BuildBot at Apache
>
>
>
> > -----Original Message-----
> > From: Gavin [mailto:gavin@16degrees.com.au]
> > Sent: Thursday, 15 October 2009 8:16 PM
> > To: rat-dev@incubator.apache.org
> > Subject: RE: RAT with BuildBot at Apache
> >
> >
> >
> > > -----Original Message-----
> > > From: David Crossley [mailto:crossley@apache.org]
> > > Sent: Thursday, 15 October 2009 10:25 AM
> > > To: rat-dev@incubator.apache.org
> > > Subject: Re: RAT with BuildBot at Apache
> > >
> > > Gav... wrote:
> > > > David Crossley wrote:
>
> <snip>
>
> > >
> > > I was trying to help you with the Ant task, not
> > > the commandline.
> > >
> > > It seems to me that the Ant task would have better
> > > abilities to "exclude" stuff.
> >
> > Ok, I give in, I'll do it that way. I would have preferred to have
> > specified
> > the excludes in a command line argument from within the python build
> > script.
> >
> > This way I have to create separate build.xml files on the filesystem,
> > messy
> > in my opinion, as all the build directories for the builds are wiped
> clean
> > each time and a fresh svn checkout provided, so I can not place the ant
> > tasks file in the same place as the rest of the build.
> >
> > Working on it now, thanks for the hints.
> >
>
> Ok, I have it working now.
>
> http://ci.apache.org/projects/forrest/rat-output.txt
>
> That is upto date using the ant task and a forrest.xml build file, which
> contains excludes for all the files/dirs from the forrest license-
> avoid.txt
> file. So we are down to 111, but I do see more than can be excluded.
>
> I think it will be easy for me to convert that list (license-avoid.txt)
> into
> a list of <exclude name="xxx"/> and have that in an external file, that
> way
> it will dynamically update whenever that file changes.
>
> So, my last question for the list in this particular saga.
>
> How would I include that external file list into the build file into th
> right place?
>
> So from:
>
> <fileset dir="${src-dir}">
> <exclude name=".buildbot-sourcedata"/>
> <exclude name="admin/"/>
> <exclude name="lib/"/>
> ...
> </fileset>
>
> to something like:
>
> <fileset dir="${src-dir}">
> -external file list of excludes goes here-
> </fileset>
>
> I'm looking at loadfile but not sure if that is appropriate.
For the archives etc, 'excludesfile' is my friend, so I now have:
<fileset dir="${src-dir}">
<excludesfile name="${src-dir}/etc/relicense-avoid.txt"/>
<exclude name=".buildbot-sourcedata"/>
<exclude name="**.htaccess"/>
</fileset>
and it reads the .txt as is, brilliant.
Apologies David for taking so long, and so long to realise that actually,
the ant task way of doing it, is actually better in this scenario now.
RAT and Buildbot combo are now once again, much more useful.
Gav...
>
> Thanks
>
> Gav...
>
> > Gav...
> >
> >
> > >
> > > -David
> > >
> > > > RAT63 will already be applied as Buildbot always compiles the latest
> > > > snapshot from RAT trunk, so buildbot is using the latest, including
> > > > the above fix.
> > > >
> > > > so,
> > > >
> > > > --exlcude somefile.name somefirstLevelDir/ someotherdir/
> > > >
> > > > does work
> > > >
> > > > but,
> > > >
> > > > --exclude firstLevel/secondlevel/file.name firstLevel/secondLevel/**
> > > >
> > > > does not.
> > > >
> > > > I can not use the maven way I dont think.
> > > >
> > > > > I am keen to help because Buildbot/RAT is giving
> > > > > Forrest a bad rap due to known exclusions not being
> > > > > sufficiently configured.
> > > >
> > > > Yes, and now having mentioned it on the infra list again this is my
> > > > number one priority, I apologise,but I have mentioned the problem
> > above
> > > > on this list before and not had a specific reply to cure it. I hope
> > > > someone can offer ideas, but i will try again to get it working
> > myself.
> > > No virus found in this incoming message.
> > > Checked by AVG - www.avg.com
> > > Version: 8.5.421 / Virus Database: 270.14.17/2436 - Release Date:
> > 10/14/09
> > > 18:32:00
> >
> > No virus found in this incoming message.
> > Checked by AVG - www.avg.com
> > Version: 8.5.421 / Virus Database: 270.14.17/2436 - Release Date:
> 10/15/09
> > 03:57:00
>
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 8.5.421 / Virus Database: 270.14.17/2436 - Release Date: 10/15/09
> 03:57:00
RE: [SOLVED] - RE: RAT with BuildBot at Apache
Posted by Gavin <ga...@16degrees.com.au>.
> -----Original Message-----
> From: David Crossley [mailto:crossley@apache.org]
> Sent: Friday, 16 October 2009 4:59 PM
> To: rat-dev@incubator.apache.org
> Subject: Re: [SOLVED] - RE: RAT with BuildBot at Apache
>
> Gav... wrote:
> > David Crossley wrote:
> > >
> > > Okay, the initial etc/rat-avoid.txt is ready now.
> > > Forrest devs can refine it.
> >
> > Ok, changed and done.
> >
> > Results at
> > http://ci.apache.org/projects/forrest/rat-output.txt
>
> That improves it, thanks.
>
> I tweaked our config some more, but do not yet see
> the results updated. How often does Buildbot run,
> or how is it triggered?
For some projects, whilst testing I had been triggering them manually from
the IRC bot. 'forrest-bot' on the #asftest channel in this case. As Forrest
does not have its own channel currently, along with a few others, that's
where it lives.
(A discussion for Forrest list as to whether we keep it or disable it.)
I have now however enabled per commit builds of the RAT report for Forrest,
so it will recheck and update after every commit.
Gav...
>
> -David
Re: [SOLVED] - RE: RAT with BuildBot at Apache
Posted by David Crossley <cr...@apache.org>.
Gav... wrote:
> David Crossley wrote:
> >
> > Okay, the initial etc/rat-avoid.txt is ready now.
> > Forrest devs can refine it.
>
> Ok, changed and done.
>
> Results at
> http://ci.apache.org/projects/forrest/rat-output.txt
That improves it, thanks.
I tweaked our config some more, but do not yet see
the results updated. How often does Buildbot run,
or how is it triggered?
-David
Re: [SOLVED] - RE: RAT with BuildBot at Apache
Posted by "Gav..." <ga...@16degrees.com.au>.
On Fri, 16 Oct 2009 16:37:27 +1100, David Crossley <cr...@apache.org>
wrote:
> Gav... wrote:
>> David Crossley wrote:
>> >
>> > So perhaps the RAT file should be "rat-avoid.txt" or "rat-exclude.txt"
>> > and we just have duplicate similar files for each purpose.
>> >
>> > Should i go ahead and create that file for Forrest?
>>
>> Yep sure, I'll switch to that when done.
>
> Okay, the initial etc/rat-avoid.txt is ready now.
> Forrest devs can refine it.
Ok, changed and done.
Results at
http://ci.apache.org/projects/forrest/rat-output.txt
>
> By the way, could you also add the following (or similar)
> to the ant build file. This assists until RAT-67.
>
> ---------
> <tstamp/>
> <echo level="info">
> Timestamp: ${DSTAMP} ${TSTAMP}
> </echo>
> ---------
Done also.
Gav...
>
>> On a case by case basis, I dont expect many projects to provide such a
>> file,
>> in which case the normal exclude pattern list will do.
>
> Don't bet on it.
>
> -David
--
Gav...
16 degrees complete web solutions
Re: [SOLVED] - RE: RAT with BuildBot at Apache
Posted by David Crossley <cr...@apache.org>.
Gav... wrote:
> David Crossley wrote:
> >
> > So perhaps the RAT file should be "rat-avoid.txt" or "rat-exclude.txt"
> > and we just have duplicate similar files for each purpose.
> >
> > Should i go ahead and create that file for Forrest?
>
> Yep sure, I'll switch to that when done.
Okay, the initial etc/rat-avoid.txt is ready now.
Forrest devs can refine it.
By the way, could you also add the following (or similar)
to the ant build file. This assists until RAT-67.
---------
<tstamp/>
<echo level="info">
Timestamp: ${DSTAMP} ${TSTAMP}
</echo>
---------
> On a case by case basis, I dont expect many projects to provide such a
> file,
> in which case the normal exclude pattern list will do.
Don't bet on it.
-David
Re: [SOLVED] - RE: RAT with BuildBot at Apache
Posted by "Gav..." <ga...@16degrees.com.au>.
On Fri, 16 Oct 2009 13:52:57 +1100, David Crossley <cr...@apache.org>
wrote:
> Gavin wrote:
>> >
>> > Ok, I have it working now.
>> >
>> > http://ci.apache.org/projects/forrest/rat-output.txt
>> >
>> > That is upto date using the ant task and a forrest.xml build file,
>> > which
>> > contains excludes for all the files/dirs from the forrest license-
>> > avoid.txt
>> > file. So we are down to 111, but I do see more than can be excluded.
>> >
>> > I think it will be easy for me to convert that list
(license-avoid.txt)
>> > into
>> > a list of <exclude name="xxx"/> and have that in an external file,
that
>> > way
>> > it will dynamically update whenever that file changes.
>> >
>> > So, my last question for the list in this particular saga.
>> >
>> > How would I include that external file list into the build file into
th
>> > right place?
>> >
>> > So from:
>> >
>> > <fileset dir="${src-dir}">
>> > <exclude name=".buildbot-sourcedata"/>
>> > <exclude name="admin/"/>
>> > <exclude name="lib/"/>
>> > ...
>> > </fileset>
>> >
>> > to something like:
>> >
>> > <fileset dir="${src-dir}">
>> > -external file list of excludes goes here-
>> > </fileset>
>> >
>> > I'm looking at loadfile but not sure if that is appropriate.
>>
>> For the archives etc, 'excludesfile' is my friend, so I now have:
>>
>> <fileset dir="${src-dir}">
>> <excludesfile name="${src-dir}/etc/relicense-avoid.txt"/>
>> <exclude name=".buildbot-sourcedata"/>
>> <exclude name="**.htaccess"/>
>> </fileset>
>>
>> and it reads the .txt as is, brilliant.
>
> Beauty.
>
> One hitch in Forrest's case. That etc/relicense-avoid.txt
> is not a list of Ant-style patterns, it is a crude list
> of exact text patterns for my old relicense.pl script.
>
> So perhaps the RAT file should be "rat-avoid.txt" or "rat-exclude.txt"
> and we just have duplicate similar files for each purpose.
>
> Should i go ahead and create that file for Forrest?
Yep sure, I'll switch to that when done.
On a case by case basis, I dont expect many projects to provide such a
file,
in which case the normal exclude pattern list will do.
Thanks
Gav...
>
>> Apologies David for taking so long,
>
> No worries. Glad that we could get RAT processing improved.
>
>> and so long to realise that actually,
>> the ant task way of doing it, is actually better in this scenario now.
>>
>> RAT and Buildbot combo are now once again, much more useful.
>>
>> Gav...
>
> Yep, fantastic ... thanks too to Ant.
>
> -David
--
Gav...
16 degrees complete web solutions
Re: [SOLVED] - RE: RAT with BuildBot at Apache
Posted by David Crossley <cr...@apache.org>.
Gavin wrote:
> >
> > Ok, I have it working now.
> >
> > http://ci.apache.org/projects/forrest/rat-output.txt
> >
> > That is upto date using the ant task and a forrest.xml build file, which
> > contains excludes for all the files/dirs from the forrest license-
> > avoid.txt
> > file. So we are down to 111, but I do see more than can be excluded.
> >
> > I think it will be easy for me to convert that list (license-avoid.txt)
> > into
> > a list of <exclude name="xxx"/> and have that in an external file, that
> > way
> > it will dynamically update whenever that file changes.
> >
> > So, my last question for the list in this particular saga.
> >
> > How would I include that external file list into the build file into th
> > right place?
> >
> > So from:
> >
> > <fileset dir="${src-dir}">
> > <exclude name=".buildbot-sourcedata"/>
> > <exclude name="admin/"/>
> > <exclude name="lib/"/>
> > ...
> > </fileset>
> >
> > to something like:
> >
> > <fileset dir="${src-dir}">
> > -external file list of excludes goes here-
> > </fileset>
> >
> > I'm looking at loadfile but not sure if that is appropriate.
>
> For the archives etc, 'excludesfile' is my friend, so I now have:
>
> <fileset dir="${src-dir}">
> <excludesfile name="${src-dir}/etc/relicense-avoid.txt"/>
> <exclude name=".buildbot-sourcedata"/>
> <exclude name="**.htaccess"/>
> </fileset>
>
> and it reads the .txt as is, brilliant.
Beauty.
One hitch in Forrest's case. That etc/relicense-avoid.txt
is not a list of Ant-style patterns, it is a crude list
of exact text patterns for my old relicense.pl script.
So perhaps the RAT file should be "rat-avoid.txt" or "rat-exclude.txt"
and we just have duplicate similar files for each purpose.
Should i go ahead and create that file for Forrest?
> Apologies David for taking so long,
No worries. Glad that we could get RAT processing improved.
> and so long to realise that actually,
> the ant task way of doing it, is actually better in this scenario now.
>
> RAT and Buildbot combo are now once again, much more useful.
>
> Gav...
Yep, fantastic ... thanks too to Ant.
-David