You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ofbiz.apache.org by Gavin McDonald <ga...@16degrees.com.au> on 2014/09/03 10:23:18 UTC
Suggestions on improving Buildbot Builds
Hi All,
So recently I worked on curing issues with your buildbot build at
http://ci.apache.org/builders/ofbiz-trunk
Part of the build involves running Apache RAT on your code and producing a report at
http://ci.apache.org/projects/ofbiz/rat-output.html
A summary of which also appears on the league table of participating projects at
http://ci.apache.org/projects/rat-master-summary.html
You have quite a bit of code that RAT has to go through in order to compile these reports and
this takes up over an hour of your build on its own. Your build gets run on a per commit basis
so this RAT report is run each and every time a commit is made.
I'd like to propose two suggestions.
Suggestion One :
==============
We remove the RAT report steps from your normal build and stick those steps
into its own build, separate of any other build steps. This build would run via a nightly trigger only.
This would bring your normal build down to just 15 minutes, and is then fine to continue to
run on a per commit basis, concentrating on compile checks, docs and uploading of snapshots.
Suggestion Two :
==============
Your RAT report says there are 2273 (currently) invalid or unlicensed files. There are obviously
some false reports in there, such as generated files that probably should not be checked.
The suggestion is, to go through the report at
http://ci.apache.org/projects/ofbiz/rat-output.html
and see what files and/or directories can be excluded from the RAT check.
Lets get it down to a realistic figure that can then show you exactly what genuine files are missing
license headers.
This can be self-served by the addition of a rat-excludes file in your svn. I then tell RAT to read this
file and the excludes are automatic.
See the 'More Reading and links' section of
http://ci.apache.org/projects/rat-master-summary.html
for example rat-exclude files.
I am happy to get you started and create a patch with a starter rat-excludes file.
If you are happy with these suggestions, let me know and I'll get started.
Gav…
Re: Suggestions on improving Buildbot Builds
Posted by Jacques Le Roux <ja...@les7arts.com>.
Hi Gavin,
We are still running RAT report (at least in trunk), normal?
Jacques
Le 03/09/2014 12:02, Jacques Le Roux a écrit :
> Ha forgot, you may still have a copy of an email "Rat xsl sylesheet used in builbot scripts" from venkatmangudi@gmail.com
> At least I have it here if you think it can behelpful
>
> Jacques
>
> Le 03/09/2014 11:57, Jacques Le Roux a écrit :
>> Hi Gavin,
>>
>> Thanks for your comments, inline...
>>
>> Le 03/09/2014 10:23, Gavin McDonald a écrit :
>>> Hi All,
>>>
>>> So recently I worked on curing issues with your buildbot build at
>>>
>>> http://ci.apache.org/builders/ofbiz-trunk
>>>
>>> Part of the build involves running Apache RAT on your code and producing a report at
>>>
>>> http://ci.apache.org/projects/ofbiz/rat-output.html
>>>
>>> A summary of which also appears on the league table of participating projects at
>>>
>>> http://ci.apache.org/projects/rat-master-summary.html
>>>
>>>
>>> You have quite a bit of code that RAT has to go through in order to compile these reports and
>>> this takes up over an hour of your build on its own. Your build gets run on a per commit basis
>>> so this RAT report is run each and every time a commit is made.
>>>
>>> I'd like to propose two suggestions.
>>>
>>> Suggestion One :
>>> ==============
>>>
>>> We remove the RAT report steps from your normal build and stick those steps
>>> into its own build, separate of any other build steps. This build would run via a nightly trigger only.
>>>
>>> This would bring your normal build down to just 15 minutes, and is then fine to continue to
>>> run on a per commit basis, concentrating on compile checks, docs and uploading of snapshots.
>>
>> I think in all cases we don't need to run the RAT report on each commit, nightly would be well enough
>>
>>>
>>> Suggestion Two :
>>> ==============
>>>
>>> Your RAT report says there are 2273 (currently) invalid or unlicensed files. There are obviously
>>> some false reports in there, such as generated files that probably should not be checked.
>>>
>>> The suggestion is, to go through the report at
>>>
>>> http://ci.apache.org/projects/ofbiz/rat-output.html
>>>
>>> and see what files and/or directories can be excluded from the RAT check.
>>> Lets get it down to a realistic figure that can then show you exactly what genuine files are missing
>>> license headers.
>>>
>>> This can be self-served by the addition of a rat-excludes file in your svn. I then tell RAT to read this
>>> file and the excludes are automatic.
>>>
>>> See the 'More Reading and links' section of
>>>
>>> http://ci.apache.org/projects/rat-master-summary.html
>>>
>>> for example rat-exclude files.
>>>
>>> I am happy to get you started and create a patch with a starter rat-excludes file.
>>>
>>> If you are happy with these suggestions, let me know and I'll get started.
>>
>> That would be great, even if at the moment we are not doing a great use of RAT.
>> This remembers me also this thread http://markmail.org/message/jkeqbdtf5im6zjlm I have completely forgottten it since :/ (as ever, bigger fishes...)
>>
>> Jacques
>>
>>>
>>> Gav…
>>>
>>>
>>>
>>
>
Re: Suggestions on improving Buildbot Builds
Posted by Jacques Le Roux <ja...@les7arts.com>.
Ha forgot, you may still have a copy of an email "Rat xsl sylesheet used in builbot scripts" from venkatmangudi@gmail.com
At least I have it here if you think it can behelpful
Jacques
Le 03/09/2014 11:57, Jacques Le Roux a écrit :
> Hi Gavin,
>
> Thanks for your comments, inline...
>
> Le 03/09/2014 10:23, Gavin McDonald a écrit :
>> Hi All,
>>
>> So recently I worked on curing issues with your buildbot build at
>>
>> http://ci.apache.org/builders/ofbiz-trunk
>>
>> Part of the build involves running Apache RAT on your code and producing a report at
>>
>> http://ci.apache.org/projects/ofbiz/rat-output.html
>>
>> A summary of which also appears on the league table of participating projects at
>>
>> http://ci.apache.org/projects/rat-master-summary.html
>>
>>
>> You have quite a bit of code that RAT has to go through in order to compile these reports and
>> this takes up over an hour of your build on its own. Your build gets run on a per commit basis
>> so this RAT report is run each and every time a commit is made.
>>
>> I'd like to propose two suggestions.
>>
>> Suggestion One :
>> ==============
>>
>> We remove the RAT report steps from your normal build and stick those steps
>> into its own build, separate of any other build steps. This build would run via a nightly trigger only.
>>
>> This would bring your normal build down to just 15 minutes, and is then fine to continue to
>> run on a per commit basis, concentrating on compile checks, docs and uploading of snapshots.
>
> I think in all cases we don't need to run the RAT report on each commit, nightly would be well enough
>
>>
>> Suggestion Two :
>> ==============
>>
>> Your RAT report says there are 2273 (currently) invalid or unlicensed files. There are obviously
>> some false reports in there, such as generated files that probably should not be checked.
>>
>> The suggestion is, to go through the report at
>>
>> http://ci.apache.org/projects/ofbiz/rat-output.html
>>
>> and see what files and/or directories can be excluded from the RAT check.
>> Lets get it down to a realistic figure that can then show you exactly what genuine files are missing
>> license headers.
>>
>> This can be self-served by the addition of a rat-excludes file in your svn. I then tell RAT to read this
>> file and the excludes are automatic.
>>
>> See the 'More Reading and links' section of
>>
>> http://ci.apache.org/projects/rat-master-summary.html
>>
>> for example rat-exclude files.
>>
>> I am happy to get you started and create a patch with a starter rat-excludes file.
>>
>> If you are happy with these suggestions, let me know and I'll get started.
>
> That would be great, even if at the moment we are not doing a great use of RAT.
> This remembers me also this thread http://markmail.org/message/jkeqbdtf5im6zjlm I have completely forgottten it since :/ (as ever, bigger fishes...)
>
> Jacques
>
>>
>> Gav…
>>
>>
>>
>
Re: Suggestions on improving Buildbot Builds
Posted by Jacques Le Roux <ja...@les7arts.com>.
Hi Gavin,
Thanks for your comments, inline...
Le 03/09/2014 10:23, Gavin McDonald a écrit :
> Hi All,
>
> So recently I worked on curing issues with your buildbot build at
>
> http://ci.apache.org/builders/ofbiz-trunk
>
> Part of the build involves running Apache RAT on your code and producing a report at
>
> http://ci.apache.org/projects/ofbiz/rat-output.html
>
> A summary of which also appears on the league table of participating projects at
>
> http://ci.apache.org/projects/rat-master-summary.html
>
>
> You have quite a bit of code that RAT has to go through in order to compile these reports and
> this takes up over an hour of your build on its own. Your build gets run on a per commit basis
> so this RAT report is run each and every time a commit is made.
>
> I'd like to propose two suggestions.
>
> Suggestion One :
> ==============
>
> We remove the RAT report steps from your normal build and stick those steps
> into its own build, separate of any other build steps. This build would run via a nightly trigger only.
>
> This would bring your normal build down to just 15 minutes, and is then fine to continue to
> run on a per commit basis, concentrating on compile checks, docs and uploading of snapshots.
I think in all cases we don't need to run the RAT report on each commit, nightly would be well enough
>
> Suggestion Two :
> ==============
>
> Your RAT report says there are 2273 (currently) invalid or unlicensed files. There are obviously
> some false reports in there, such as generated files that probably should not be checked.
>
> The suggestion is, to go through the report at
>
> http://ci.apache.org/projects/ofbiz/rat-output.html
>
> and see what files and/or directories can be excluded from the RAT check.
> Lets get it down to a realistic figure that can then show you exactly what genuine files are missing
> license headers.
>
> This can be self-served by the addition of a rat-excludes file in your svn. I then tell RAT to read this
> file and the excludes are automatic.
>
> See the 'More Reading and links' section of
>
> http://ci.apache.org/projects/rat-master-summary.html
>
> for example rat-exclude files.
>
> I am happy to get you started and create a patch with a starter rat-excludes file.
>
> If you are happy with these suggestions, let me know and I'll get started.
That would be great, even if at the moment we are not doing a great use of RAT.
This remembers me also this thread http://markmail.org/message/jkeqbdtf5im6zjlm I have completely forgottten it since :/ (as ever, bigger fishes...)
Jacques
>
> Gav…
>
>
>