Re: Speed up jclouds builds

[Andrew Gaul <ga...@apache.org>]

On Tue, May 27, 2014 at 09:33:37PM +0200, Andrew Phillips wrote:
> Just a note to say that I've removed the parallel stuff from the
> jclouds-java-7 PR builder for now. With Checkstyle no longer in the
> parallel bit it doesn't barf, but still fails reliably with the test
> failures seen in [1].
> 
> Bizarrely, it also doesn't appear to be faster? 58min [2] vs. 57min
> [3]? Command line seems to include the right option:
> 
> -B ... clean verify -e -T 2C -Dmaven.javadoc.skip=true

I pushed a commit to upgrade jclouds to Checkstyle 2.13 which addresses
these symptoms.  However, I suspect that the small BuildHive instances
will not benefit from additional parallelism.

I recently profiled "mvn clean test" and found that OpenStack unit tests
sleeping in RetryOnRenew account for 30% of the wall clock time.  I will
investigate how to reduce this.

-- 
Andrew Gaul
http://gaul.org/

Re: Speed up jclouds builds

[Andrew Phillips <ap...@qrmedia.com>]

> I've now re-enabled parallel builds on the jclouds-java-8 job [1]. If
> that goes well

Unfortunately, the java-8 job has become pretty unstable recently. I'm  
not sure if that's caused by the parallelism, but I've disabled it  
again to see if that makes a difference.

ap

Re: Speed up jclouds builds

[Andrew Phillips <ap...@qrmedia.com>]

> I pushed a commit to upgrade jclouds to Checkstyle 2.13 which addresses
> these symptoms.  However, I suspect that the small BuildHive instances
> will not benefit from additional parallelism.

I've now re-enabled parallel builds on the jclouds-java-8 job [1]. If  
that goes well (no new test failures), I suggest we try it on  
jclouds-pull-requests [2].

ap

[1] https://jclouds.ci.cloudbees.com/job/jclouds-java-8/
[2] https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests/

Re: Speed up jclouds builds

[Andrew Phillips <ap...@qrmedia.com>]

> I recently profiled "mvn clean test" and found that OpenStack unit tests
> sleeping in RetryOnRenew account for 30% of the wall clock time.  I will
> investigate how to reduce this.

Thanks for picking this up! I'll also see what happens if we enable  
parallelism again...

ap

Re: Speed up jclouds builds

[Jeremy Daggett <je...@RACKSPACE.COM>]

I would really like to understand what is going on with the OpenStack unit
tests! If there is something we can do to improve those times, I am all
for it.

How are you profiling the build? Can you please share any of that info?
Thanks!

/jd

On 10/1/14, 1:43 PM, "Andrew Gaul" <ga...@apache.org> wrote:

>On Tue, May 27, 2014 at 09:33:37PM +0200, Andrew Phillips wrote:
>> Just a note to say that I've removed the parallel stuff from the
>> jclouds-java-7 PR builder for now. With Checkstyle no longer in the
>> parallel bit it doesn't barf, but still fails reliably with the test
>> failures seen in [1].
>> 
>> Bizarrely, it also doesn't appear to be faster? 58min [2] vs. 57min
>> [3]? Command line seems to include the right option:
>> 
>> -B ... clean verify -e -T 2C -Dmaven.javadoc.skip=true
>
>I pushed a commit to upgrade jclouds to Checkstyle 2.13 which addresses
>these symptoms.  However, I suspect that the small BuildHive instances
>will not benefit from additional parallelism.
>
>I recently profiled "mvn clean test" and found that OpenStack unit tests
>sleeping in RetryOnRenew account for 30% of the wall clock time.  I will
>investigate how to reduce this.
>
>-- 
>Andrew Gaul
>http://gaul.org/