You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by Andrew Phillips <an...@apache.org> on 2014/05/03 02:55:51 UTC
Re: Speed up jclouds builds
> Added "-T 2C" to "jclouds-pull-requests" but *not* to
> "jclouds-java-7-pull-requests", so we should be able to see what (if
> any) the difference is.
I'm afraid I've had to revert this change. This has been reliably
causing the jclouds-pull-request job [1] to fail with weird
compilation errors in dynect where the jclouds-java-7-pull-request job
(which didn't have this change) was fine [2].
Looks like our builds are not parallelizable? Or, at any rate, not on
DEV@cloud? Any other settings/different values for the settings we
could try?
ap
[1]
https://jclouds.ci.cloudbees.com/view/pull%20requests/job/jclouds-pull-requests/780/console
[2]
https://jclouds.ci.cloudbees.com/view/pull%20requests/job/jclouds-java-7-pull-requests/1250/console
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> I'm ok reverting the change, although it would be good to see if we
> can determine why those weird compilation error occur, as in theory,
> they shouldn't appear. Perhaps could be good to have a new build to
> play with this?
Sure! See:
https://jclouds.ci.cloudbees.com/job/EXPERIMENTAL-jclouds-java-7-parallel/
Starting a build ("clean compile -T 2C") now. Curious to see if the
Maven version (currently 3.0.3) has anything to do with it...
ap
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> I've now re-enabled parallel builds on the jclouds-java-8 job [1]. If
> that goes well
Unfortunately, the java-8 job has become pretty unstable recently. I'm
not sure if that's caused by the parallelism, but I've disabled it
again to see if that makes a difference.
ap
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> I pushed a commit to upgrade jclouds to Checkstyle 2.13 which addresses
> these symptoms. However, I suspect that the small BuildHive instances
> will not benefit from additional parallelism.
I've now re-enabled parallel builds on the jclouds-java-8 job [1]. If
that goes well (no new test failures), I suggest we try it on
jclouds-pull-requests [2].
ap
[1] https://jclouds.ci.cloudbees.com/job/jclouds-java-8/
[2] https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests/
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> I recently profiled "mvn clean test" and found that OpenStack unit tests
> sleeping in RetryOnRenew account for 30% of the wall clock time. I will
> investigate how to reduce this.
Thanks for picking this up! I'll also see what happens if we enable
parallelism again...
ap
Re: Speed up jclouds builds
Posted by Jeremy Daggett <je...@RACKSPACE.COM>.
I would really like to understand what is going on with the OpenStack unit
tests! If there is something we can do to improve those times, I am all
for it.
How are you profiling the build? Can you please share any of that info?
Thanks!
/jd
On 10/1/14, 1:43 PM, "Andrew Gaul" <ga...@apache.org> wrote:
>On Tue, May 27, 2014 at 09:33:37PM +0200, Andrew Phillips wrote:
>> Just a note to say that I've removed the parallel stuff from the
>> jclouds-java-7 PR builder for now. With Checkstyle no longer in the
>> parallel bit it doesn't barf, but still fails reliably with the test
>> failures seen in [1].
>>
>> Bizarrely, it also doesn't appear to be faster? 58min [2] vs. 57min
>> [3]? Command line seems to include the right option:
>>
>> -B ... clean verify -e -T 2C -Dmaven.javadoc.skip=true
>
>I pushed a commit to upgrade jclouds to Checkstyle 2.13 which addresses
>these symptoms. However, I suspect that the small BuildHive instances
>will not benefit from additional parallelism.
>
>I recently profiled "mvn clean test" and found that OpenStack unit tests
>sleeping in RetryOnRenew account for 30% of the wall clock time. I will
>investigate how to reduce this.
>
>--
>Andrew Gaul
>http://gaul.org/
Re: Speed up jclouds builds
Posted by Andrew Gaul <ga...@apache.org>.
On Tue, May 27, 2014 at 09:33:37PM +0200, Andrew Phillips wrote:
> Just a note to say that I've removed the parallel stuff from the
> jclouds-java-7 PR builder for now. With Checkstyle no longer in the
> parallel bit it doesn't barf, but still fails reliably with the test
> failures seen in [1].
>
> Bizarrely, it also doesn't appear to be faster? 58min [2] vs. 57min
> [3]? Command line seems to include the right option:
>
> -B ... clean verify -e -T 2C -Dmaven.javadoc.skip=true
I pushed a commit to upgrade jclouds to Checkstyle 2.13 which addresses
these symptoms. However, I suspect that the small BuildHive instances
will not benefit from additional parallelism.
I recently profiled "mvn clean test" and found that OpenStack unit tests
sleeping in RetryOnRenew account for 30% of the wall clock time. I will
investigate how to reduce this.
--
Andrew Gaul
http://gaul.org/
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
I can't see any immediately obvious way this is related to parallel
builds, but seeing as the other PR builder (which is not parallel)
didn't barf on these, I guess that must be the cause?
https://jclouds.ci.cloudbees.com/job/jclouds-java-7-pull-requests/org.apache.jclouds.common$azure-common/1291/testReport/junit/org.jclouds.azure.storage.filters/SharedKeyLiteAuthenticationTest/testIdempotent/
ap
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> Otherwise we can disable parallelism for now. Do we have any benchmarks
> which demonstrate the benefits of this?
Well, those are precisely the benchmarks we were trying to gather in
order to decide whether to enable parallelism ;-)
What I'm going to do for now is take checkstyle:checkstyle out of the
java-7 build and make that parallel. Then at lease we can get some
numbers relative to the java-6 PR builder.
ap
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
Just a note to say that I've removed the parallel stuff from the
jclouds-java-7 PR builder for now. With Checkstyle no longer in the
parallel bit it doesn't barf, but still fails reliably with the test
failures seen in [1].
Bizarrely, it also doesn't appear to be faster? 58min [2] vs. 57min
[3]? Command line seems to include the right option:
-B ... clean verify -e -T 2C -Dmaven.javadoc.skip=true
ap
[1] https://jclouds.ci.cloudbees.com/job/EXPERIMENTAL-jclouds-java-7-parallel/
[2] https://jclouds.ci.cloudbees.com/job/jclouds-java-7-pull-requests/1305/
[3] https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests/835/
Re: Speed up jclouds builds
Posted by Andrew Gaul <ga...@apache.org>.
On Wed, May 21, 2014 at 05:12:58PM +0200, Andrew Phillips wrote:
> >let's work with the upstream projects and disable parallelism for
> >the checkstyle goal for now.
>
> How do you envisage this? Take the checkstyle:checkstyle goal out of
> the build and add it as a post-build step? We have mainly Maven
> (rather than freestyle) builds, so unfortunately we can't add this
> as another build step :-(
We could break out checkstyle into another builder, similar to the
java-7 builder. We might as well make violations fail the build while
we are at it.
Otherwise we can disable parallelism for now. Do we have any benchmarks
which demonstrate the benefits of this?
--
Andrew Gaul
http://gaul.org/
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> let's work with the upstream projects and disable parallelism for
> the checkstyle goal for now.
How do you envisage this? Take the checkstyle:checkstyle goal out of
the build and add it as a post-build step? We have mainly Maven
(rather than freestyle) builds, so unfortunately we can't add this as
another build step :-(
ap
Re: Speed up jclouds builds
Posted by Andrew Gaul <ga...@apache.org>.
On Wed, May 21, 2014 at 03:06:08AM +0200, Andrew Phillips wrote:
> >I submitted a possible fix to the underlying plexus-resources:
> >
> >http://jira.codehaus.org/browse/PLXCOMP-237
> >
> >I can reliably trigger these symptoms so until both projects incorporate
> >a workaround we may need to run the checkstyle phase without
> >parallelism.
>
> Thanks, Gaul! Would it make sense to create a "private" build of
> this plugin and try with that while we're waiting for the Checkstyle
> team..?
A private build has a poor effort to benefit ratio; let's work with the
upstream projects and disable parallelism for the checkstyle goal for
now.
--
Andrew Gaul
http://gaul.org/
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> I submitted a possible fix to the underlying plexus-resources:
>
> http://jira.codehaus.org/browse/PLXCOMP-237
>
> I can reliably trigger these symptoms so until both projects incorporate
> a workaround we may need to run the checkstyle phase without
> parallelism.
Thanks, Gaul! Would it make sense to create a "private" build of this
plugin and try with that while we're waiting for the Checkstyle team..?
ap
Re: Speed up jclouds builds
Posted by Andrew Gaul <ga...@apache.org>.
Andrew Phillips encountered another issue with parallel builds and
maven-checkstyle-plugin:
https://github.com/jclouds/jclouds/pull/375#issuecomment-43446787
I submitted a possible fix to the underlying plexus-resources:
http://jira.codehaus.org/browse/PLXCOMP-237
I can reliably trigger these symptoms so until both projects incorporate
a workaround we may need to run the checkstyle phase without
parallelism.
On Mon, May 12, 2014 at 10:01:51AM +0200, Ignasi Barrera wrote:
> Here it is: https://github.com/jclouds/jclouds/pull/361
> The warning about the RAT plugin has disappeared with this parent pom.
>
> On 11 May 2014 23:18, Andrew Phillips <ap...@qrmedia.com> wrote:
> >> We could update the parent pom to version 14, which already updates the
> >> rat
> >> plugin from 0.8 to 0.10 and see if the warning still appears.
> >
> >
> > Seems like an obvious PR candidate to me ;-)
> >
> > ap
--
Andrew Gaul
http://gaul.org/
Re: Speed up jclouds builds
Posted by Ignasi Barrera <na...@apache.org>.
Here it is: https://github.com/jclouds/jclouds/pull/361
The warning about the RAT plugin has disappeared with this parent pom.
On 11 May 2014 23:18, Andrew Phillips <ap...@qrmedia.com> wrote:
>> We could update the parent pom to version 14, which already updates the
>> rat
>> plugin from 0.8 to 0.10 and see if the warning still appears.
>
>
> Seems like an obvious PR candidate to me ;-)
>
> ap
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> We could update the parent pom to version 14, which already updates the rat
> plugin from 0.8 to 0.10 and see if the warning still appears.
Seems like an obvious PR candidate to me ;-)
ap
Re: Speed up jclouds builds
Posted by Ignasi Barrera <na...@apache.org>.
We could update the parent pom to version 14, which already updates the rat
plugin from 0.8 to 0.10 and see if the warning still appears.
El 11/05/2014 18:11, "Andrew Phillips" <ap...@qrmedia.com> escribió:
> I've also seen this warning when running concurrent builds with Maven
>> 3.0.4, so perhaps we should check if those plugins can be updated,
>>
>
> +1. Looks like a good question for the RAT plugin mailing lists..? [1]
> Note that they seem to have already annotated rat:help as being thread-safe
> [2], but not rat:check or rat:rat.
>
> ap
>
> [1] http://creadur.apache.org/rat/apache-rat-plugin/mail-lists.html
> [2] https://creadur.apache.org/rat/apache-rat-plugin/help-mojo.html
>
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> I've also seen this warning when running concurrent builds with Maven
> 3.0.4, so perhaps we should check if those plugins can be updated,
+1. Looks like a good question for the RAT plugin mailing lists..? [1]
Note that they seem to have already annotated rat:help as being
thread-safe [2], but not rat:check or rat:rat.
ap
[1] http://creadur.apache.org/rat/apache-rat-plugin/mail-lists.html
[2] https://creadur.apache.org/rat/apache-rat-plugin/help-mojo.html
Re: Speed up jclouds builds
Posted by Ignasi Barrera <ig...@gmail.com>.
I've also seen this warning when running concurrent builds with Maven
3.0.4, so perhaps we should check if those plugins can be updated,
otherwise we might see invalid results (and the rat report is
something we need to trust) when running builds in parallel.
[WARNING] *****************************************************************
[WARNING] * Your build is requesting parallel execution, but project *
[WARNING] * contains the following plugin(s) that are not marked as *
[WARNING] * @threadSafe to support parallel building. *
[WARNING] * While this /may/ work fine, please look for plugin updates *
[WARNING] * and/or request plugins be made thread-safe. *
[WARNING] * If reporting an issue, report it against the plugin in *
[WARNING] * question, not against maven-core *
[WARNING] *****************************************************************
[WARNING] The following plugins are not marked @threadSafe in Apache jclouds:
[WARNING] com.ning.maven.plugins:maven-duplicate-finder-plugin:1.0.5
[WARNING] org.apache.rat:apache-rat-plugin:0.10
[WARNING] *****************************************************************
On 5 May 2014 17:52, Andrew Phillips <ap...@qrmedia.com> wrote:
> Some interesting initial data [1]. First point to note: unable to reproduce
> the compilation failure so far:
>
> Maven 3.1.0, "clean verify -e -T 2C -Dmaven.javadoc.skip=true": Aborted
> Maven 3.1.0, "clean verify -e -T 2C -Dmaven.javadoc.skip=true": Failed
> Maven 3.0.5, "clean verify -e -T 2C -Dmaven.javadoc.skip=true": Aborted
> Maven 3.0.5, "clean verify checkstyle:checkstyle -e -T 2C
> -Dmaven.javadoc.skip=true": Aborted
> Maven 3.0.5, "clean compile -e -T 2C -Dmaven.javadoc.skip=true": Success
> Maven 3.0.3, "clean compile -e -T 2C -Dmaven.javadoc.skip=true": Success
>
> ap
>
> [1]
> https://jclouds.ci.cloudbees.com/job/EXPERIMENTAL-jclouds-java-7-parallel/
Re: Speed up jclouds builds
Posted by Andrew Phillips <ap...@qrmedia.com>.
Some interesting initial data [1]. First point to note: unable to
reproduce the compilation failure so far:
Maven 3.1.0, "clean verify -e -T 2C -Dmaven.javadoc.skip=true": Aborted
Maven 3.1.0, "clean verify -e -T 2C -Dmaven.javadoc.skip=true": Failed
Maven 3.0.5, "clean verify -e -T 2C -Dmaven.javadoc.skip=true": Aborted
Maven 3.0.5, "clean verify checkstyle:checkstyle -e -T 2C
-Dmaven.javadoc.skip=true": Aborted
Maven 3.0.5, "clean compile -e -T 2C -Dmaven.javadoc.skip=true": Success
Maven 3.0.3, "clean compile -e -T 2C -Dmaven.javadoc.skip=true": Success
ap
[1] https://jclouds.ci.cloudbees.com/job/EXPERIMENTAL-jclouds-java-7-parallel/
Re: Speed up jclouds builds
Posted by Ignasi Barrera <ig...@gmail.com>.
They *should* be parallelizable, as Maven takes the dependency graph
computed by the reactor, to set which modules can be build in parallel
[1]. Anyway, the feature is experimental, and they already say there
might be still some undiscovered issues, so I'm not sure if we can
rely on it yet.
I'm ok reverting the change, although it would be good to see if we
can determine why those weird compilation error occur, as in theory,
they shouldn't appear. Perhaps could be good to have a new build to
play with this?
I.
[1] https://cwiki.apache.org/confluence/display/MAVEN/Parallel+builds+in+Maven+3
On 3 May 2014 02:55, Andrew Phillips <an...@apache.org> wrote:
>> Added "-T 2C" to "jclouds-pull-requests" but *not* to
>> "jclouds-java-7-pull-requests", so we should be able to see what (if
>> any) the difference is.
>
>
> I'm afraid I've had to revert this change. This has been reliably
> causing the jclouds-pull-request job [1] to fail with weird
> compilation errors in dynect where the jclouds-java-7-pull-request job
> (which didn't have this change) was fine [2].
>
> Looks like our builds are not parallelizable? Or, at any rate, not on
> DEV@cloud? Any other settings/different values for the settings we
> could try?
>
> ap
>
> [1]
> https://jclouds.ci.cloudbees.com/view/pull%20requests/job/jclouds-pull-requests/780/console
> [2]
> https://jclouds.ci.cloudbees.com/view/pull%20requests/job/jclouds-java-7-pull-requests/1250/console
>