You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Kenneth Knowles <kl...@google.com> on 2017/12/01 21:18:14 UTC
Re: How to cope with Maven transient network issues?
I've repeatedly searched around for a way to just add proper retry to maven
or gradle, and haven't found anything :-(
I had thought that we altered our builds in such a way that the .m2
directory was permitted to survive across builds. True that it isn't
hermetic, precisely, but it is pretty safe to treat as a cache of immutable
data, which is no more dangerous than having a caching incremental build
system.
Kenn
On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov <ki...@google.com>
wrote:
> Our builds often hit transient Maven network issues, e.g. this one
> https://builds.apache.org/job/beam_PostCommit_Java_
> MavenInstall/5331/consoleFull
>
> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve dependencies for
> project org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT:
> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0 from/to
> central (https://repo.maven.apache.org/maven2): GET request of:
> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central failed:
> Connection reset -> [Help 1]
>
> It'd be good to increase reliability of our builds. repo.maven.apache.org
> seems quite unreliable.
>
> I tried finding a way to configure Maven to retry such network errors and
> it appears to be impossible [will be happy if someone proves me wrong].
>
> Would this issue be resolved if we used multiple mirrors?
> https://maven.apache.org/guides/mini/guide-mirror-settings.html Any other
> suggestions?
>
Re: How to cope with Maven transient network issues?
Posted by Eugene Kirpichov <ki...@google.com>.
Thank you!
On Tue, Dec 5, 2017 at 3:11 AM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> FYI https://github.com/apache/maven-wagon/pull/37
>
> Romain Manni-Bucau
> @rmannibucau | Blog | Old Blog | Github | LinkedIn
>
>
> 2017-12-05 6:54 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
> >
> >
> > Le 5 déc. 2017 06:20, "Eugene Kirpichov" <ki...@google.com> a écrit
> :
> >
> >
> >
> > On Mon, Dec 4, 2017 at 1:45 PM Romain Manni-Bucau <rmannibucau@gmail.com
> >
> > wrote:
> >>
> >>
> >>
> >> Le 4 déc. 2017 21:45, "Eugene Kirpichov" <ki...@google.com> a
> écrit :
> >>
> >> Romain - as far as I understand, Maven *does* have a retry strategy, but
> >> it is a poor retry strategy and there is no way to tweak it. In
> particular,
> >> if it uses
> https://maven.apache.org/guides/mini/guide-http-settings.html ,
> >> that means it uses Apache Http Client 4.1.2 whose default retry
> settings are
> >> extremely conservative
> >>
> >>
> https://github.com/apache/httpcomponents-client/blob/4.1.x/httpclient/src/main/java/org/apache/http/impl/client/DefaultHttpRequestRetryHandler.java#L93
> >>
> >>
> >> Right
> >>
> >>
> >>
> >> The errors we're hitting with Maven are some of those: "connection
> >> refused" and alike.
> >>
> >>
> >>
> >> Would a retry guarantee you it works. Should be quite easy to switch
> wagon
> >> with a configurable retry handler instance through mvnwrapper or custom
> mvn
> >> setup and test it but the logic looks sane to me.
> >
> > Retries indeed don't give any guarantees, but they're very good at
> reducing
> > error rate, and our builds could definitely use some of that.
> >
> > I tried to figure out how to make Maven use a configurable http wagon
> with a
> > custom retry handler, but I couldn't - if you know more, could you give
> some
> > pointers? I think we should definitely do that.
> >
> >
> > It is not configurable but to ensure it is worth a mvn patch we can add a
> > mvn wrapper script where we patch wagon and use that on the CI.
> >
> >
> >
> >>
> >>
> >> If it is on asf repo we must work with infra to fix it rather than
> working
> >> around the clients. If on a repo we dont control we can try to get rid
> of it
> >> maybe?
> >
> > It happens often enough with the maven central repo
> repo.maven.apache.org.
> > At Beam's (moderate) scale of hundreds of builds per day resolving
> hundreds
> > of dependencies each, even 99.99% availability with a poor retry strategy
> > would translate to builds failing regularly.
> >
> >
> > Hmm, this is an asf server and behing should be repo1 and repo2 so either
> > there is a huge client issue or the proxy can be better configure. If you
> > have some failure logs, do you mind pinging infra with them?
> >
> > There's also multiple parts that can fail - Maven Central itself, network
> > issues between Jenkins and Maven Central, network issues on the Jenkins
> > machine itself and so on - I doubt that ASF can fix all of this for us.
> > I think this sort of issue *has* to be resolved on the client.
> >
> >
> > Not all kind of failures, connection breakdown during the transfert yes
> but
> > other kind means the server is down so retry shouldnt help.
> >
> > Let me know if you nees help with wagon patching/testing. Can have a few
> > cycles end of the week to give some help.
> >
> >
> >>
> >>
> >>
> >>
> >> On Mon, Dec 4, 2017 at 12:15 PM Romain Manni-Bucau <
> rmannibucau@gmail.com>
> >> wrote:
> >>>
> >>> 2017-12-04 18:58 GMT+01:00 Kenneth Knowles <kl...@google.com>:
> >>> > On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau
> >>> > <rm...@gmail.com>
> >>> > wrote:
> >>> >>
> >>> >> Need to check but if plugin dependencies were not tuned it should be
> >>> >> the
> >>> >> default with a retry of 3 IIRC.
> >>> >>
> >>> >> But arent you sure it was a repo/server issue any client cant solve?
> >>> >
> >>> >
> >>> > Not sure what you mean here. What we need is for mvn and gradle to
> >>> > succeed
> >>> > even though there are frequent transient failures of the repo.
> Anything
> >>> > that
> >>> > makes this happen is success.
> >>>
> >>> What I meant is you will likely never get it with whatever build
> >>> system until you go 100% local which wouldn't make the build
> >>> reproducible.
> >>>
> >>> Concretely you can do
> >>>
> >>> 1.
> >>>
> >>> for (int i = 0; i < 3; i++) {
> >>> if (download()) break;
> >>> }
> >>>
> >>>
> >>> 2.
> >>>
> >>> for (int i = 0; i < 3; i++) {
> >>> if (download()) break;
> >>> exponentialBackoff();
> >>> }
> >>>
> >>>
> >>> Or other wait strategy with a timeout.
> >>> but you can't have:
> >>>
> >>> while (!download()) {
> >>> waitUntilItWorks();
> >>> }
> >>>
> >>>
> >>> So concretely if your down time > reasonnable time you will see the
> >>> failure anyway.
> >>>
> >>> Maven default has a retry (you see it sometimes in the logs) so should
> >>> tolerante a restart or small downtime. Ivy (gradle) uses httpclient as
> >>> well so same kind of config is available but with the same limitation
> >>> by design.
> >>>
> >>> Having a local cache is the best way to avoid these issues but it
> >>> requires at least one green build for dependencies.
> >>>
> >>> What I often saw was to do a pre-build step just resolving
> >>> dependencies+plugins and use a .m2/repository caching.
> >>>
> >>> >
> >>> > Kenn
> >>> >
> >>> >>
> >>> >>
> >>> >> Le 1 déc. 2017 23:27, "Kenneth Knowles" <kl...@google.com> a écrit :
> >>> >>>
> >>> >>> How do you instruct maven or gradle to use that all the time?
> >>> >>>
> >>> >>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau
> >>> >>> <rm...@gmail.com> wrote:
> >>> >>>>
> >>> >>>> If you use wagon http - and not lightweigh - it should have
> retries
> >>> >>>> by
> >>> >>>> default.
> >>> >>>>
> >>> >>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com>
> a
> >>> >>>> écrit :
> >>> >>>>>
> >>> >>>>> Has this ever been brought up in Maven dev community? Perhaps
> they
> >>> >>>>> have
> >>> >>>>> some suggestions. It sounds like a reasonable feature request.
> >>> >>>>>
> >>> >>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com>
> >>> >>>>> wrote:
> >>> >>>>>>
> >>> >>>>>> I've repeatedly searched around for a way to just add proper
> retry
> >>> >>>>>> to
> >>> >>>>>> maven or gradle, and haven't found anything :-(
> >>> >>>>>>
> >>> >>>>>> I had thought that we altered our builds in such a way that the
> >>> >>>>>> .m2
> >>> >>>>>> directory was permitted to survive across builds. True that it
> >>> >>>>>> isn't
> >>> >>>>>> hermetic, precisely, but it is pretty safe to treat as a cache
> of
> >>> >>>>>> immutable
> >>> >>>>>> data, which is no more dangerous than having a caching
> incremental
> >>> >>>>>> build
> >>> >>>>>> system.
> >>> >>>>>>
> >>> >>>>>> Kenn
> >>> >>>>>>
> >>> >>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov
> >>> >>>>>> <ki...@google.com> wrote:
> >>> >>>>>>>
> >>> >>>>>>> Our builds often hit transient Maven network issues, e.g. this
> >>> >>>>>>> one
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>>
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/5331/consoleFull
> >>> >>>>>>>
> >>> >>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on
> project
> >>> >>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve
> >>> >>>>>>> dependencies for
> >>> >>>>>>> project
> >>> >>>>>>>
> >>> >>>>>>>
> org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT:
> >>> >>>>>>> Could not transfer artifact
> org.apache.derby:derby:jar:10.10.2.0
> >>> >>>>>>> from/to
> >>> >>>>>>> central (https://repo.maven.apache.org/maven2): GET request
> of:
> >>> >>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from
> central
> >>> >>>>>>> failed:
> >>> >>>>>>> Connection reset -> [Help 1]
> >>> >>>>>>>
> >>> >>>>>>> It'd be good to increase reliability of our builds.
> >>> >>>>>>> repo.maven.apache.org seems quite unreliable.
> >>> >>>>>>>
> >>> >>>>>>> I tried finding a way to configure Maven to retry such network
> >>> >>>>>>> errors
> >>> >>>>>>> and it appears to be impossible [will be happy if someone
> proves
> >>> >>>>>>> me wrong].
> >>> >>>>>>>
> >>> >>>>>>> Would this issue be resolved if we used multiple mirrors?
> >>> >>>>>>>
> https://maven.apache.org/guides/mini/guide-mirror-settings.html
> >>> >>>>>>> Any other
> >>> >>>>>>> suggestions?
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>
> >>> >>>
> >>> >
> >>
> >>
> >
>
Re: How to cope with Maven transient network issues?
Posted by Romain Manni-Bucau <rm...@gmail.com>.
FYI https://github.com/apache/maven-wagon/pull/37
Romain Manni-Bucau
@rmannibucau | Blog | Old Blog | Github | LinkedIn
2017-12-05 6:54 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
>
>
> Le 5 déc. 2017 06:20, "Eugene Kirpichov" <ki...@google.com> a écrit :
>
>
>
> On Mon, Dec 4, 2017 at 1:45 PM Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>>
>>
>>
>> Le 4 déc. 2017 21:45, "Eugene Kirpichov" <ki...@google.com> a écrit :
>>
>> Romain - as far as I understand, Maven *does* have a retry strategy, but
>> it is a poor retry strategy and there is no way to tweak it. In particular,
>> if it uses https://maven.apache.org/guides/mini/guide-http-settings.html ,
>> that means it uses Apache Http Client 4.1.2 whose default retry settings are
>> extremely conservative
>>
>> https://github.com/apache/httpcomponents-client/blob/4.1.x/httpclient/src/main/java/org/apache/http/impl/client/DefaultHttpRequestRetryHandler.java#L93
>>
>>
>> Right
>>
>>
>>
>> The errors we're hitting with Maven are some of those: "connection
>> refused" and alike.
>>
>>
>>
>> Would a retry guarantee you it works. Should be quite easy to switch wagon
>> with a configurable retry handler instance through mvnwrapper or custom mvn
>> setup and test it but the logic looks sane to me.
>
> Retries indeed don't give any guarantees, but they're very good at reducing
> error rate, and our builds could definitely use some of that.
>
> I tried to figure out how to make Maven use a configurable http wagon with a
> custom retry handler, but I couldn't - if you know more, could you give some
> pointers? I think we should definitely do that.
>
>
> It is not configurable but to ensure it is worth a mvn patch we can add a
> mvn wrapper script where we patch wagon and use that on the CI.
>
>
>
>>
>>
>> If it is on asf repo we must work with infra to fix it rather than working
>> around the clients. If on a repo we dont control we can try to get rid of it
>> maybe?
>
> It happens often enough with the maven central repo repo.maven.apache.org.
> At Beam's (moderate) scale of hundreds of builds per day resolving hundreds
> of dependencies each, even 99.99% availability with a poor retry strategy
> would translate to builds failing regularly.
>
>
> Hmm, this is an asf server and behing should be repo1 and repo2 so either
> there is a huge client issue or the proxy can be better configure. If you
> have some failure logs, do you mind pinging infra with them?
>
> There's also multiple parts that can fail - Maven Central itself, network
> issues between Jenkins and Maven Central, network issues on the Jenkins
> machine itself and so on - I doubt that ASF can fix all of this for us.
> I think this sort of issue *has* to be resolved on the client.
>
>
> Not all kind of failures, connection breakdown during the transfert yes but
> other kind means the server is down so retry shouldnt help.
>
> Let me know if you nees help with wagon patching/testing. Can have a few
> cycles end of the week to give some help.
>
>
>>
>>
>>
>>
>> On Mon, Dec 4, 2017 at 12:15 PM Romain Manni-Bucau <rm...@gmail.com>
>> wrote:
>>>
>>> 2017-12-04 18:58 GMT+01:00 Kenneth Knowles <kl...@google.com>:
>>> > On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau
>>> > <rm...@gmail.com>
>>> > wrote:
>>> >>
>>> >> Need to check but if plugin dependencies were not tuned it should be
>>> >> the
>>> >> default with a retry of 3 IIRC.
>>> >>
>>> >> But arent you sure it was a repo/server issue any client cant solve?
>>> >
>>> >
>>> > Not sure what you mean here. What we need is for mvn and gradle to
>>> > succeed
>>> > even though there are frequent transient failures of the repo. Anything
>>> > that
>>> > makes this happen is success.
>>>
>>> What I meant is you will likely never get it with whatever build
>>> system until you go 100% local which wouldn't make the build
>>> reproducible.
>>>
>>> Concretely you can do
>>>
>>> 1.
>>>
>>> for (int i = 0; i < 3; i++) {
>>> if (download()) break;
>>> }
>>>
>>>
>>> 2.
>>>
>>> for (int i = 0; i < 3; i++) {
>>> if (download()) break;
>>> exponentialBackoff();
>>> }
>>>
>>>
>>> Or other wait strategy with a timeout.
>>> but you can't have:
>>>
>>> while (!download()) {
>>> waitUntilItWorks();
>>> }
>>>
>>>
>>> So concretely if your down time > reasonnable time you will see the
>>> failure anyway.
>>>
>>> Maven default has a retry (you see it sometimes in the logs) so should
>>> tolerante a restart or small downtime. Ivy (gradle) uses httpclient as
>>> well so same kind of config is available but with the same limitation
>>> by design.
>>>
>>> Having a local cache is the best way to avoid these issues but it
>>> requires at least one green build for dependencies.
>>>
>>> What I often saw was to do a pre-build step just resolving
>>> dependencies+plugins and use a .m2/repository caching.
>>>
>>> >
>>> > Kenn
>>> >
>>> >>
>>> >>
>>> >> Le 1 déc. 2017 23:27, "Kenneth Knowles" <kl...@google.com> a écrit :
>>> >>>
>>> >>> How do you instruct maven or gradle to use that all the time?
>>> >>>
>>> >>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau
>>> >>> <rm...@gmail.com> wrote:
>>> >>>>
>>> >>>> If you use wagon http - and not lightweigh - it should have retries
>>> >>>> by
>>> >>>> default.
>>> >>>>
>>> >>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a
>>> >>>> écrit :
>>> >>>>>
>>> >>>>> Has this ever been brought up in Maven dev community? Perhaps they
>>> >>>>> have
>>> >>>>> some suggestions. It sounds like a reasonable feature request.
>>> >>>>>
>>> >>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>> I've repeatedly searched around for a way to just add proper retry
>>> >>>>>> to
>>> >>>>>> maven or gradle, and haven't found anything :-(
>>> >>>>>>
>>> >>>>>> I had thought that we altered our builds in such a way that the
>>> >>>>>> .m2
>>> >>>>>> directory was permitted to survive across builds. True that it
>>> >>>>>> isn't
>>> >>>>>> hermetic, precisely, but it is pretty safe to treat as a cache of
>>> >>>>>> immutable
>>> >>>>>> data, which is no more dangerous than having a caching incremental
>>> >>>>>> build
>>> >>>>>> system.
>>> >>>>>>
>>> >>>>>> Kenn
>>> >>>>>>
>>> >>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov
>>> >>>>>> <ki...@google.com> wrote:
>>> >>>>>>>
>>> >>>>>>> Our builds often hit transient Maven network issues, e.g. this
>>> >>>>>>> one
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/5331/consoleFull
>>> >>>>>>>
>>> >>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
>>> >>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve
>>> >>>>>>> dependencies for
>>> >>>>>>> project
>>> >>>>>>>
>>> >>>>>>> org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT:
>>> >>>>>>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0
>>> >>>>>>> from/to
>>> >>>>>>> central (https://repo.maven.apache.org/maven2): GET request of:
>>> >>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central
>>> >>>>>>> failed:
>>> >>>>>>> Connection reset -> [Help 1]
>>> >>>>>>>
>>> >>>>>>> It'd be good to increase reliability of our builds.
>>> >>>>>>> repo.maven.apache.org seems quite unreliable.
>>> >>>>>>>
>>> >>>>>>> I tried finding a way to configure Maven to retry such network
>>> >>>>>>> errors
>>> >>>>>>> and it appears to be impossible [will be happy if someone proves
>>> >>>>>>> me wrong].
>>> >>>>>>>
>>> >>>>>>> Would this issue be resolved if we used multiple mirrors?
>>> >>>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html
>>> >>>>>>> Any other
>>> >>>>>>> suggestions?
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>
>>> >
>>
>>
>
Re: How to cope with Maven transient network issues?
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Le 5 déc. 2017 06:20, "Eugene Kirpichov" <ki...@google.com> a écrit :
On Mon, Dec 4, 2017 at 1:45 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
>
>
> Le 4 déc. 2017 21:45, "Eugene Kirpichov" <ki...@google.com> a écrit :
>
> Romain - as far as I understand, Maven *does* have a retry strategy, but
> it is a poor retry strategy and there is no way to tweak it. In particular,
> if it uses https://maven.apache.org/guides/mini/guide-http-settings.html ,
> that means it uses Apache Http Client 4.1.2 whose default retry settings
> are extremely conservative
> https://github.com/apache/httpcomponents-client/blob/4.
> 1.x/httpclient/src/main/java/org/apache/http/impl/client/
> DefaultHttpRequestRetryHandler.java#L93
>
>
> Right
>
>
>
> The errors we're hitting with Maven are some of those: "connection
> refused" and alike.
>
>
>
> Would a retry guarantee you it works. Should be quite easy to switch wagon
> with a configurable retry handler instance through mvnwrapper or custom mvn
> setup and test it but the logic looks sane to me.
>
Retries indeed don't give any guarantees, but they're very good at reducing
error rate, and our builds could definitely use some of that.
I tried to figure out how to make Maven use a configurable http wagon with
a custom retry handler, but I couldn't - if you know more, could you give
some pointers? I think we should definitely do that.
It is not configurable but to ensure it is worth a mvn patch we can add a
mvn wrapper script where we patch wagon and use that on the CI.
>
> If it is on asf repo we must work with infra to fix it rather than working
> around the clients. If on a repo we dont control we can try to get rid of
> it maybe?
>
It happens often enough with the maven central repo repo.maven.apache.org.
At Beam's (moderate) scale of hundreds of builds per day resolving hundreds
of dependencies each, even 99.99% availability with a poor retry strategy
would translate to builds failing regularly.
Hmm, this is an asf server and behing should be repo1 and repo2 so either
there is a huge client issue or the proxy can be better configure. If you
have some failure logs, do you mind pinging infra with them?
There's also multiple parts that can fail - Maven Central itself, network
issues between Jenkins and Maven Central, network issues on the Jenkins
machine itself and so on - I doubt that ASF can fix all of this for us.
I think this sort of issue *has* to be resolved on the client.
Not all kind of failures, connection breakdown during the transfert yes but
other kind means the server is down so retry shouldnt help.
Let me know if you nees help with wagon patching/testing. Can have a few
cycles end of the week to give some help.
>
>
>
> On Mon, Dec 4, 2017 at 12:15 PM Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> 2017-12-04 18:58 GMT+01:00 Kenneth Knowles <kl...@google.com>:
>> > On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau <
>> rmannibucau@gmail.com>
>> > wrote:
>> >>
>> >> Need to check but if plugin dependencies were not tuned it should be
>> the
>> >> default with a retry of 3 IIRC.
>> >>
>> >> But arent you sure it was a repo/server issue any client cant solve?
>> >
>> >
>> > Not sure what you mean here. What we need is for mvn and gradle to
>> succeed
>> > even though there are frequent transient failures of the repo. Anything
>> that
>> > makes this happen is success.
>>
>> What I meant is you will likely never get it with whatever build
>> system until you go 100% local which wouldn't make the build
>> reproducible.
>>
>> Concretely you can do
>>
>> 1.
>>
>> for (int i = 0; i < 3; i++) {
>> if (download()) break;
>> }
>>
>>
>> 2.
>>
>> for (int i = 0; i < 3; i++) {
>> if (download()) break;
>> exponentialBackoff();
>> }
>>
>>
>> Or other wait strategy with a timeout.
>> but you can't have:
>>
>> while (!download()) {
>> waitUntilItWorks();
>> }
>>
>>
>> So concretely if your down time > reasonnable time you will see the
>> failure anyway.
>>
>> Maven default has a retry (you see it sometimes in the logs) so should
>> tolerante a restart or small downtime. Ivy (gradle) uses httpclient as
>> well so same kind of config is available but with the same limitation
>> by design.
>>
>> Having a local cache is the best way to avoid these issues but it
>> requires at least one green build for dependencies.
>>
>> What I often saw was to do a pre-build step just resolving
>> dependencies+plugins and use a .m2/repository caching.
>>
>> >
>> > Kenn
>> >
>> >>
>> >>
>> >> Le 1 déc. 2017 23:27, "Kenneth Knowles" <kl...@google.com> a écrit :
>> >>>
>> >>> How do you instruct maven or gradle to use that all the time?
>> >>>
>> >>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau
>> >>> <rm...@gmail.com> wrote:
>> >>>>
>> >>>> If you use wagon http - and not lightweigh - it should have retries
>> by
>> >>>> default.
>> >>>>
>> >>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a
>> >>>> écrit :
>> >>>>>
>> >>>>> Has this ever been brought up in Maven dev community? Perhaps they
>> have
>> >>>>> some suggestions. It sounds like a reasonable feature request.
>> >>>>>
>> >>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com>
>> wrote:
>> >>>>>>
>> >>>>>> I've repeatedly searched around for a way to just add proper retry
>> to
>> >>>>>> maven or gradle, and haven't found anything :-(
>> >>>>>>
>> >>>>>> I had thought that we altered our builds in such a way that the .m2
>> >>>>>> directory was permitted to survive across builds. True that it
>> isn't
>> >>>>>> hermetic, precisely, but it is pretty safe to treat as a cache of
>> immutable
>> >>>>>> data, which is no more dangerous than having a caching incremental
>> build
>> >>>>>> system.
>> >>>>>>
>> >>>>>> Kenn
>> >>>>>>
>> >>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov
>> >>>>>> <ki...@google.com> wrote:
>> >>>>>>>
>> >>>>>>> Our builds often hit transient Maven network issues, e.g. this one
>> >>>>>>>
>> >>>>>>> https://builds.apache.org/job/beam_PostCommit_Java_
>> MavenInstall/5331/consoleFull
>> >>>>>>>
>> >>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
>> >>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve
>> dependencies for
>> >>>>>>> project
>> >>>>>>> org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:
>> jar:2.3.0-SNAPSHOT:
>> >>>>>>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0
>> from/to
>> >>>>>>> central (https://repo.maven.apache.org/maven2): GET request of:
>> >>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from
>> central failed:
>> >>>>>>> Connection reset -> [Help 1]
>> >>>>>>>
>> >>>>>>> It'd be good to increase reliability of our builds.
>> >>>>>>> repo.maven.apache.org seems quite unreliable.
>> >>>>>>>
>> >>>>>>> I tried finding a way to configure Maven to retry such network
>> errors
>> >>>>>>> and it appears to be impossible [will be happy if someone proves
>> me wrong].
>> >>>>>>>
>> >>>>>>> Would this issue be resolved if we used multiple mirrors?
>> >>>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html
>> Any other
>> >>>>>>> suggestions?
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>
>> >
>>
>
>
Re: How to cope with Maven transient network issues?
Posted by Eugene Kirpichov <ki...@google.com>.
On Mon, Dec 4, 2017 at 1:45 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
>
>
> Le 4 déc. 2017 21:45, "Eugene Kirpichov" <ki...@google.com> a écrit :
>
> Romain - as far as I understand, Maven *does* have a retry strategy, but
> it is a poor retry strategy and there is no way to tweak it. In particular,
> if it uses https://maven.apache.org/guides/mini/guide-http-settings.html ,
> that means it uses Apache Http Client 4.1.2 whose default retry settings
> are extremely conservative
>
> https://github.com/apache/httpcomponents-client/blob/4.1.x/httpclient/src/main/java/org/apache/http/impl/client/DefaultHttpRequestRetryHandler.java#L93
>
>
> Right
>
>
>
> The errors we're hitting with Maven are some of those: "connection
> refused" and alike.
>
>
>
> Would a retry guarantee you it works. Should be quite easy to switch wagon
> with a configurable retry handler instance through mvnwrapper or custom mvn
> setup and test it but the logic looks sane to me.
>
Retries indeed don't give any guarantees, but they're very good at reducing
error rate, and our builds could definitely use some of that.
I tried to figure out how to make Maven use a configurable http wagon with
a custom retry handler, but I couldn't - if you know more, could you give
some pointers? I think we should definitely do that.
>
> If it is on asf repo we must work with infra to fix it rather than working
> around the clients. If on a repo we dont control we can try to get rid of
> it maybe?
>
It happens often enough with the maven central repo repo.maven.apache.org.
At Beam's (moderate) scale of hundreds of builds per day resolving hundreds
of dependencies each, even 99.99% availability with a poor retry strategy
would translate to builds failing regularly.
There's also multiple parts that can fail - Maven Central itself, network
issues between Jenkins and Maven Central, network issues on the Jenkins
machine itself and so on - I doubt that ASF can fix all of this for us.
I think this sort of issue *has* to be resolved on the client.
>
>
>
> On Mon, Dec 4, 2017 at 12:15 PM Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> 2017-12-04 18:58 GMT+01:00 Kenneth Knowles <kl...@google.com>:
>> > On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau <
>> rmannibucau@gmail.com>
>> > wrote:
>> >>
>> >> Need to check but if plugin dependencies were not tuned it should be
>> the
>> >> default with a retry of 3 IIRC.
>> >>
>> >> But arent you sure it was a repo/server issue any client cant solve?
>> >
>> >
>> > Not sure what you mean here. What we need is for mvn and gradle to
>> succeed
>> > even though there are frequent transient failures of the repo. Anything
>> that
>> > makes this happen is success.
>>
>> What I meant is you will likely never get it with whatever build
>> system until you go 100% local which wouldn't make the build
>> reproducible.
>>
>> Concretely you can do
>>
>> 1.
>>
>> for (int i = 0; i < 3; i++) {
>> if (download()) break;
>> }
>>
>>
>> 2.
>>
>> for (int i = 0; i < 3; i++) {
>> if (download()) break;
>> exponentialBackoff();
>> }
>>
>>
>> Or other wait strategy with a timeout.
>> but you can't have:
>>
>> while (!download()) {
>> waitUntilItWorks();
>> }
>>
>>
>> So concretely if your down time > reasonnable time you will see the
>> failure anyway.
>>
>> Maven default has a retry (you see it sometimes in the logs) so should
>> tolerante a restart or small downtime. Ivy (gradle) uses httpclient as
>> well so same kind of config is available but with the same limitation
>> by design.
>>
>> Having a local cache is the best way to avoid these issues but it
>> requires at least one green build for dependencies.
>>
>> What I often saw was to do a pre-build step just resolving
>> dependencies+plugins and use a .m2/repository caching.
>>
>> >
>> > Kenn
>> >
>> >>
>> >>
>> >> Le 1 déc. 2017 23:27, "Kenneth Knowles" <kl...@google.com> a écrit :
>> >>>
>> >>> How do you instruct maven or gradle to use that all the time?
>> >>>
>> >>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau
>> >>> <rm...@gmail.com> wrote:
>> >>>>
>> >>>> If you use wagon http - and not lightweigh - it should have retries
>> by
>> >>>> default.
>> >>>>
>> >>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a
>> >>>> écrit :
>> >>>>>
>> >>>>> Has this ever been brought up in Maven dev community? Perhaps they
>> have
>> >>>>> some suggestions. It sounds like a reasonable feature request.
>> >>>>>
>> >>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com>
>> wrote:
>> >>>>>>
>> >>>>>> I've repeatedly searched around for a way to just add proper retry
>> to
>> >>>>>> maven or gradle, and haven't found anything :-(
>> >>>>>>
>> >>>>>> I had thought that we altered our builds in such a way that the .m2
>> >>>>>> directory was permitted to survive across builds. True that it
>> isn't
>> >>>>>> hermetic, precisely, but it is pretty safe to treat as a cache of
>> immutable
>> >>>>>> data, which is no more dangerous than having a caching incremental
>> build
>> >>>>>> system.
>> >>>>>>
>> >>>>>> Kenn
>> >>>>>>
>> >>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov
>> >>>>>> <ki...@google.com> wrote:
>> >>>>>>>
>> >>>>>>> Our builds often hit transient Maven network issues, e.g. this one
>> >>>>>>>
>> >>>>>>>
>> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/5331/consoleFull
>> >>>>>>>
>> >>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
>> >>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve
>> dependencies for
>> >>>>>>> project
>> >>>>>>>
>> org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT:
>> >>>>>>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0
>> from/to
>> >>>>>>> central (https://repo.maven.apache.org/maven2): GET request of:
>> >>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from
>> central failed:
>> >>>>>>> Connection reset -> [Help 1]
>> >>>>>>>
>> >>>>>>> It'd be good to increase reliability of our builds.
>> >>>>>>> repo.maven.apache.org seems quite unreliable.
>> >>>>>>>
>> >>>>>>> I tried finding a way to configure Maven to retry such network
>> errors
>> >>>>>>> and it appears to be impossible [will be happy if someone proves
>> me wrong].
>> >>>>>>>
>> >>>>>>> Would this issue be resolved if we used multiple mirrors?
>> >>>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html
>> Any other
>> >>>>>>> suggestions?
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>
>> >
>>
>
>
Re: How to cope with Maven transient network issues?
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Le 4 déc. 2017 21:45, "Eugene Kirpichov" <ki...@google.com> a écrit :
Romain - as far as I understand, Maven *does* have a retry strategy, but it
is a poor retry strategy and there is no way to tweak it. In particular, if
it uses https://maven.apache.org/guides/mini/guide-http-settings.html ,
that means it uses Apache Http Client 4.1.2 whose default retry settings
are extremely conservative
https://github.com/apache/httpcomponents-client/blob/4.
1.x/httpclient/src/main/java/org/apache/http/impl/client/
DefaultHttpRequestRetryHandler.java#L93
Right
The errors we're hitting with Maven are some of those: "connection refused"
and alike.
Would a retry guarantee you it works. Should be quite easy to switch wagon
with a configurable retry handler instance through mvnwrapper or custom mvn
setup and test it but the logic looks sane to me.
If it is on asf repo we must work with infra to fix it rather than working
around the clients. If on a repo we dont control we can try to get rid of
it maybe?
On Mon, Dec 4, 2017 at 12:15 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> 2017-12-04 18:58 GMT+01:00 Kenneth Knowles <kl...@google.com>:
> > On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau <
> rmannibucau@gmail.com>
> > wrote:
> >>
> >> Need to check but if plugin dependencies were not tuned it should be the
> >> default with a retry of 3 IIRC.
> >>
> >> But arent you sure it was a repo/server issue any client cant solve?
> >
> >
> > Not sure what you mean here. What we need is for mvn and gradle to
> succeed
> > even though there are frequent transient failures of the repo. Anything
> that
> > makes this happen is success.
>
> What I meant is you will likely never get it with whatever build
> system until you go 100% local which wouldn't make the build
> reproducible.
>
> Concretely you can do
>
> 1.
>
> for (int i = 0; i < 3; i++) {
> if (download()) break;
> }
>
>
> 2.
>
> for (int i = 0; i < 3; i++) {
> if (download()) break;
> exponentialBackoff();
> }
>
>
> Or other wait strategy with a timeout.
> but you can't have:
>
> while (!download()) {
> waitUntilItWorks();
> }
>
>
> So concretely if your down time > reasonnable time you will see the
> failure anyway.
>
> Maven default has a retry (you see it sometimes in the logs) so should
> tolerante a restart or small downtime. Ivy (gradle) uses httpclient as
> well so same kind of config is available but with the same limitation
> by design.
>
> Having a local cache is the best way to avoid these issues but it
> requires at least one green build for dependencies.
>
> What I often saw was to do a pre-build step just resolving
> dependencies+plugins and use a .m2/repository caching.
>
> >
> > Kenn
> >
> >>
> >>
> >> Le 1 déc. 2017 23:27, "Kenneth Knowles" <kl...@google.com> a écrit :
> >>>
> >>> How do you instruct maven or gradle to use that all the time?
> >>>
> >>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau
> >>> <rm...@gmail.com> wrote:
> >>>>
> >>>> If you use wagon http - and not lightweigh - it should have retries by
> >>>> default.
> >>>>
> >>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a
> >>>> écrit :
> >>>>>
> >>>>> Has this ever been brought up in Maven dev community? Perhaps they
> have
> >>>>> some suggestions. It sounds like a reasonable feature request.
> >>>>>
> >>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com>
> wrote:
> >>>>>>
> >>>>>> I've repeatedly searched around for a way to just add proper retry
> to
> >>>>>> maven or gradle, and haven't found anything :-(
> >>>>>>
> >>>>>> I had thought that we altered our builds in such a way that the .m2
> >>>>>> directory was permitted to survive across builds. True that it isn't
> >>>>>> hermetic, precisely, but it is pretty safe to treat as a cache of
> immutable
> >>>>>> data, which is no more dangerous than having a caching incremental
> build
> >>>>>> system.
> >>>>>>
> >>>>>> Kenn
> >>>>>>
> >>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov
> >>>>>> <ki...@google.com> wrote:
> >>>>>>>
> >>>>>>> Our builds often hit transient Maven network issues, e.g. this one
> >>>>>>>
> >>>>>>> https://builds.apache.org/job/beam_PostCommit_Java_
> MavenInstall/5331/consoleFull
> >>>>>>>
> >>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
> >>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve
> dependencies for
> >>>>>>> project
> >>>>>>> org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:
> jar:2.3.0-SNAPSHOT:
> >>>>>>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0
> from/to
> >>>>>>> central (https://repo.maven.apache.org/maven2): GET request of:
> >>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central
> failed:
> >>>>>>> Connection reset -> [Help 1]
> >>>>>>>
> >>>>>>> It'd be good to increase reliability of our builds.
> >>>>>>> repo.maven.apache.org seems quite unreliable.
> >>>>>>>
> >>>>>>> I tried finding a way to configure Maven to retry such network
> errors
> >>>>>>> and it appears to be impossible [will be happy if someone proves
> me wrong].
> >>>>>>>
> >>>>>>> Would this issue be resolved if we used multiple mirrors?
> >>>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html
> Any other
> >>>>>>> suggestions?
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >
>
Re: How to cope with Maven transient network issues?
Posted by Eugene Kirpichov <ki...@google.com>.
Romain - as far as I understand, Maven *does* have a retry strategy, but it
is a poor retry strategy and there is no way to tweak it. In particular, if
it uses https://maven.apache.org/guides/mini/guide-http-settings.html ,
that means it uses Apache Http Client 4.1.2 whose default retry settings
are extremely conservative
https://github.com/apache/httpcomponents-client/blob/4.1.x/httpclient/src/main/java/org/apache/http/impl/client/DefaultHttpRequestRetryHandler.java#L93
The errors we're hitting with Maven are some of those: "connection refused"
and alike.
On Mon, Dec 4, 2017 at 12:15 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> 2017-12-04 18:58 GMT+01:00 Kenneth Knowles <kl...@google.com>:
> > On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau <
> rmannibucau@gmail.com>
> > wrote:
> >>
> >> Need to check but if plugin dependencies were not tuned it should be the
> >> default with a retry of 3 IIRC.
> >>
> >> But arent you sure it was a repo/server issue any client cant solve?
> >
> >
> > Not sure what you mean here. What we need is for mvn and gradle to
> succeed
> > even though there are frequent transient failures of the repo. Anything
> that
> > makes this happen is success.
>
> What I meant is you will likely never get it with whatever build
> system until you go 100% local which wouldn't make the build
> reproducible.
>
> Concretely you can do
>
> 1.
>
> for (int i = 0; i < 3; i++) {
> if (download()) break;
> }
>
>
> 2.
>
> for (int i = 0; i < 3; i++) {
> if (download()) break;
> exponentialBackoff();
> }
>
>
> Or other wait strategy with a timeout.
> but you can't have:
>
> while (!download()) {
> waitUntilItWorks();
> }
>
>
> So concretely if your down time > reasonnable time you will see the
> failure anyway.
>
> Maven default has a retry (you see it sometimes in the logs) so should
> tolerante a restart or small downtime. Ivy (gradle) uses httpclient as
> well so same kind of config is available but with the same limitation
> by design.
>
> Having a local cache is the best way to avoid these issues but it
> requires at least one green build for dependencies.
>
> What I often saw was to do a pre-build step just resolving
> dependencies+plugins and use a .m2/repository caching.
>
> >
> > Kenn
> >
> >>
> >>
> >> Le 1 déc. 2017 23:27, "Kenneth Knowles" <kl...@google.com> a écrit :
> >>>
> >>> How do you instruct maven or gradle to use that all the time?
> >>>
> >>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau
> >>> <rm...@gmail.com> wrote:
> >>>>
> >>>> If you use wagon http - and not lightweigh - it should have retries by
> >>>> default.
> >>>>
> >>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a
> >>>> écrit :
> >>>>>
> >>>>> Has this ever been brought up in Maven dev community? Perhaps they
> have
> >>>>> some suggestions. It sounds like a reasonable feature request.
> >>>>>
> >>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com>
> wrote:
> >>>>>>
> >>>>>> I've repeatedly searched around for a way to just add proper retry
> to
> >>>>>> maven or gradle, and haven't found anything :-(
> >>>>>>
> >>>>>> I had thought that we altered our builds in such a way that the .m2
> >>>>>> directory was permitted to survive across builds. True that it isn't
> >>>>>> hermetic, precisely, but it is pretty safe to treat as a cache of
> immutable
> >>>>>> data, which is no more dangerous than having a caching incremental
> build
> >>>>>> system.
> >>>>>>
> >>>>>> Kenn
> >>>>>>
> >>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov
> >>>>>> <ki...@google.com> wrote:
> >>>>>>>
> >>>>>>> Our builds often hit transient Maven network issues, e.g. this one
> >>>>>>>
> >>>>>>>
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/5331/consoleFull
> >>>>>>>
> >>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
> >>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve
> dependencies for
> >>>>>>> project
> >>>>>>>
> org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT:
> >>>>>>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0
> from/to
> >>>>>>> central (https://repo.maven.apache.org/maven2): GET request of:
> >>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central
> failed:
> >>>>>>> Connection reset -> [Help 1]
> >>>>>>>
> >>>>>>> It'd be good to increase reliability of our builds.
> >>>>>>> repo.maven.apache.org seems quite unreliable.
> >>>>>>>
> >>>>>>> I tried finding a way to configure Maven to retry such network
> errors
> >>>>>>> and it appears to be impossible [will be happy if someone proves
> me wrong].
> >>>>>>>
> >>>>>>> Would this issue be resolved if we used multiple mirrors?
> >>>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html
> Any other
> >>>>>>> suggestions?
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >
>
Re: How to cope with Maven transient network issues?
Posted by Romain Manni-Bucau <rm...@gmail.com>.
2017-12-04 18:58 GMT+01:00 Kenneth Knowles <kl...@google.com>:
> On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>>
>> Need to check but if plugin dependencies were not tuned it should be the
>> default with a retry of 3 IIRC.
>>
>> But arent you sure it was a repo/server issue any client cant solve?
>
>
> Not sure what you mean here. What we need is for mvn and gradle to succeed
> even though there are frequent transient failures of the repo. Anything that
> makes this happen is success.
What I meant is you will likely never get it with whatever build
system until you go 100% local which wouldn't make the build
reproducible.
Concretely you can do
1.
for (int i = 0; i < 3; i++) {
if (download()) break;
}
2.
for (int i = 0; i < 3; i++) {
if (download()) break;
exponentialBackoff();
}
Or other wait strategy with a timeout.
but you can't have:
while (!download()) {
waitUntilItWorks();
}
So concretely if your down time > reasonnable time you will see the
failure anyway.
Maven default has a retry (you see it sometimes in the logs) so should
tolerante a restart or small downtime. Ivy (gradle) uses httpclient as
well so same kind of config is available but with the same limitation
by design.
Having a local cache is the best way to avoid these issues but it
requires at least one green build for dependencies.
What I often saw was to do a pre-build step just resolving
dependencies+plugins and use a .m2/repository caching.
>
> Kenn
>
>>
>>
>> Le 1 déc. 2017 23:27, "Kenneth Knowles" <kl...@google.com> a écrit :
>>>
>>> How do you instruct maven or gradle to use that all the time?
>>>
>>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau
>>> <rm...@gmail.com> wrote:
>>>>
>>>> If you use wagon http - and not lightweigh - it should have retries by
>>>> default.
>>>>
>>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a
>>>> écrit :
>>>>>
>>>>> Has this ever been brought up in Maven dev community? Perhaps they have
>>>>> some suggestions. It sounds like a reasonable feature request.
>>>>>
>>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com> wrote:
>>>>>>
>>>>>> I've repeatedly searched around for a way to just add proper retry to
>>>>>> maven or gradle, and haven't found anything :-(
>>>>>>
>>>>>> I had thought that we altered our builds in such a way that the .m2
>>>>>> directory was permitted to survive across builds. True that it isn't
>>>>>> hermetic, precisely, but it is pretty safe to treat as a cache of immutable
>>>>>> data, which is no more dangerous than having a caching incremental build
>>>>>> system.
>>>>>>
>>>>>> Kenn
>>>>>>
>>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov
>>>>>> <ki...@google.com> wrote:
>>>>>>>
>>>>>>> Our builds often hit transient Maven network issues, e.g. this one
>>>>>>>
>>>>>>> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/5331/consoleFull
>>>>>>>
>>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
>>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve dependencies for
>>>>>>> project
>>>>>>> org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT:
>>>>>>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0 from/to
>>>>>>> central (https://repo.maven.apache.org/maven2): GET request of:
>>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central failed:
>>>>>>> Connection reset -> [Help 1]
>>>>>>>
>>>>>>> It'd be good to increase reliability of our builds.
>>>>>>> repo.maven.apache.org seems quite unreliable.
>>>>>>>
>>>>>>> I tried finding a way to configure Maven to retry such network errors
>>>>>>> and it appears to be impossible [will be happy if someone proves me wrong].
>>>>>>>
>>>>>>> Would this issue be resolved if we used multiple mirrors?
>>>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html Any other
>>>>>>> suggestions?
>>>>>>
>>>>>>
>>>>>
>>>
>
Re: How to cope with Maven transient network issues?
Posted by Kenneth Knowles <kl...@google.com>.
On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Need to check but if plugin dependencies were not tuned it should be the
> default with a retry of 3 IIRC.
>
> But arent you sure it was a repo/server issue any client cant solve?
>
Not sure what you mean here. What we need is for mvn and gradle to succeed
even though there are frequent transient failures of the repo. Anything
that makes this happen is success.
Kenn
>
> Le 1 déc. 2017 23:27, "Kenneth Knowles" <kl...@google.com> a écrit :
>
>> How do you instruct maven or gradle to use that all the time?
>>
>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau <rmannibucau@gmail.com
>> > wrote:
>>
>>> If you use wagon http - and not lightweigh - it should have retries by
>>> default.
>>>
>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a
>>> écrit :
>>>
>>>> Has this ever been brought up in Maven dev community? Perhaps they have
>>>> some suggestions. It sounds like a reasonable feature request.
>>>>
>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com> wrote:
>>>>
>>>>> I've repeatedly searched around for a way to just add proper retry to
>>>>> maven or gradle, and haven't found anything :-(
>>>>>
>>>>> I had thought that we altered our builds in such a way that the .m2
>>>>> directory was permitted to survive across builds. True that it isn't
>>>>> hermetic, precisely, but it is pretty safe to treat as a cache of immutable
>>>>> data, which is no more dangerous than having a caching incremental build
>>>>> system.
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov <
>>>>> kirpichov@google.com> wrote:
>>>>>
>>>>>> Our builds often hit transient Maven network issues, e.g. this one
>>>>>> https://builds.apache.org/job/beam_PostCommit_Java_MavenInst
>>>>>> all/5331/consoleFull
>>>>>>
>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve
>>>>>> dependencies for project org.apache.beam:beam-sdks-java
>>>>>> -io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT: Could not transfer
>>>>>> artifact org.apache.derby:derby:jar:10.10.2.0 from/to central (
>>>>>> https://repo.maven.apache.org/maven2): GET request of:
>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central
>>>>>> failed: Connection reset -> [Help 1]
>>>>>>
>>>>>> It'd be good to increase reliability of our builds.
>>>>>> repo.maven.apache.org seems quite unreliable.
>>>>>>
>>>>>> I tried finding a way to configure Maven to retry such network errors
>>>>>> and it appears to be impossible [will be happy if someone proves me wrong].
>>>>>>
>>>>>> Would this issue be resolved if we used multiple mirrors?
>>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html Any
>>>>>> other suggestions?
>>>>>>
>>>>>
>>>>>
>>>>
>>
Re: How to cope with Maven transient network issues?
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Need to check but if plugin dependencies were not tuned it should be the
default with a retry of 3 IIRC.
But arent you sure it was a repo/server issue any client cant solve?
Le 1 déc. 2017 23:27, "Kenneth Knowles" <kl...@google.com> a écrit :
> How do you instruct maven or gradle to use that all the time?
>
> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> If you use wagon http - and not lightweigh - it should have retries by
>> default.
>>
>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a
>> écrit :
>>
>>> Has this ever been brought up in Maven dev community? Perhaps they have
>>> some suggestions. It sounds like a reasonable feature request.
>>>
>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com> wrote:
>>>
>>>> I've repeatedly searched around for a way to just add proper retry to
>>>> maven or gradle, and haven't found anything :-(
>>>>
>>>> I had thought that we altered our builds in such a way that the .m2
>>>> directory was permitted to survive across builds. True that it isn't
>>>> hermetic, precisely, but it is pretty safe to treat as a cache of immutable
>>>> data, which is no more dangerous than having a caching incremental build
>>>> system.
>>>>
>>>> Kenn
>>>>
>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov <kirpichov@google.com
>>>> > wrote:
>>>>
>>>>> Our builds often hit transient Maven network issues, e.g. this one
>>>>> https://builds.apache.org/job/beam_PostCommit_Java_MavenInst
>>>>> all/5331/consoleFull
>>>>>
>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve dependencies
>>>>> for project org.apache.beam:beam-sdks-java
>>>>> -io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT: Could not transfer
>>>>> artifact org.apache.derby:derby:jar:10.10.2.0 from/to central (
>>>>> https://repo.maven.apache.org/maven2): GET request of:
>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central
>>>>> failed: Connection reset -> [Help 1]
>>>>>
>>>>> It'd be good to increase reliability of our builds.
>>>>> repo.maven.apache.org seems quite unreliable.
>>>>>
>>>>> I tried finding a way to configure Maven to retry such network errors
>>>>> and it appears to be impossible [will be happy if someone proves me wrong].
>>>>>
>>>>> Would this issue be resolved if we used multiple mirrors?
>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html Any
>>>>> other suggestions?
>>>>>
>>>>
>>>>
>>>
>
Re: How to cope with Maven transient network issues?
Posted by Kenneth Knowles <kl...@google.com>.
How do you instruct maven or gradle to use that all the time?
On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau <rm...@gmail.com>
wrote:
> If you use wagon http - and not lightweigh - it should have retries by
> default.
>
> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a
> écrit :
>
>> Has this ever been brought up in Maven dev community? Perhaps they have
>> some suggestions. It sounds like a reasonable feature request.
>>
>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com> wrote:
>>
>>> I've repeatedly searched around for a way to just add proper retry to
>>> maven or gradle, and haven't found anything :-(
>>>
>>> I had thought that we altered our builds in such a way that the .m2
>>> directory was permitted to survive across builds. True that it isn't
>>> hermetic, precisely, but it is pretty safe to treat as a cache of immutable
>>> data, which is no more dangerous than having a caching incremental build
>>> system.
>>>
>>> Kenn
>>>
>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov <ki...@google.com>
>>> wrote:
>>>
>>>> Our builds often hit transient Maven network issues, e.g. this one
>>>> https://builds.apache.org/job/beam_PostCommit_Java_MavenInst
>>>> all/5331/consoleFull
>>>>
>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve dependencies
>>>> for project org.apache.beam:beam-sdks-java
>>>> -io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT: Could not transfer
>>>> artifact org.apache.derby:derby:jar:10.10.2.0 from/to central (
>>>> https://repo.maven.apache.org/maven2): GET request of:
>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central
>>>> failed: Connection reset -> [Help 1]
>>>>
>>>> It'd be good to increase reliability of our builds.
>>>> repo.maven.apache.org seems quite unreliable.
>>>>
>>>> I tried finding a way to configure Maven to retry such network errors
>>>> and it appears to be impossible [will be happy if someone proves me wrong].
>>>>
>>>> Would this issue be resolved if we used multiple mirrors?
>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html Any
>>>> other suggestions?
>>>>
>>>
>>>
>>
Re: How to cope with Maven transient network issues?
Posted by Romain Manni-Bucau <rm...@gmail.com>.
If you use wagon http - and not lightweigh - it should have retries by
default.
Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <va...@google.com> a écrit :
> Has this ever been brought up in Maven dev community? Perhaps they have
> some suggestions. It sounds like a reasonable feature request.
>
> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com> wrote:
>
>> I've repeatedly searched around for a way to just add proper retry to
>> maven or gradle, and haven't found anything :-(
>>
>> I had thought that we altered our builds in such a way that the .m2
>> directory was permitted to survive across builds. True that it isn't
>> hermetic, precisely, but it is pretty safe to treat as a cache of immutable
>> data, which is no more dangerous than having a caching incremental build
>> system.
>>
>> Kenn
>>
>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov <ki...@google.com>
>> wrote:
>>
>>> Our builds often hit transient Maven network issues, e.g. this one
>>> https://builds.apache.org/job/beam_PostCommit_Java_MavenInst
>>> all/5331/consoleFull
>>>
>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve dependencies
>>> for project org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT:
>>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0
>>> from/to central (https://repo.maven.apache.org/maven2): GET request of:
>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central
>>> failed: Connection reset -> [Help 1]
>>>
>>> It'd be good to increase reliability of our builds.
>>> repo.maven.apache.org seems quite unreliable.
>>>
>>> I tried finding a way to configure Maven to retry such network errors
>>> and it appears to be impossible [will be happy if someone proves me wrong].
>>>
>>> Would this issue be resolved if we used multiple mirrors?
>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html Any
>>> other suggestions?
>>>
>>
>>
>
Re: How to cope with Maven transient network issues?
Posted by Valentyn Tymofieiev <va...@google.com>.
Has this ever been brought up in Maven dev community? Perhaps they have
some suggestions. It sounds like a reasonable feature request.
On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <kl...@google.com> wrote:
> I've repeatedly searched around for a way to just add proper retry to
> maven or gradle, and haven't found anything :-(
>
> I had thought that we altered our builds in such a way that the .m2
> directory was permitted to survive across builds. True that it isn't
> hermetic, precisely, but it is pretty safe to treat as a cache of immutable
> data, which is no more dangerous than having a caching incremental build
> system.
>
> Kenn
>
> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov <ki...@google.com>
> wrote:
>
>> Our builds often hit transient Maven network issues, e.g. this one
>> https://builds.apache.org/job/beam_PostCommit_Java_MavenInst
>> all/5331/consoleFull
>>
>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project
>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve dependencies
>> for project org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT:
>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0 from/to
>> central (https://repo.maven.apache.org/maven2): GET request of:
>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central
>> failed: Connection reset -> [Help 1]
>>
>> It'd be good to increase reliability of our builds. repo.maven.apache.org
>> seems quite unreliable.
>>
>> I tried finding a way to configure Maven to retry such network errors and
>> it appears to be impossible [will be happy if someone proves me wrong].
>>
>> Would this issue be resolved if we used multiple mirrors?
>> https://maven.apache.org/guides/mini/guide-mirror-settings.html Any
>> other suggestions?
>>
>
>