You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/10/27 05:56:20 UTC

[GitHub] [pulsar] lhotari opened a new pull request #8386: [Issue 6231][Build] Configure maven to retry downloads 3 times

lhotari opened a new pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386


   Fixes #6231
   
   ### Motivation
   
   See #6231 . This is a common problem in Pulsar CI running on Github Actions. 
   
   ### Modifications
   
   - use `${maven.projectBasedir}/.mvn/maven.config` config file to configure
     options that should be set for all mvn command executions (since Maven
     3.3.1+)
     - see https://maven.apache.org/configure.html#mvn-maven-config-file


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717193698


   @lhotari I see the same problem in BookKeeper repository
   https://github.com/apache/bookkeeper/pull/1901/checks?check_run_id=1314277589
   
   do you mind porting this PR to Apache BooKeeper repository as well ?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717394497


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717362047


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717259826


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717371606


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven to retry downloads 3 times

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717086317


   There is an issue https://issues.apache.org/jira/browse/WAGON-486 , besides disabling the connection pool completely, it would be possible to tweak [the `maven.wagon.httpconnectionManager.ttlSeconds` setting](
   https://github.com/apache/maven-wagon/blob/a7c8e3470dd968961e87a7cb9a3829d3bec77383/wagon-providers/wagon-http-shared/src/main/java/org/apache/maven/wagon/shared/http/AbstractHttpClientWagon.java#L297-L305).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717321769


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717250572


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven to retry downloads 3 times

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717068810


   >But it would be better to apply this change to mvn invocation in github actions files.
   If we commit this file the change will apply to every local environment and to other CI systems (for people who test Pulsar on their own CI systems)
   
   @eolivelli Eventually I will go with this proposal, so that the settings apply only for github actions. It turns out that `-Dmaven.wagon.http.retryHandler.count=3` doesn't help and the correct solution seems to be to use `-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false`. This is originally mentioned in an Azure related issue: https://developercommunity.visualstudio.com/content/problem/357696/maven-project-build-failing-with-connection-reset.html . (linked from the SO Q&A https://stackoverflow.com/a/58769870)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717136052


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on a change in pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#discussion_r512608064



##########
File path: .github/workflows/ci-cpp.yaml
##########
@@ -26,6 +26,9 @@ on:
     branches:
       - branch-*
 
+env:
+  MAVEN_OPTS: -Dmaven.wagon.httpconnectionManager.ttlSeconds=25 -Dmaven.wagon.http.retryHandler.count=3

Review comment:
       shall we use 240 seconds as you written in your comment ?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717108246


   I also found https://issues.apache.org/jira/browse/WAGON-545 which is a close match to the issue we are having.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-718397416


   I now noticed that there's https://github.com/actions/virtual-environments/issues/1499 . We are covered with the changes we are now using. That issue gives the wrong advice (disabling the connection pool and also configuring the TTL for the pool doesn't make sense) and I added a comment there.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli edited a comment on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
eolivelli edited a comment on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717193698


   @lhotari I see the same problem in BookKeeper repository
   https://github.com/apache/bookkeeper/pull/1901/checks?check_run_id=1314277589
   
   > Error:  Failed to execute goal on project bookkeeper-common: Could not resolve dependencies for project org.apache.bookkeeper:bookkeeper-common:jar:4.12.0-SNAPSHOT: Could not transfer artifact org.jctools:jctools-core:jar:2.1.2 from/to central (https://repo.maven.apache.org/maven2): Transfer failed for https://repo.maven.apache.org/maven2/org/jctools/jctools-core/2.1.2/jctools-core-2.1.2.jar: Connection reset -> [Help 1]
   
   do you mind porting this PR to Apache BooKeeper repository as well ?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on a change in pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on a change in pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#discussion_r512627254



##########
File path: .github/workflows/ci-cpp.yaml
##########
@@ -26,6 +26,9 @@ on:
     branches:
       - branch-*
 
+env:
+  MAVEN_OPTS: -Dmaven.wagon.httpconnectionManager.ttlSeconds=25 -Dmaven.wagon.http.retryHandler.count=3

Review comment:
       I was thinking about that. However I'm not exactly sure if the GitHub Flows environment is an Azure environment as referenced in WAGON-545.
   
   The reason for the 25 second TTL is to ensure that it's shorter than any common firewall or NAT timeout. Some NATs have a 30 second idle timeout although that is very rare.
   
   There shouldn't be harm in using the 25 second TTL since the connection pool will be able to pool connections well with a 25 second TTL. I don't see a reason to extend it to 240 seconds since there's not really a benefit.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] merlimat merged pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
merlimat merged pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari edited a comment on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717196046


   > do you mind porting this PR to Apache BooKeeper repository as well ?
   
   @eolivelli sure, I can do that. It's https://github.com/apache/bookkeeper/pull/2460 .


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717423591


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari edited a comment on pull request #8386: [Issue 6231][Build] Configure maven to retry downloads 3 times

Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717063550


   Documentation for `maven.wagon.http.retryHandler.count` can be found in https://maven.apache.org/wagon/wagon-providers/wagon-http/ . 
   
   It mentions "Any retry handler can only react to exceptions when executing the request and receiving the response head. It will not salvage in-flight failures of ongoing response body streams." . It seems that it won't be useful for retrying downloads.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari removed a comment on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari removed a comment on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717035218


   >But it would be better to apply this change to mvn invocation in github actions files.
   If we commit this file the change will apply to every local environment and to other CI systems (for people who test Pulsar on their own CI systems)
   
   That was intentional to apply this setting to every environment since I don't see this setting (`-Dmaven.wagon.http.retryHandler.count=3`) causing any harm in any environment. Therefore I suggest that we go forward with the proposed PR changes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven to retry downloads 3 times

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717035218


   >But it would be better to apply this change to mvn invocation in github actions files.
   If we commit this file the change will apply to every local environment and to other CI systems (for people who test Pulsar on their own CI systems)
   
   That was intentional to apply this setting to every environment since I don't see this setting (`-Dmaven.wagon.http.retryHandler.count=3`) causing any harm in any environment. Therefore I suggest that we go forward with the proposed PR changes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari removed a comment on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari removed a comment on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-718397416


   I now noticed that there's https://github.com/actions/virtual-environments/issues/1499 . We are covered with the changes we are now using. That issue gives the wrong advice (disabling the connection pool and also configuring the TTL for the pool doesn't make sense) and I added a comment there.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717214350


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717189283


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven to retry downloads 3 times

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717062226


   It seems like the setting `-Dmaven.wagon.http.retryHandler.count=3` isn't effective for the download problem since one of the jobs for this PR failed: https://github.com/apache/pulsar/pull/8386/checks?check_run_id=1313003063


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717364412


   btw. Github Actions / Flows do run on Azure. 
   https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#cloud-hosts-for-github-hosted-runners
   > GitHub hosts Linux and Windows runners on Standard_DS2_v2 virtual machines in Microsoft Azure with the GitHub Actions runner application installed. The GitHub-hosted runner application is a fork of the Azure Pipelines Agent. Inbound ICMP packets are blocked for all Azure virtual machines, so ping or traceroute commands might not work. For more information about the Standard_DS2_v2 machine resources, see "Dv2 and DSv2-series" in the Microsoft Azure documentation.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717196046


   > do you mind porting this PR to Apache BooKeeper repository as well ?
   
   @eolivelli sure, I can do that.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven to retry downloads 3 times

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717063550


   Documentation for `maven.wagon.http.retryHandler.count` can be found in https://maven.apache.org/wagon/wagon-providers/wagon-http/


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari edited a comment on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-717108246


   I also found https://issues.apache.org/jira/browse/WAGON-545 which is a close match to the issue we are having.
   In the [last comment](https://issues.apache.org/jira/browse/WAGON-545?focusedCommentId=16755035&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16755035), it says "Azure users shall set the TTL to 240 seconds or less."


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #8386: [Issue 6231][Build] Configure maven http connection pool setting and maven http retry setting in Github Flows

Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #8386:
URL: https://github.com/apache/pulsar/pull/8386#issuecomment-718402722


   Actually, I just now noticed yet another connection reset when using `MAVEN_OPTS=-Dmaven.wagon.httpconnectionManager.ttlSeconds=25 -Dmaven.wagon.http.retryHandler.count=3`
   https://github.com/apache/pulsar/runs/1325014611?check_suite_focus=true
   
   Perhaps we need to switch to `-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false` and disable the connection pool completely. That's what others are doing in https://github.com/actions/virtual-environments/issues/1499


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org