You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "EnricoMi (via GitHub)" <gi...@apache.org> on 2023/10/27 11:18:17 UTC
[PR] [BUILD] Retry mvn deploy [spark]
EnricoMi opened a new pull request, #43559:
URL: https://github.com/apache/spark/pull/43559
### What changes were proposed in this pull request?
Retry uploading artifacts to the Apache repository as often as possible (default is `1`).
### Why are the changes needed?
It is common to see `408 Request Timeout` and `502 Proxy Error` when deploying artifacts to Apache snapshot repository:
https://github.com/apache/spark/actions/runs/6437635317
> 2023-10-07T01:09:51.1719360Z [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:3.0.0-M1:deploy (default-deploy) on project spark-streaming_2.13: ArtifactDeployerException: Failed to deploy artifacts: Could not transfer artifact org.apache.spark:spark-streaming_2.13:jar:tests:3.3.4-20231007.005815-57 from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): transfer failed for https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-streaming_2.13/3.3.4-SNAPSHOT/spark-streaming_2.13-3.3.4-20231007.005815-57-tests.jar, status: 502 Proxy Error -> [Help 1]
> 2023-10-07T01:11:48.5651501Z [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:3.0.0:deploy (default-deploy) on project spark-connect-common_2.12: Failed to deploy artifacts: Could not transfer artifact org.apache.spark:spark-connect-common_2.12:xml:cyclonedx:3.4.2-20231007.001102-103 from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): transfer failed for https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-connect-common_2.12/3.4.2-SNAPSHOT/spark-connect-common_2.12-3.4.2-20231007.001102-103-cyclonedx.xml, status: 408 Request Timeout -> [Help 1]
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Locally: `./buil/mvn deploy`:
```
[INFO] --- deploy:3.1.1:deploy (default-deploy) @ spark-parent_2.13 ---
Downloading from apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/maven-metadata.xml
Downloaded from apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/maven-metadata.xml (1.3 kB at 3.6 kB/s)
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92.pom
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-tests.jar
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-cyclonedx.xml
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-cyclonedx.json
[WARNING] Encountered issue during deployment: Failed to deploy artifacts: Could not transfer artifact org.apache.spark:spark-parent_2.13:pom:4.0.0-20231027.110850-92 from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): status code: 401, reason phrase: Unauthorized (401)
[INFO] Retrying deployment attempt 2 of 10
...
[INFO] Retrying deployment attempt 10 of 10
Downloading from apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/maven-metadata.xml
Downloaded from apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/maven-metadata.xml (1.3 kB at 7.9 kB/s)
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92.pom
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-tests.jar
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-cyclonedx.xml
Uploading to apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/4.0.0-SNAPSHOT/spark-parent_2.13-4.0.0-20231027.110850-92-cyclonedx.json
...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:3.1.1:deploy (default-deploy) on project spark-parent_2.13: Failed to deploy artifacts: Could not transfer artifact org.apache.spark:spark-parent_2.13:pom:4.0.0-20231027.110850-92 from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): status code: 401, reason phrase: Unauthorized (401) -> [Help 1]
```
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45708][BUILD] Retry mvn deploy [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #43559:
URL: https://github.com/apache/spark/pull/43559#issuecomment-1797103474
Will adding the `- Dmaven.resolver.transport=wagon` in `MAVEN_OPTS` have any effect? The default implementation of Maven 3.9 pieces of resolver has changed from wagon to httpclient
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45708][BUILD] Retry mvn deploy [spark]
Posted by "EnricoMi (via GitHub)" <gi...@apache.org>.
EnricoMi commented on PR #43559:
URL: https://github.com/apache/spark/pull/43559#issuecomment-1798145198
> Will adding the `- Dmaven.resolver.transport=wagon` in `MAVEN_OPTS` have any effect? The default implementation of Maven 3.9 pieces of resolver has changed from wagon to httpclient
Not sure, looks like wagon is an abstraction on top of HTTP, so under the hood HTTP might still experience `408 Request Timeout`s: https://maven.apache.org/wagon/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45708][BUILD] Retry mvn deploy [spark]
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen commented on code in PR #43559:
URL: https://github.com/apache/spark/pull/43559#discussion_r1471528003
##########
pom.xml:
##########
@@ -3301,6 +3301,9 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>3.1.1</version>
+ <configuration>
+ <retryFailedDeploymentCount>10</retryFailedDeploymentCount>
Review Comment:
10 seems like a lot, why not start with like 3?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45708][BUILD] Retry mvn deploy [spark]
Posted by "EnricoMi (via GitHub)" <gi...@apache.org>.
EnricoMi commented on PR #43559:
URL: https://github.com/apache/spark/pull/43559#issuecomment-1796354605
@LuciferYang @HyukjinKwon the publish snapshot workflow keeps failing due to HTTP errors: https://github.com/apache/spark/actions/workflows/publish_snapshot.yml
Please consider this fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45708][BUILD] Retry mvn deploy [spark]
Posted by "EnricoMi (via GitHub)" <gi...@apache.org>.
EnricoMi commented on PR #43559:
URL: https://github.com/apache/spark/pull/43559#issuecomment-1867793048
@HyukjinKwon @LuciferYang this is still an issue: https://github.com/apache/spark/actions/runs/7294496840/job/19879448651
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45708][BUILD] Retry mvn deploy [spark]
Posted by "EnricoMi (via GitHub)" <gi...@apache.org>.
EnricoMi commented on PR #43559:
URL: https://github.com/apache/spark/pull/43559#issuecomment-1782784993
CC @LuciferYang @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org