You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tim Tim (Jira)" <ji...@apache.org> on 2025/02/12 03:15:00 UTC
[jira] [Created] (SPARK-51174) Deployment of Spark 4.0+ fails on the module common/network-common
Tim Tim created SPARK-51174:
-------------------------------
Summary: Deployment of Spark 4.0+ fails on the module common/network-common
Key: SPARK-51174
URL: https://issues.apache.org/jira/browse/SPARK-51174
Project: Spark
Issue Type: Bug
Components: Deploy
Affects Versions: 4.0.0, 4.1.0
Environment: Ubuntu20.04, JDK17
Reporter: Tim Tim
"Deployment of Spark 4.0+ fails on the module {{common/network-common}} as described below:
To reproduce the issue, use the following CLI command:
bash
./build/mvn -DskipTests --settings=local-maven-repo-settings.xml clean deploy
This issue first occurred after the PR was merged: [https://github.com/apache/spark/pull/49765]. The problem disappears if I revert that PR.
The failure is not observed in Spark 3.0+ versions; it only occurs in Spark 4.0+ versions."
```
./build/mvn -DskipTests --settings=local-maven-repo-settings.xml clean deploy
....
*[**INFO**]* *Downloaded* *from snapshots: https:*{*}//test.example/maven-repo/org/bouncycastle/bcprov-jdk18on/1.80/bcprov-jdk18on-1.80.jar (8.7 MB at 3.2 MB/s){*}
*[**INFO**]* *Downloading* *from snapshots: https:*{*}//test.example/maven-repo/org/bouncycastle/bcpkix-jdk18on/1.80/bcpkix-jdk18on-1.80.jar{*}
*[**INFO**]* *Downloaded* *from snapshots: https:*{*}//test.example/maven-repo/org/bouncycastle/bcpkix-jdk18on/1.80/bcpkix-jdk18on-1.80.jar (1.1 MB at 2.6 MB/s){*}
{*}[{*}*INFO*{*}] ------------------------------------------------------------------------{*}
*[*{*}INFO{*}*]* *Reactor* ** *Summary* ** *for* ** *Spark* ** *Project* ** *Parent* ** *POM* ** *4.1**.**0**-*{*}SNAPSHOT{*}*:*
*[*{*}INFO{*}*]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Parent* ** *POM* *...........................* *SUCCESS* *[* *22.795* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Tags* *.................................* *SUCCESS* *[* *29.952* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Sketch* *...............................* *SUCCESS* *[* *20.523* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Common* ** *Utils* *.........................* *SUCCESS* *[* *33.674* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Local* ** *DB* *.............................* *SUCCESS* *[* *22.940* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Networking* *...........................* *FAILURE* *[* *6.459* *s]*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Shuffle* ** *Streaming* ** *Service* *............* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Variant* *..............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Unsafe* *...............................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Connect* ** *Shims* *........................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Launcher* *.............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Core* *.................................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *ML* ** *Local* ** *Library* *.....................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *GraphX* *...............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Streaming* *............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *SQL* ** *API* *..............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Catalyst* *.............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *SQL* *..................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *ML* ** *Library* *...........................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Tools* *................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Hive* *.................................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Connect* ** *Common* *.......................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Avro* *.........................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Protobuf* *.....................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *REPL* *.................................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Connect* ** *Server* *.......................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Connect* ** *Client* *.......................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *YARN* ** *Shuffle* ** *Service* *.................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *YARN* *.................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Kubernetes* *...........................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Assembly* *.............................* *SKIPPED*
*[*{*}INFO{*}*]* *Kafka* ** *0.10**+* *Token* ** *Provider* ** *for* ** *Streaming* *...........* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Integration* ** *for* ** *Kafka* ** *0.10* *...................* *SKIPPED*
*[*{*}INFO{*}*]* *Kafka* ** *0.10**+* *Source* ** *for* ** *Structured* ** *Streaming* *........* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Examples* *.............................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Integration* ** *for* ** *Kafka* ** *0.10* ** *Assembly* *..........* *SKIPPED*
{*}[{*}*INFO*{*}] ------------------------------------------------------------------------{*}
*[*{*}INFO{*}*]* *BUILD* ** *FAILURE*
{*}[{*}*INFO*{*}] ------------------------------------------------------------------------{*}
{*}[{*}*INFO*{*}]{*} *Total* *time:* *02*{*}:{*}*17* *min*
{*}[{*}*INFO*{*}]{*} *Finished* *at:* *2025*{*}-{*}*02*{*}-12T02:{*}*13*{*}:45Z{*}
{*}[{*}*INFO*{*}] ------------------------------------------------------------------------{*}
{*}[{*}*ERROR*{*}]{*} *Failed* \{*}to execute goal on project spark-network-common_2.{*}*13*{*}:{*} *Could* *not resolve dependencies* *for* \{*}project org.{*}*apache*{*}.{*}*spark*{*}:spark-network-common_2.{*}*13*{*}:jar:{*}*4.1*{*}.{*}*0*{*}-{*}*SNAPSHOT*
{*}[{*}*ERROR*{*}] dependency: org.{*}*bouncycastle*{*}:bcutil-jdk18on:jar:{*}*1.80* *(test)*
{*}[{*}*ERROR*{*}] {*} *Could* \{*}not transfer artifact org.{*}*bouncycastle*{*}:bcutil-jdk18on:jar:{*}*1.80* *from/to* *central* \{*}(https:{*}*//test.example/maven-repo): status code: 401, reason phrase: (401)*
{*}[{*}*ERROR*{*}] -> [{*}*Help* ** *1*{*}]{*}
*[*{*}ERROR{*}*]*
{*}[{*}*ERROR*{*}]{*} *To* *see the full stack trace of the errors, re-run* *Maven* *with the -e* *switch*{*}.{*}
{*}[{*}*ERROR*{*}]{*} *Re*{*}-run{*} *Maven* \{*}using the -{*}*X* ** *switch* *to enable full debug logging.*
*[*{*}ERROR{*}*]*
{*}[{*}*ERROR*{*}]{*} *For* *more information about the errors and possible solutions, please read the following articles:*
*[**ERROR**] [**Help* ** *1**] http:*{*}//cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException{*}
*[*{*}ERROR{*}*]*
{*}[{*}*ERROR*{*}]{*} *After* *correcting the problems, you can resume the build with the command*
{*}[{*}*ERROR*{*}] mvn <args> -rf :spark-network-common_2.{*}*13*
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org