You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Stavros Kontopoulos (JIRA)" <ji...@apache.org> on 2018/07/17 20:21:00 UTC
[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration
tests don't trace the maven project dependency structure
[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547061#comment-16547061 ]
Stavros Kontopoulos commented on SPARK-24825:
---------------------------------------------
To reproduce it try build the test suite:
```
../../../build/mvn install
Using `mvn` from path:...spark/build/apache-maven-3.3.9/bin/mvn
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Spark Project Kubernetes Integration Tests 2.4.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repository.apache.org/snapshots/org/apache/spark/spark-core_2.11/2.4.0-SNAPSHOT/maven-metadata.xml
Downloaded: https://repository.apache.org/snapshots/org/apache/spark/spark-core_2.11/2.4.0-SNAPSHOT/maven-metadata.xml (2 KB at 0.4 KB/sec)
Downloading: https://repository.apache.org/snapshots/org/apache/spark/spark-core_2.11/2.4.0-SNAPSHOT/spark-core_2.11-2.4.0-20180717.095738-170.pom
[WARNING] The POM for org.apache.spark:spark-core_2.11:jar:2.4.0-20180717.095738-170 is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-core_2.11:jar:tests:2.4.0-20180717.095737-170 is missing, no dependency information available
Downloading: https://repository.apache.org/snapshots/org/apache/spark/spark-tags_2.11/2.4.0-SNAPSHOT/maven-metadata.xml
Downloaded: https://repository.apache.org/snapshots/org/apache/spark/spark-tags_2.11/2.4.0-SNAPSHOT/maven-metadata.xml (2 KB at 0.7 KB/sec)
Downloading: https://repository.apache.org/snapshots/org/apache/spark/spark-tags_2.11/2.4.0-SNAPSHOT/spark-tags_2.11-2.4.0-20180717.095221-170.pom
[WARNING] The POM for org.apache.spark:spark-tags_2.11:jar:tests:2.4.0-20180717.095220-170 is missing, no dependency information available
Downloading: https://repository.apache.org/snapshots/org/apache/spark/spark-core_2.11/2.4.0-SNAPSHOT/spark-core_2.11-2.4.0-20180717.095738-170.jar
Downloading: https://repository.apache.org/snapshots/org/apache/spark/spark-core_2.11/2.4.0-SNAPSHOT/spark-core_2.11-2.4.0-20180717.095737-170-tests.jar
Downloading: https://repository.apache.org/snapshots/org/apache/spark/spark-tags_2.11/2.4.0-SNAPSHOT/spark-tags_2.11-2.4.0-20180717.095220-170-tests.jar
Downloaded: https://repository.apache.org/snapshots/org/apache/spark/spark-tags_2.11/2.4.0-SNAPSHOT/spark-tags_2.11-2.4.0-20180717.095220-170-tests.jar (9 KB at 5.2 KB/sec)
Downloaded: https://repository.apache.org/snapshots/org/apache/spark/spark-core_2.11/2.4.0-SNAPSHOT/spark-core_2.11-2.4.0-20180717.095737-170-tests.jar (10438 KB at 577.5 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26.346 s
[INFO] Finished at: 2018-07-17T23:15:29+03:00
[INFO] Final Memory: 14M/204M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project spark-kubernetes-integration-tests_2.11: Could not resolve dependencies for project spark-kubernetes-integration-tests:spark-kubernetes-integration-tests_2.11:jar:2.4.0-SNAPSHOT: Could not find artifact org.apache.spark:spark-core_2.11:jar:2.4.0-20180717.095738-170 in apache.snapshots (https://repository.apache.org/snapshots) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] [http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionExceptio|http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException]
```
``
> [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure
> -------------------------------------------------------------------------------------------
>
> Key: SPARK-24825
> URL: https://issues.apache.org/jira/browse/SPARK-24825
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes, Tests
> Affects Versions: 2.4.0
> Reporter: Matt Cheah
> Priority: Critical
>
> The Kubernetes integration tests will currently fail if maven installation is not performed first, because the integration test build believes it should be pulling the Spark parent artifact from maven central. However, this is incorrect because the integration test should be building the Spark parent pom as a dependency in the multi-module build, and the integration test should just use the dynamically built artifact. Or to put it another way, the integration test builds should never be pulling Spark dependencies from maven central.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org