You are viewing a plain text version of this content. The canonical link for it is here.
Posted to submarine-dev@hadoop.apache.org by "Szilard Nemeth (JIRA)" <ji...@apache.org> on 2019/04/16 08:18:00 UTC
[jira] [Comment Edited] (SUBMARINE-54) Add test coverage for
YarnServiceJobSubmitter and make it ready to extension for PyTorch
[ https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818743#comment-16818743 ]
Szilard Nemeth edited comment on SUBMARINE-54 at 4/16/19 8:17 AM:
------------------------------------------------------------------
Hi [~sunilg], [~tangzhankun]!
I'm blocked on the mvn dependency resolution issues:
In file hadoop/hadoop-submarine/hadoop-submarine-core/pom.xml, we have this already:
{code:java}
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-common</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
{code}
I need some yarn-common dependency so I inserted the same block as above to the file hadoop/hadoop-submarine/hadoop-submarine-yarnservice-runtime/pom.xml.
Unfortunately, maven came back with this:
{code:java}
[INFO] Building Hadoop Submarine YARN Service Runtime 0.2.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-submarine/0.2.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-submarine/0.2.0-SNAPSHOT/hadoop-submarine-0.2.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.290 s
[INFO] Finished at: 2019-04-15T21:21:05+00:00
[INFO] Final Memory: 24M/382M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hadoop-submarine-score-yarnservice-runtime: Could not resolve dependencies for project org.apache.hadoop:hadoop-submarine-score-yarnservice-runtime:jar:0.2.0-SNAPSHOT: Failed to collect dependencies at org.apache.hadoop:hadoop-submarine-core:jar:tests:0.2.0-SNAPSHOT: Failed to read artifact descriptor for org.apache.hadoop:hadoop-submarine-core:jar:tests:0.2.0-SNAPSHOT: Could not find artifact org.apache.hadoop:hadoop-submarine:pom:0.2.0-SNAPSHOT in apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
[ERROR]
{code}
On my mac, the build goes without any issues with this command:
{code:java}
mvn clean package -Pdist -DskipTests -Dmaven.javadoc.skip=true
{code}
Do you have an idea?
Thanks!
was (Author: snemeth):
Hi [~sunilg], [~tangzhankun]!
I'm blocked on the mvn dependency resolution issues:
In file hadoop/hadoop-submarine/hadoop-submarine-core/pom.xml, we have this already:
{code:java}
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-common</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
{code}
I need some yarn-common dependency so I inserted the same block as above to the file hadoop/hadoop-submarine/hadoop-submarine-yarnservice-runtime/pom.xml.
Unfortunately, maven came back with this:
{code:java}
[INFO] Building Hadoop Submarine YARN Service Runtime 0.2.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-submarine/0.2.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-submarine/0.2.0-SNAPSHOT/hadoop-submarine-0.2.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.290 s
[INFO] Finished at: 2019-04-15T21:21:05+00:00
[INFO] Final Memory: 24M/382M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hadoop-submarine-score-yarnservice-runtime: Could not resolve dependencies for project org.apache.hadoop:hadoop-submarine-score-yarnservice-runtime:jar:0.2.0-SNAPSHOT: Failed to collect dependencies at org.apache.hadoop:hadoop-submarine-core:jar:tests:0.2.0-SNAPSHOT: Failed to read artifact descriptor for org.apache.hadoop:hadoop-submarine-core:jar:tests:0.2.0-SNAPSHOT: Could not find artifact org.apache.hadoop:hadoop-submarine:pom:0.2.0-SNAPSHOT in apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
[ERROR]
{code}
Do you have an idea?
Thanks!
> Add test coverage for YarnServiceJobSubmitter and make it ready to extension for PyTorch
> ----------------------------------------------------------------------------------------
>
> Key: SUBMARINE-54
> URL: https://issues.apache.org/jira/browse/SUBMARINE-54
> Project: Hadoop Submarine
> Issue Type: Sub-task
> Reporter: Szilard Nemeth
> Assignee: Szilard Nemeth
> Priority: Major
> Attachments: SUBMARINE-54.001.patch
>
>
> This crucial class has no associated test yet. We need to improve this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)