You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2018/12/10 05:47:00 UTC
[jira] [Commented] (YARN-8913) Add helper scripts to launch MaWo
App to run Hadoop unit tests on Hadoop Cluster
[ https://issues.apache.org/jira/browse/YARN-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714313#comment-16714313 ]
Allen Wittenauer commented on YARN-8913:
----------------------------------------
This should absolutely be using the dev-support/docker/Dockerfile and not something custom. As it stands, this Dockerfile is missing some absolutely critical parts (eg, everything native) and is downloading components that are very different than what the ASF actually uses for its tests. (Why is maven coming from Fedora instead of, you know, Apache?)
> Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
> --------------------------------------------------------------------------------
>
> Key: YARN-8913
> URL: https://issues.apache.org/jira/browse/YARN-8913
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Yesha Vora
> Assignee: Yesha Vora
> Priority: Blocker
> Attachments: YARN-8913.001.patch, YARN-8913.002.patch, YARN-8913.003.patch
>
>
> MaWo application can be used to run Hadoop UT faster in a Hadoop cluster.
> Develop helper scripts to orchestrate end-to-end workflow for running Hadoop UT using MaWo app.
> Pre-requisite:
> * A Hadoop Cluster with HDFS and YARN installed
> * Enable Docker on YARN feature
>
> Helper-scripts
> * MaWo_Driver
> ** create a docker image with latest hadoop source code
> ** create payload to MaWo app (This is input to mawo app where Each MaWo Task = UT execution of each Hadoop Module)
> ** Upload payload file to HDFS
> ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically
> ** Launch MaWo app in Hadoop cluster
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org