You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by "Andrew Chung (JIRA)" <ji...@apache.org> on 2015/05/30 01:52:17 UTC
[jira] [Comment Edited] (REEF-227) Add implementation of the REEF
Client for HDInsight
[ https://issues.apache.org/jira/browse/REEF-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565667#comment-14565667 ]
Andrew Chung edited comment on REEF-227 at 5/29/15 11:51 PM:
-------------------------------------------------------------
Is it possible for driver.conf to be generated on the cluster during submission instead? i.e. add a command to generate driver.conf before the Java command to start launcher in the commands object for REST submission (commands = "\{generate driver.conf\} & \{launch driver\}"). I haven't looked into the submission code yet, but I think this might just work.
was (Author: afchung90):
Is it possible for driver.conf to be generated on the cluster during submission instead? i.e. add a command to generate driver.conf before the Java command to start launcher in the commands object for REST submission (commands = "{generate driver.conf} & {launch driver}"). I haven't looked into the submission code yet, but I think this might just work.
> Add implementation of the REEF Client for HDInsight
> ---------------------------------------------------
>
> Key: REEF-227
> URL: https://issues.apache.org/jira/browse/REEF-227
> Project: REEF
> Issue Type: Sub-task
> Components: REEF.NET
> Reporter: Markus Weimer
> Assignee: Beysim Sezgin
>
> For this implementation, we can go a slightly different route. Both the local and YARN implementations prepare the Driver folder in C# but then hand it over to Java for the actual submission.
> HDInsight offers a REST API for job submission. Hence, we can do the following approach:
> # C# prepares the Driver folder
> # Java adds the java side (mostly the Driver configuration)
> # C# zips it up and submits.
> This would get us one step closer to having a client for machines with no Java installed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)