[jira] [Updated] (YARN-2518) Support in-process container executor

["Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vinod Kumar Vavilapalli updated YARN-2518:
------------------------------------------
    Labels:   (was: container dispatch in-process job node)

Today's containers in YARN are necessarily process-trees - meaning they are separate processes with their own command-lines, environment etc. If you want an 'in-process' container, given NMs are JVMs, you need native support for some sort of a first-class JVM container in YARN. This doesn't exist today, so this ticket doesn't make much sense as of now.

Your use-case is valid. And as the above comment says, the solution is to launch a container and reuse it.

One other relevant work you may be interested in is YARN-896.

> Support in-process container executor
> -------------------------------------
>
>                 Key: YARN-2518
>                 URL: https://issues.apache.org/jira/browse/YARN-2518
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager
>    Affects Versions: 2.5.0
>         Environment: Linux, Windows
>            Reporter: BoYang
>            Priority: Minor
>
> Node Manage always creates a new process for a new application. We have hit a scenario where we want the node manager to execute the application inside its own process, so we get fast response time. It would be nice if Node Manager or YARN can provide native support for that.
> In general, the scenario is that we have a long running process which can accept requests and process the requests inside its own process. Since YARN is good at scheduling jobs, we want to use YARN to dispatch jobs (e.g. requests in JSON) to the long running process. In that case, we do not want YARN container to spin up a new process for each request. Instead, we want YARN container to send the request to the long running process for further processing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)