[jira] [Comment Edited] (YARN-7654) Support ENTRY_POINT for docker container

["Eric Yang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389883#comment-16389883 ] 

Eric Yang edited comment on YARN-7654 at 3/7/18 6:01 PM:
---------------------------------------------------------

Hi [~ebadger] [~billie.rinaldi] [~shanekumpf@gmail.com] Here is an early draft of the patch.  The current patch allows to run both ENTRY_POINT enabled or disabled with limitation that user environment variables are not forwarded to ENTRY_POINT enabled container.  Given the complexity of the code base, and having to work on two different modes.  I think it is best to get some early feedbacks.

For patch 001 to work, you will need to apply YARN-7221 patch 6 and YARN-7677 patch 7.

I did some review of the environment variables, and I am only comfortable to expose user defined environment variables in ENTRY_POINT enabled container.  The global hadoop environment variable and node manager constructed environment variables will be available in prelaunch, but not forwarded to container.  Let me know your thoughts on this approach.  Thanks

Example job that I used to submit:
{code}
{
  "name": "sleeper-service",
  "kerberos_principal" : {
    "principal_name" : "hbase/_HOST@EXAMPLE.COM",
    "keytab" : "file:///etc/security/keytabs/hbase.service.keytab"
  },
  "components" :
  [
    {
      "name": "sleeper",
      "number_of_containers": 2,
      "artifact": {
        "id": "hadoop/centos:latest",
        "type": "DOCKER"
      },
      "privileged": true,
      "launch_command": "sleep 90",
      "resource": {
        "cpus": 1,
        "memory": "256"
      },
      "configuration": {
        "env": {
          "YARN_CONTAINER_RUNTIME_DOCKER_DELAYED_REMOVAL":"true",
          "YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE":"true",
          "USER_DEFINE":"1",
          "USER_DEFINE":"2"
        },
        "properties": {
          "docker.network": "host"
        }
      }
    }
  ]
}
{code}


was (Author: eyang):
Hi [~ebadger] [~billie.rinaldi] [~shanekumpf@gmail.com] Here is an early draft of the patch.  The current patch allows to run both ENTRY_POINT enabled or disabled with limitation that user environment variables are not forwarded to ENTRY_POINT enabled container.  Given the complexity of the code base, and having to work on two different modes.  I think it is best to get some early feedbacks.

For patch 001 to work, you will need to apply YARN-7221 patch 6 and YARN-7677 patch 7.

I did some review of the environment variables, and I am only comfortable to expose user defined environment variables.  The global hadoop environment variable and node manager constructed environment variables will be available in prelaunch, but not forwarded to container.  Let me know your thoughts on this approach.  Thanks

> Support ENTRY_POINT for docker container
> ----------------------------------------
>
>                 Key: YARN-7654
>                 URL: https://issues.apache.org/jira/browse/YARN-7654
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>    Affects Versions: 3.1.0
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Blocker
>         Attachments: YARN-7654.001.patch
>
>
> Docker image may have ENTRY_POINT predefined, but this is not supported in the current implementation.  It would be nice if we can detect existence of {{launch_command}} and base on this variable launch docker container in different ways:
> h3. Launch command exists
> {code}
> docker run [image]:[version]
> docker exec [container_id] [launch_command]
> {code}
> h3. Use ENTRY_POINT
> {code}
> docker run [image]:[version]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org