[jira] [Comment Edited] (YARN-7129) Application Catalog for YARN applications

["Eric Yang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541904#comment-16541904 ] 

Eric Yang edited comment on YARN-7129 at 7/12/18 5:20 PM:
----------------------------------------------------------

[~Zian Chen] Thank you for reviewing this patch.  Here is how I run the application catalog:

1.  Create a yarnfile with content:
{code}
{
  "name": "appcatalog",
  "kerberos_principal" : {
    "principal_name" : "eyang/_HOST@EXAMPLE.COM",
    "keytab" : "file:///etc/security/keytabs/eyang.service.keytab"
  },
  "version": "1",
  "components" :
  [
    {
      "name": "appcatalog",
      "number_of_containers": 1,
      "artifact": {
        "id": "hadoop/appcatalog-docker:3.2.0-SNAPSHOT",
        "type": "DOCKER"
      },
      "resource": {
        "cpus": 1,
        "memory": "256"
      },
      "run_privileged_container": true,
      "configuration": {
        "env": {
          "YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE":"true",
          "YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS":"/usr/local/hadoop-3.2.0-SNAPSHOT/etc/hadoop:/etc/hadoop/conf:ro,/etc/krb5.conf:/etc/krb5.conf:ro,/etc/security/keytabs/eyang.service.keytab:/etc/security/keytabs/eyang.service.keytab:ro",
          "KEYTAB":"/etc/security/keytabs/eyang.service.keytab",
          "PRINCIPAL":"eyang@EXAMPLE.COM"
        },
        "properties": {
          "docker.network": "host"
        }
      }
    }
  ]
}
{code}

2. Launch the application with:
{code}
yarn app -launch appcatalog yarnfile
{code}

3.  Look at the application master log file, and it will report where the application is launched, and use web browser to visit port 8080 of the application catalog.

Mount paths used in this yarn file:
| Source | Destination | Purpose |
| /usr/local/hadoop-3.2.0-SNAPSHOT/etc/hadoop | /etc/hadoop/conf | Read only Hadoop configuration |
| /etc/krb5.conf | /etc/krb5.conf | Read only Kerberos configuration |
| /etc/security/keytabs/eyang.service.keytab | /etc/security/keytabs/eyang.service.keytab | Kerberos keytab used by application |

KEYTAB, and PRINCIPAL environment variables are used to generate jaas configuration for application catalog.


was (Author: eyang):
[~Zian Chen] Thank you for reviewing this patch.  Here is how I run the application catalog:

1.  Create a yarnfile with content:
{code}
{
  "name": "appcatalog",
  "kerberos_principal" : {
    "principal_name" : "eyang/_HOST@EXAMPLE.COM",
    "keytab" : "file:///etc/security/keytabs/eyang.service.keytab"
  },
  "version": "1",
  "components" :
  [
    {
      "name": "appcatalog",
      "number_of_containers": 1,
      "artifact": {
        "id": "hadoop/appcatalog:latest",
        "type": "DOCKER"
      },
      "resource": {
        "cpus": 1,
        "memory": "256"
      },
      "run_privileged_container": true,
      "configuration": {
        "env": {
          "YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE":"true",
          "YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS":"/usr/local/hadoop-3.2.0-SNAPSHOT/etc/hadoop:/etc/hadoop/conf:ro,/etc/krb5.conf:/etc/krb5.conf:ro,/etc/security/keytabs/eyang.service.keytab:/etc/security/keytabs/eyang.service.keytab:ro",
          "KEYTAB":"/etc/security/keytabs/eyang.service.keytab",
          "PRINCIPAL":"eyang@EXAMPLE.COM"
        },
        "properties": {
          "docker.network": "host"
        }
      }
    }
  ]
}
{code}

2. Launch the application with:
{code}
yarn app -launch appcatalog yarnfile
{code}

3.  Look at the application master log file, and it will report where the application is launched, and use web browser to visit port 8080 of the application catalog.

Mount paths used in this yarn file:
| Source | Destination | Purpose |
| /usr/local/hadoop-3.2.0-SNAPSHOT/etc/hadoop | /etc/hadoop/conf | Read only Hadoop configuration |
| /etc/krb5.conf | /etc/krb5.conf | Read only Kerberos configuration |
| /etc/security/keytabs/eyang.service.keytab | /etc/security/keytabs/eyang.service.keytab | Kerberos keytab used by application |

KEYTAB, and PRINCIPAL environment variables are used to generate jaas configuration for application catalog.

> Application Catalog for YARN applications
> -----------------------------------------
>
>                 Key: YARN-7129
>                 URL: https://issues.apache.org/jira/browse/YARN-7129
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: applications
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Major
>         Attachments: YARN Appstore.pdf, YARN-7129.001.patch, YARN-7129.002.patch, YARN-7129.003.patch, YARN-7129.004.patch, YARN-7129.005.patch
>
>
> YARN native services provides web services API to improve usability of application deployment on Hadoop using collection of docker images.  It would be nice to have an application catalog system which provides an editorial and search interface for YARN applications.  This improves usability of YARN for manage the life cycle of applications.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org