[jira] [Commented] (YARN-9244) Document docker registry deployment with direct S3 driver

["Suma Shivaprasad (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-9244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760294#comment-16760294 ] 

Suma Shivaprasad commented on YARN-9244:
----------------------------------------

 Configure docker registry to use S3 storage driver with /etc/docker/registry/config.yml having the following configs 

version: 0.1
log:
    fields:
        service: registry
http:
    addr: :5000
storage:
    cache:
        layerinfo: inmemory
    s3:
        accesskey: #AWS_KEY#
        secretkey: #AWS_SECRET#
        region: #AWS_REGION#
        bucket: #AWS_BUCKET#
        encrypt: #ENCRYPT#
        secure:  #SECURE#
        chunksize: 5242880
        multipartcopychunksize: 33554432
        multipartcopymaxconcurrency: 100
        multipartcopythresholdsize: 33554432
        rootdirectory: #STORAGE_PATH#

docker run -e AWS_BUCKET=<BUCKET_NAME>   -e AWS_KEY=accessKey  -e AWS_SECRET=secretKey   -e AWS_REGION="aws-region"   -e STORAGE_PATH=/registry  -p 5000:5000 registry

time docker push  localhost:5000/centos:latest
The push refers to repository [localhost:5000/centos]
071d8bd76517: Pushed

real    0m13.188s
user    0m0.085s
sys    0m0.007s

{noformat}
docker tag centos:latest localhost:5000/centos:latest
time docker pull  localhost:5000/centos:latest
{noformat}
latest: Pulling from centos
a02a4930cb5d: Pull complete
Status: Downloaded newer image for localhost:5000/centos:latest

*real    0m2.918s*
user    0m0.048s
sys    0m0.023s



> Document docker registry deployment with direct S3 driver
> ---------------------------------------------------------
>
>                 Key: YARN-9244
>                 URL: https://issues.apache.org/jira/browse/YARN-9244
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Eric Yang
>            Assignee: Suma Shivaprasad
>            Priority: Major
>
> On Amazon, it would be possible to deploy docker registry in YARN using Docker's own S3 driver.  The same micro-benchmark can also be applied to show the docker registry performance difference.  The sample YARN service configuration looks like this:
> {code}
> {
>   "name": "docker-registry",
>   "version": "1.0",
>   "kerberos_principal" : {
>     "principal_name" : "hbase/_HOST@EXAMPLE.COM",
>     "keytab" : "file:///etc/security/keytabs/hbase.service.keytab"
>   },
>   "components" :
>   [
>     {
>       "name": "registry",
>       "number_of_containers": 1,
>       "artifact": {
>         "id": "registry:latest",
>         "type": "DOCKER"
>       },
>       "resource": {
>         "cpus": 1,
>         "memory": "256"
>       },
>       "run_privileged_container": true,
>       "configuration": {
>         "env": {
>           "YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE":"true",
>           "YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS":"/path/to/config.yml:/etc/docker/registry/config.yml"
>         },
>         "properties": {
>           "docker.network": "host"
>         }
>       }
>     }
>   ]
> }
> {code} 
> Where /path/to/config.yml maps to a Docker S3 storage option config.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org