[jira] [Commented] (YARN-9230) Write a go hdfs driver for Docker Registry

["Eric Yang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-9230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755444#comment-16755444 ] 

Eric Yang commented on YARN-9230:
---------------------------------

Thanks to [~shanekumpf@gmail.com], I was able to rebase his [hdfs docker driver|https://github.com/sakserv/distribution] for Docker Registry to latest Docker registry code, and run the same benchmark as YARN-9229.  The performance is quite good:

| | Remote | Local |
| Pull | 18.673s | 13.109s |
| Push | 26.113s | 25.917s |

The numbers are measured on the same hardware that was used to measure NFS Gateway flavor of Docker Registry.

The only drawback of this approach is Go HDFS client library does not fully support Kerberos enabled environment.  User will encounter ["unexpected sequence number"|https://github.com/colinmarc/hdfs/issues/144] issue.  Some GSSAPI encryption related problems that are not fully resolved in Colin's Go HDFS library.


> Write a go hdfs driver for Docker Registry
> ------------------------------------------
>
>                 Key: YARN-9230
>                 URL: https://issues.apache.org/jira/browse/YARN-9230
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Eric Yang
>            Priority: Major
>
> A number of people have developed go client library for HDFS.  [Example 1|https://medium.com/sqooba/making-hdfs-a-hundred-times-faster-ac75b8b5e0b4] [Example 2|https://github.com/colinmarc/hdfs].
> This can be enhanced into a real storage driver for Docker registry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org