[jira] [Commented] (YARN-6556) Implement a FileSystem that reads from HTTP

["Jason Lowe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997281#comment-15997281 ] 

Jason Lowe commented on YARN-6556:
----------------------------------

Wouldn't the new HTTP filesystem appear in the Hadoop Common project rather than YARN?  I don't see why it would need to be something specific to YARN.


> Implement a FileSystem that reads from HTTP
> -------------------------------------------
>
>                 Key: YARN-6556
>                 URL: https://issues.apache.org/jira/browse/YARN-6556
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>
> We have a use case where YARN applications would like to localize resources from Artifactory. Putting the resources on HDFS itself might not be ideal as we would like to leverage Artifactory to manage different versions of the resources.
> It would be nice to have something like {{HttpFileSystem}} that implements the Hadoop filesystem API and reads from a HTTP endpoint.
> Note that Samza has implemented the proposal by themselves:
> https://github.com/apache/samza/blob/master/samza-yarn/src/main/scala/org/apache/samza/util/hadoop/HttpFileSystem.scala
> The downside of this approach is that it requires the YARN cluster to put the Samza jar into the classpath for each NM.
> It would be much nicer for Hadoop to have this feature built-in.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org