You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Andrew Wang (JIRA)" <ji...@apache.org> on 2016/09/09 23:28:21 UTC
[jira] [Updated] (HDFS-10823) Implement
HttpFSFileSystem#listStatusIterator
[ https://issues.apache.org/jira/browse/HDFS-10823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Wang updated HDFS-10823:
-------------------------------
Attachment: HDFS-10823.001.patch
WIP Patch attached. This depends on HDFS-10837 which isn't committed yet, so not marking this Patch Available.
As described above, we have an unfortunate mismatch between the FS API and WebHDFS for implementing a generic iterator. This is why we have both a DirectoryListing and DirectoryEntries class that are slightly different. We could discard DL in favor of the more generic DE, though then the WebHDFS protocol would diverge a bit from the HDFS protobuf protocol. As it stands, if you are a downstream who wants to be generic to WebHDFS and HTTPFS, you need to treat {{remainingEntries}} like a boolean. This is kind of true anyway, since there could always be a concurrent writer adding more files to a listing which throws off the count.
I also didn't implement the subclass tricks that would let us make the FileSystem additions protected. I annotated them Private with strong warnings in javadoc, which I hope is sufficient.
> Implement HttpFSFileSystem#listStatusIterator
> ---------------------------------------------
>
> Key: HDFS-10823
> URL: https://issues.apache.org/jira/browse/HDFS-10823
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: httpfs
> Affects Versions: 2.6.4
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Attachments: HDFS-10823.001.patch
>
>
> Let's expose the same functionality added in HDFS-10784 for WebHDFS in HttpFS too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org