You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2010/04/08 23:36:36 UTC
[jira] Created: (HADOOP-6692) Propose change to
FileContext#listStatus
Propose change to FileContext#listStatus
----------------------------------------
Key: HADOOP-6692
URL: https://issues.apache.org/jira/browse/HADOOP-6692
Project: Hadoop Common
Issue Type: Sub-task
Reporter: Hairong Kuang
Assignee: Hairong Kuang
Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
I've committed this!
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch, listStatusItor2.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Attachment: listStatusItor1.patch
Thank Suresh and Eli for review my patch. Those are good feedback. This patch addresses most of the comments except for
> FileContext.listStatus() that returns iterator - add information about throwing UnsupportedOperationException and NoSuchElementException in the RuntimeExceptions section on attempt to delete an entry from iterator.
Those exceptions are not thrown in listStatus method. Those may be thrown when iterating happens.
> AFS.listStatusItertor() - method javadoc should point @link to the right method in FileContext.
It points to the right method. It happens that in FileContext the method that returns an iterator is named as listStatus.
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6692:
--------------------------------
Summary: Add FileContext#listStatus that returns an iterator (was: Propose change to FileContext#listStatus)
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Attachment: listStatusItor2.patch
This patch in addition made a change to TestFilterFs to fix the failed test.
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch, listStatusItor2.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Status: Patch Available (was: Open)
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch, listStatusItor2.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860420#action_12860420 ]
Hadoop QA commented on HADOOP-6692:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12442712/listStatusItor1.patch
against trunk revision 937183.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/476/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/476/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/476/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/476/console
This message is automatically generated.
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6692) Propose change to
FileContext#listStatus
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12855482#action_12855482 ]
Hadoop QA commented on HADOOP-6692:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12441211/listStatusItor.patch
against trunk revision 932115.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/455/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/455/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/455/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/455/console
This message is automatically generated.
> Propose change to FileContext#listStatus
> ----------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Status: Patch Available (was: Open)
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860428#action_12860428 ]
Suresh Srinivas commented on HADOOP-6692:
-----------------------------------------
+1 for the patch.
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860486#action_12860486 ]
Hadoop QA commented on HADOOP-6692:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12442734/listStatusItor2.patch
against trunk revision 937577.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/477/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/477/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/477/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/477/console
This message is automatically generated.
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch, listStatusItor2.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Status: Open (was: Patch Available)
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch, listStatusItor2.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Status: Open (was: Patch Available)
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6692) Propose change to
FileContext#listStatus
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12855508#action_12855508 ]
Suresh Srinivas commented on HADOOP-6692:
-----------------------------------------
Hairong, this is a good change.
In order to simplify transition to FileContext, should we retain the old FileContext.listStatus() method as it is and add iterative list status as listStatusIterator()? It also captures the semantics quite well.
Minor:
# FileContext.listStatus() that returns iterator - add information about throwing UnsupportedOperationException and NoSuchElementException in the RuntimeExceptions section on attempt to delete an entry from iterator.
# FileContext.listStatus() that retuns array - has invalid description in @return. Also add a note to use the itertor version if the need is to iterate over the list, with a @link to the other method
# AFS.listStatusItertor() - method javadoc should point @link to the right method in FileContext.
# AFS.listStatus() - method javadoc should point @link to the right method in FileContext. Current link is invalid and is missing # prefix to listStatus().
> Propose change to FileContext#listStatus
> ----------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6692) Propose change to
FileContext#listStatus
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12856242#action_12856242 ]
Eli Collins commented on HADOOP-6692:
-------------------------------------
Hey Hairong,
The patch looks good to me. Only one minor edit in addition to Suresh's comments: FileContext#listStatus will never throw UnresolvedLinkException so you can remove that from the throws clause and javadoc.
Thanks,
Eli
> Propose change to FileContext#listStatus
> ----------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-6692) Add FileContext#listStatus that
returns an iterator
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Release Note: This issue adds Iterator<FileStatus> listStatus(Path) to FileContext, moves FileStatus[] listStatus(Path) to FileContext#Util, and adds Iterator<FileStatus> listStatusItor(Path) to AbstractFileSystem which provides a default implementation by using FileStatus[] listStatus(Path).
Component/s: fs
> Add FileContext#listStatus that returns an iterator
> ---------------------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch, listStatusItor1.patch, listStatusItor2.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6692) Propose change to
FileContext#listStatus
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Attachment: listStatusItor.patch
This patch:
# add Iterator<FileStatus> listStatus(Path) to FileContext;
# move FileStatus[] listStatus(Path) to FileContext#Util;
# add Iterator<FileStatus> listStatusItor(Path) to AbstractFileSystem which provides a default implementation by using FileStatus[] listStatus(Path);
# add unit test to test Iterator<FileStatus> listStatus(Path).
> Propose change to FileContext#listStatus
> ----------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Attachments: listStatusItor.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6692) Propose change to
FileContext#listStatus
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6692:
----------------------------------
Fix Version/s: 0.22.0
Affects Version/s: 0.22.0
Status: Patch Available (was: Open)
> Propose change to FileContext#listStatus
> ----------------------------------------
>
> Key: HADOOP-6692
> URL: https://issues.apache.org/jira/browse/HADOOP-6692
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: listStatusItor.patch
>
>
> Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS client not to have the whole listing in the memory, benefit more from the iterative listing added in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.