You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "BELUGA BEHR (JIRA)" <ji...@apache.org> on 2019/03/01 13:54:00 UTC
[jira] [Commented] (HDFS-13623) getContentSummary to return
ContentSummary without hidden files
[ https://issues.apache.org/jira/browse/HDFS-13623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781689#comment-16781689 ]
BELUGA BEHR commented on HDFS-13623:
------------------------------------
I would suggest updating the existing:
{code}
public ContentSummary getContentSummary(Path f, boolean includeHidden)
throws IOException
Return the ContentSummary of a given Path.
Parameters:
f - path to use
includeHidden - Whether or not to include hidden files in the return values
{code}
> getContentSummary to return ContentSummary without hidden files
> ---------------------------------------------------------------
>
> Key: HDFS-13623
> URL: https://issues.apache.org/jira/browse/HDFS-13623
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs, namenode
> Affects Versions: 3.1.0
> Reporter: Miklos Szurap
> Priority: Major
>
> Improve the [FileSystem.getContentSummary()|http://hadoop.apache.org/docs/r3.1.0/api/org/apache/hadoop/fs/FileSystem.html#getContentSummary-org.apache.hadoop.fs.Path-] method to return ContentSummary object with "getFileCountWithoutHiddenFiles()" and "getLengthWithoutHiddenFiles()".
> That two new counter should not include hidden files and hidden directories (and it's sub-contents).
> {code:java}
> public static final PathFilter HIDDEN_FILES_PATH_FILTER = new PathFilter() {
> public boolean accept(Path p) {
> String name = p.getName();
> return !name.startsWith("_") && !name.startsWith(".");
> }
> };{code}
> This would be especially useful for Hive: to compute table statistics with a single {{contentSummary}} call instead of {{globStatus}} (multiple {{listStatus}} calls) and iterating over multiple thousand of objects on client side.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org