You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2025/01/21 08:48:00 UTC
[jira] [Commented] (IMPALA-13666) Add a new constructor for ImpalaHdfsPartition after IMPALA-13154
[ https://issues.apache.org/jira/browse/IMPALA-13666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914897#comment-17914897 ]
ASF subversion and git services commented on IMPALA-13666:
----------------------------------------------------------
Commit 5371e0c6df3e329398712af3ebb739465b947454 in impala's branch refs/heads/master from Fang-Yu Rao
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=5371e0c6d ]
IMPALA-13666: Provide a non-null fileMetadataStats for HdfsPartition
IMPALA-13154 added the method getFileMetadataStats() to
HdfsPartition.java that would return the file metadata statistics. The
method requires the corresponding HdfsPartition instance to have a
non-null field of 'fileMetadataStats_'.
This patch revises two existing constructors of HdfsPartition to provide
a non-null value for 'fileMetadataStats'. This makes it easier for a
third party extension to set up and update the field of
'fileMetadataStats_'. A third party extension has to update the field of
'fileMetadataStats_' if it would like to use this field to get the size
of the partition since all three fields in 'fileMetadataStats_' are
defaulted to 0.
A new constructor was also added for HdfsPartition that allows a third
party extension to provide their own FileMetadataStats when
instantiating an HdfsPartition. To facilitate instantiating a
FileMetadataStats, a new constructor was added for FileMetadataStats
that takes in a List of FileDescriptor's to construct a
FileMetadataStats.
Change-Id: I7e690729fcaebb1e380cc61f2b746783c86dcbf7
Reviewed-on: http://gerrit.cloudera.org:8080/22340
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Add a new constructor for ImpalaHdfsPartition after IMPALA-13154
> ----------------------------------------------------------------
>
> Key: IMPALA-13666
> URL: https://issues.apache.org/jira/browse/IMPALA-13666
> Project: IMPALA
> Issue Type: Task
> Reporter: Fang-Yu Rao
> Assignee: Fang-Yu Rao
> Priority: Major
>
> IMPALA-13154 added the method {{getFileMetadataStats()}} to [HdfsPartition.java|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java] that would return the file metadata statistics. The method requires the corresponding {{HdfsPartition}} instance to have a non-null field of '{{{}fileMetadataStats_{}}}'.
> However, currently in order to set up the field of '{{{}fileMetadataStats_{}}}' when instantiating an {{{}HdfsPartition{}}}, we also need to provide the value of the field '{{{}id_{}}}', which should be unique across the whole catalog for each partition. Since it's possible that we may not always know what value we should be using for '{{{}id_{}}}' when instantiating an {{HdfsPartition}} via a third party extension, if would be good that we add a new constructor for {{HdfsPartition}} to allow us to set up the field of '{{{}fileMetadataStats_{}}}' when instantiating an {{HdfsPartition}} without having to provide the value of the field '{{{}id_{}}}'.
> An alternative would be revising the existing constructors of {{HdfsPartition}} to provide a non-null value for '{{{}fileMetadataStats_{}}}'.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org