You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Zoltán Borók-Nagy (Jira)" <ji...@apache.org> on 2019/08/26 15:07:00 UTC
[jira] [Commented] (IMPALA-8739) FileMetadataLoader skips empty
directories
[ https://issues.apache.org/jira/browse/IMPALA-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16915877#comment-16915877 ]
Zoltán Borók-Nagy commented on IMPALA-8739:
-------------------------------------------
I follow Hive's behavior in the TRUNCATE patch set (method createEmptyBaseDirectories()) [https://gerrit.cloudera.org/#/c/14071/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java]
I think it's also a good solution for S3, I was always worried about that "empty directories" on S3 would cause surprises.
> FileMetadataLoader skips empty directories
> ------------------------------------------
>
> Key: IMPALA-8739
> URL: https://issues.apache.org/jira/browse/IMPALA-8739
> Project: IMPALA
> Issue Type: Bug
> Reporter: Vihang Karajgaonkar
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-acid
>
> {{FileMetadataLoader}} has certain code paths like the one below which using {{listFiles}} API on the filesystem. This API ignores empty directories which is okay for non-transactional tables. However, in case of transactional table an empty base directory provides writeId information which is used to skip loading files which are not relevant for a given writeId. See {{AcidUtils#filterFilesForAcidState}} usage for details.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org