You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Yu-Wen Lai (Jira)" <ji...@apache.org> on 2022/01/28 04:39:00 UTC
[jira] [Updated] (IMPALA-11093) Fine grained table refreshing doesn't refresh table file metadata
[ https://issues.apache.org/jira/browse/IMPALA-11093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yu-Wen Lai updated IMPALA-11093:
--------------------------------
Description:
If we insert data into an acid partitioned table from Hive, the generated events will be like open_txn -> alter_partition -> commit_txn.
In alter_partition event, we will refresh the partition (without refreshing file metadata) because the write id with the partition object in the event > write id of the cached partition. Then, in commit_txn event, we still do the same write id check. However, the write id is still the same, so we end up don't fresh file metadata.
was:
If we insert data into an acid partitioned table from Hive, the generated events will be like open_txn -> alter_partition -> commit_txn.
Let's assume the allocated write id in this transaction is 3.
In alter_partition event, we will refresh the partition (without refreshing file metadata) because the write id with the partition object in the event > write id of the cache partition. Then, in commit_txn event, we still do the same write id check. However, the write id is still the same, so we end up don't fresh file metadata.
> Fine grained table refreshing doesn't refresh table file metadata
> -----------------------------------------------------------------
>
> Key: IMPALA-11093
> URL: https://issues.apache.org/jira/browse/IMPALA-11093
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Yu-Wen Lai
> Assignee: Yu-Wen Lai
> Priority: Major
>
> If we insert data into an acid partitioned table from Hive, the generated events will be like open_txn -> alter_partition -> commit_txn.
> In alter_partition event, we will refresh the partition (without refreshing file metadata) because the write id with the partition object in the event > write id of the cached partition. Then, in commit_txn event, we still do the same write id check. However, the write id is still the same, so we end up don't fresh file metadata.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org