You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Sankar Hariappan (JIRA)" <ji...@apache.org> on 2018/03/13 20:06:00 UTC
[jira] [Updated] (HIVE-18747) Cleaner for TXN_TO_WRITE_ID table
entries/MIN_HISTORY_LEVEL.
[ https://issues.apache.org/jira/browse/HIVE-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sankar Hariappan updated HIVE-18747:
------------------------------------
Description:
Per table write ID implementation (HIVE-18192) maintains a map between txn ID and table write ID in TXN_TO_WRITE_ID meta table.
The entries in this table is used to generate ValidWriteIdList for the given ValidTxnList to ensure snapshot isolation.
When table or database is dropped, then these entries are cleaned-up. But, it is necessary to clean-up for active tables too for better performance.
Need to have another table MIN_HISTORY_LEVEL to maintain the least txn which is referred by any active ValidTxnList snapshot as open/aborted txn. If no references found in this table for any txn, then it is eligible for cleanup.
After clean-up, need to maintain just one entry (highest committed txn) per table to mark as LWM (low water mark).
was:
Per table write ID implementation (HIVE-18192) maintains a map between txn ID and table write ID in TXN_TO_WRITE_ID meta table.
The entries in this table is used to generate ValidWriteIdList for the given ValidTxnList to ensure snapshot isolation.
When table or database is dropped, then these entries are cleaned-up. But, it is necessary to clean-up for active tables too for better performance.
Need to have another table MIN_HISTORY_LEVEL to maintain the least txn which is referred by any active ValidTxnList snapshot as open/aborted txn. If no references found in this table for any txn, then it is eligible for cleanup.
After clean-up, need to maintain just one entry per table to mark as LWM (low water mark).
> Cleaner for TXN_TO_WRITE_ID table entries/MIN_HISTORY_LEVEL.
> ------------------------------------------------------------
>
> Key: HIVE-18747
> URL: https://issues.apache.org/jira/browse/HIVE-18747
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
> Affects Versions: 3.0.0
> Reporter: Sankar Hariappan
> Assignee: Sankar Hariappan
> Priority: Minor
> Labels: ACID
> Fix For: 3.0.0
>
>
> Per table write ID implementation (HIVE-18192) maintains a map between txn ID and table write ID in TXN_TO_WRITE_ID meta table.
> The entries in this table is used to generate ValidWriteIdList for the given ValidTxnList to ensure snapshot isolation.
> When table or database is dropped, then these entries are cleaned-up. But, it is necessary to clean-up for active tables too for better performance.
> Need to have another table MIN_HISTORY_LEVEL to maintain the least txn which is referred by any active ValidTxnList snapshot as open/aborted txn. If no references found in this table for any txn, then it is eligible for cleanup.
> After clean-up, need to maintain just one entry (highest committed txn) per table to mark as LWM (low water mark).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)