[jira] [Updated] (HIVE-18923) ValidWriteIdList snapshot per table can be cached for multi-statement transactions.

["Vineet Garg (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-18923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vineet Garg updated HIVE-18923:
-------------------------------
    Fix Version/s:     (was: 3.1.0)
                   3.2.0

Deferring this to 3.2.0 since the branch for 3.1.0 has been cut off.

> ValidWriteIdList snapshot per table can be cached for multi-statement transactions.
> -----------------------------------------------------------------------------------
>
>                 Key: HIVE-18923
>                 URL: https://issues.apache.org/jira/browse/HIVE-18923
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Transactions
>    Affects Versions: 3.0.0
>            Reporter: Sankar Hariappan
>            Assignee: Sankar Hariappan
>            Priority: Minor
>              Labels: ACID
>             Fix For: 3.2.0
>
>
> Currently, for each query within a multi-statement transaction, it would request metastore/TxnHandler to build ValidWriteIdList snapshot for the given table. This is costly as it need to talk to metastore RDBMS. But, the snapshot won't change within the duration of transaction. So, it make sense to cache it within QueryTxnManager for better performance.
> However, each txn should be able to view their own written rows. So, when a transaction allocates writeId to write on a table, then the cached ValidWriteIdList on this table should be recalculated as follows.
> *Original ValidWriteIdList:* \{hwm=10, open/aborted=5,6} – (10 is allocated by txn < current txn_id).
> *Allocated writeId for this txn:* 13 – (11 and 12 are taken by some other txn > current txn_id)
> *New ValidWriteIdList:* \{hwm=12, open/aborted=5,6,11, 12} – (11, 12 are added to invalid list, so the snapshot remains same).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)