[jira] [Updated] (HIVE-21774) Support partition level filtering for events with multiple partitions

["Stamatis Zampetakis (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stamatis Zampetakis updated HIVE-21774:
---------------------------------------
    Fix Version/s:     (was: 4.0.0)

I cleared the fixVersion field since this ticket is still open. Please review this ticket and if the fix is already committed to a specific version please set the version accordingly and mark the ticket as RESOLVED.

According to the [JIRA guidelines|https://cwiki.apache.org/confluence/display/Hive/HowToContribute] the fixVersion should be set only when the issue is resolved/closed.

> Support partition level filtering for events with multiple partitions
> ---------------------------------------------------------------------
>
>                 Key: HIVE-21774
>                 URL: https://issues.apache.org/jira/browse/HIVE-21774
>             Project: Hive
>          Issue Type: Sub-task
>          Components: HiveServer2, repl
>    Affects Versions: 4.0.0
>            Reporter: mahesh kumar behera
>            Assignee: mahesh kumar behera
>            Priority: Major
>
> Some of the events in hive can span across multiple partitions, table or even database. Events related to transactions, can span across multiple databases. When a transaction does some write operation, it is added to the write notification log table. During dump of commit transaction event, al the entries present in the write notification log table for that transaction is read and is added to the commit transaction message. In case partition filter is supplied for the dump, only those partitions which are part of the policy should be added to the commit txn message.
>  * All the events which are not partition level will be added to the list of events to be dumped.
>  * Pass the filter condition for the policy to commit transaction message handler (events which are not partition level).
>  * During dump for commit transaction event, extract the events added in the write notification log table and compare it with the filter condition.
>  * If the event from write notification log satisfies the filter condition, then add it to the commit transaction message.
>  * If filter condition is null, then add all the events from write notification log table to commit transaction message.
>  * For events which does not have partition level info like open txn, abort txn etc, just dump the events without any filtering. So it may happen that some of events which are not related to any of the satisfying partition, may get replayed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)