[jira] [Created] (HIVE-21772) Support dynamic addition and deletion of partitions in the policy

["mahesh kumar behera (JIRA)" <ji...@apache.org>]

mahesh kumar behera created HIVE-21772:
------------------------------------------

             Summary: Support dynamic addition and deletion of partitions in the policy
                 Key: HIVE-21772
                 URL: https://issues.apache.org/jira/browse/HIVE-21772
             Project: Hive
          Issue Type: Sub-task
    Affects Versions: 4.0.0
            Reporter: mahesh kumar behera
            Assignee: mahesh kumar behera
             Fix For: 4.0.0


If the user modify the filter condition in the policy, then the participating partitions of the policy can change. During such scenarios, user needs to provide the old filter condition along with the REPL DUMP command.
 * The old filter will be passed as a string along with ‘with’ clause of the REPL dump command. Need to create the AST from the string to be used for filtering.
 * Convert the string to list of AST, one for each table and make a list of the partitions satisfying the old filter condition.
 * List of partition satisfying the new filter condition will be compared with the old list.
 * If the partition is not present in old but is present in new, then the partition will be added to the list of partitions to be bootstrapped.
 * If the partition is present in old, but not present in new then the partition will be added to the list of partitions to be deleted.
 * During load operation, after all the events are replayed, the list of bootstrap and list of deleted will be read and corresponding action will be executed at target.
 * There is a possibility that the partitioned to be deleted is already deleted using some event replayed, in that case delete will be ignored.
 * Similarly if some partition from the list of bootstrap is already present, then bootstrap will be ignored.
 * As the partition can not be present in both bootstrap and delete list, the list can be executed in parallel.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)