[jira] [Work logged] (HIVE-22068) Return the last event id dumped as repl status to avoid notification event missing error.

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-22068?focusedWorklogId=296104&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-296104 ]

ASF GitHub Bot logged work on HIVE-22068:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 16/Aug/19 06:32
            Start Date: 16/Aug/19 06:32
    Worklog Time Spent: 10m 
      Work Description: sankarh commented on pull request #742: HIVE-22068 : Add more logging to notification cleaner and replication to track events
URL: https://github.com/apache/hive/pull/742#discussion_r314595417
 
 

 ##########
 File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java
 ##########
 @@ -522,6 +525,25 @@ private int executeIncrementalLoad(DriverContext driverContext) {
       // bootstrap of tables if exist.
       if (builder.hasMoreWork() || work.getPathsToCopyIterator().hasNext() || work.hasBootstrapLoadTasks()) {
         DAGTraversal.traverse(childTasks, new AddDependencyToLeaves(TaskFactory.get(work, conf)));
+      } else if (work.dbNameToLoadIn != null) {
 
 Review comment:
   I think, work.dbNameToLoadIn will be null if you don't specify the name in REPL LOAD command. In this case, we should get the name from DumpMetadata to set the last repl ID.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 296104)
    Time Spent: 20m  (was: 10m)

> Return the last event id dumped as repl status to avoid notification event missing error.
> -----------------------------------------------------------------------------------------
>
>                 Key: HIVE-22068
>                 URL: https://issues.apache.org/jira/browse/HIVE-22068
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Ashutosh Bapat
>            Assignee: Ashutosh Bapat
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: HIVE-22068.01.patch, HIVE-22068.02.patch, HIVE-22068.03.patch, HIVE-22068.04.patch
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> In repl load, update the status of target database to the last event dumped so that repl status returns that and next incremental can specify it as the event from which to start the dump. WIthout that repl status might return and old event which might cause, older events to be dumped again and/or a notification event missing error if the older events are cleaned by the cleaner.
> While at it
>  * Add more logging to DB notification listener cleaner thread
>  ** The time when it considered cleaning, the interval and time before which events were cleared, the min and max id at that time
>  ** how many events were cleared
>  ** min and max id after the cleaning.
>  * In REPL::START document the starting event, end event if specified and the maximum number of events, if specified.
>  *



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)