[jira] [Work logged] (HIVE-25796) Allow metastore clients to fetch remaining events if some of the events are cleaned up

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25796?focusedWorklogId=694206&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-694206 ]

ASF GitHub Bot logged work on HIVE-25796:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 10/Dec/21 20:47
            Start Date: 10/Dec/21 20:47
    Worklog Time Spent: 10m 
      Work Description: vihangk1 opened a new pull request #2864:
URL: https://github.com/apache/hive/pull/2864


   ### What changes were proposed in this pull request?
   Metastore client throws an exception when events are cleaned up from the metastore side and the lastEventId which is being requested is too old. However, it is possible that client does not care if the events are in sequence or not. In such a case, client should be able to pass in a flag to ignore throwing the exception and return the remaining events from the server.
   
   ### Why are the changes needed?
   Introduces a new HMS client API for a use-case which it doesn't handle currently.
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   ### How was this patch tested?
   Added a new unit test.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 694206)
    Remaining Estimate: 0h
            Time Spent: 10m

> Allow metastore clients to fetch remaining events if some of the events are cleaned up
> --------------------------------------------------------------------------------------
>
>                 Key: HIVE-25796
>                 URL: https://issues.apache.org/jira/browse/HIVE-25796
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Vihang Karajgaonkar
>            Assignee: Vihang Karajgaonkar
>            Priority: Minor
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> This is the code snippet from HiveMetastoreClient.java's getNextNotification method
> {noformat}
>       for (NotificationEvent e : rsp.getEvents()) {
>         LOG.debug("Got event with id : {}", e.getEventId());
>         if (e.getEventId() != nextEventId) {
>           if (e.getEventId() == prevEventId) {
>             LOG.error("NOTIFICATION_LOG table has multiple events with the same event Id {}. " +
>                     "Something went wrong when inserting notification events.  Bootstrap the system " +
>                     "again to get back teh consistent replicated state.", prevEventId);
>             throw new IllegalStateException(REPL_EVENTS_WITH_DUPLICATE_ID_IN_METASTORE);
>           } else {
>             LOG.error("Requested events are found missing in NOTIFICATION_LOG table. Expected: {}, Actual: {}. "
>                             + "Probably, cleaner would've cleaned it up. "
>                             + "Try setting higher value for hive.metastore.event.db.listener.timetolive. "
>                             + "Also, bootstrap the system again to get back the consistent replicated state.",
>                     nextEventId, e.getEventId());
>             throw new IllegalStateException(REPL_EVENTS_MISSING_IN_METASTORE);
>           }
>         }
> {noformat}
> Consider the case when a client which caches a event id and tries to fetch the next events since the eventid after long time. In this case, it is possible that Metastore has cleaned up the events because they were more than 24 hrs old. In such a case, this API throws an exception. It is possible that client does not care if the events are not in sequence and hence this exception should be thrown optionally depending on what the client wants.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)