[jira] [Updated] (HIVE-23574) [HMS] Use notification log to keep the cache up-to-date with all the changes

["Kishen Das (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-23574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kishen Das updated HIVE-23574:
------------------------------
    Summary: [HMS] Use notification log to keep the cache up-to-date with all the changes   (was: Use notification log to keep the cache up-to-date with all the changes )

> [HMS] Use notification log to keep the cache up-to-date with all the changes 
> -----------------------------------------------------------------------------
>
>                 Key: HIVE-23574
>                 URL: https://issues.apache.org/jira/browse/HIVE-23574
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Kishen Das
>            Priority: Major
>
> If the cache is stale, HMS will serve the request from ObjectStore. We need to catch up the cache with the latest change. This can be done by the existing notification log based cache update mechanism. A thread in HMS constantly poll from notification log, update the cache with the entries from notification log. The interesting entries in notification log are table/partition writes, and corresponding commit transaction message. When processing table/partition writes, HMS will put the table/partition entry in cache. However, the entry is not immediately usable until the commit message of the corresponding writes is processed, and mark writeid of corresponding table entry committed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)