You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Naveen Gangam (Jira)" <ji...@apache.org> on 2023/09/21 13:56:00 UTC

[jira] [Created] (HIVE-27720) Optimize NotificationLog Cleaner

Naveen Gangam created HIVE-27720:
------------------------------------

             Summary: Optimize NotificationLog Cleaner 
                 Key: HIVE-27720
                 URL: https://issues.apache.org/jira/browse/HIVE-27720
             Project: Hive
          Issue Type: Improvement
            Reporter: Naveen Gangam
            Assignee: Naveen Gangam


Currently this thread fetches the events first in batches of 10k (or whatever configured) and then deletes them. This is not very optimal. This table has an index on the EVENT_ID which are unique. So the records have high cardinality. The cleaner thread deletes using high EVENT_TIME which also has high cardinality. So the fetching of this events appears to be doing a full table scan each time even for a following batches.

Instead of fetching, we should be able to do a delete directly using JDO. This way we can delete them all in one go at the DB level.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)