[jira] [Created] (HDDS-6312) Recon took too much time to delete records in ContainerKeyPrefix Table

["Janus Chow (Jira)" <ji...@apache.org>]

Janus Chow created HDDS-6312:
--------------------------------

             Summary: Recon took too much time to delete records in ContainerKeyPrefix Table
                 Key: HDDS-6312
                 URL: https://issues.apache.org/jira/browse/HDDS-6312
             Project: Apache Ozone
          Issue Type: Improvement
            Reporter: Janus Chow
            Assignee: Janus Chow


Recon stores the mapping of ContainerKeyPrefix in local RocksDB. When Recon is applying DELETE or UPDATE events from OM, it will run search the whole table for each to_be_deleted record.

In a big cluster, the record count in this table could be very large, and the search loop for each records is very slow. In our cluster there are 90m records, each loop cost over 70 seconds, if a delta OM events have 100 DELETE or UPDATE events, it will took about two hours to apply these updates.

 

This ticket is to accelerate the process with the help of a new local table.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org