[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1255: HDDS-4021. Organize Recon DBs into a 'DBDefinition'.

[GitBox <gi...@apache.org>]

avijayanhwx commented on pull request #1255:
URL: https://github.com/apache/hadoop-ozone/pull/1255#issuecomment-665693849


   @elek Thanks for looking at the patch.
   
   > I suppose they are separated because the 1 and 2 supposed to be a mirror of original SCM and OM dbs. OM db is updated 
   > with downloaded snapshots which (I guess) required to be a separated DB
   
   Yes, that is correct. The container key DB is a by product of processing the OM snapshot DB. It hosts the reverse mapping of the OM data, the Container->Key mapping in a specific format. Based on whether we take a whole snapshot or just apply delta changes from the OM DB, we have to create a "new" container key DB or just add more entries to it. Hence, the "timestamp" is associated with its name. In a steady state Recon, the container key DB will be created once, and only delta updates will be added to it. The same thing could also have been done by adding a new column family to the OM snapshot DB, and then just storing the reverse mapping there. But, to maintain separation of concerns, and possible scale issues in the future (since num_record(container key DB) == num_records(OM DB)), we had gone with this approach. We can definitely revisit this logic. 
   
   Moreover, in the future, the recon container key DB will likely be reused by Recon to manage all its LSM metadata, thereby limiting the number of RocksDBs in Recon to 3 (OM, SCM and Recon's internal). 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org