You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Devesh Kumar Singh (Jira)" <ji...@apache.org> on 2024/02/09 09:26:00 UTC
[jira] [Updated] (HDDS-10341) Implement a task in Recon to sync full/delta SCM Metadata DB updates at regular intervals.
[ https://issues.apache.org/jira/browse/HDDS-10341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devesh Kumar Singh updated HDDS-10341:
--------------------------------------
Description:
For Recon to have an accurate and updated information related to various states of containers, blocks pending for deletion or to process any other SCM metadata information in Recon, it is important to have full SCM DB snapshot sync at startup of Recon or fall back on full SCM DB snapshot in failure scenarios like while applying delta updates from SCM metadata DB.
There are some gaps where Recon may not know of updates in SCM metadata. E.g. Recon may be down and during downtime, some containers may get created and deleted in SCM, so Recon will never get to know about such containers even when it is started again.
If we implement delta updates and during Recon downtime, this sync task will try to sync SCM DB delta updates since its rocks DB last sequence number and apply all delta updates to various SCM tasks in Recon.
> Implement a task in Recon to sync full/delta SCM Metadata DB updates at regular intervals.
> ------------------------------------------------------------------------------------------
>
> Key: HDDS-10341
> URL: https://issues.apache.org/jira/browse/HDDS-10341
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Ozone Recon
> Environment: For Recon to have an accurate and updated information related to various states of containers, blocks pending for deletion or to process any other SCM metadata information in Recon, it is important to have full SCM DB snapshot sync at startup of Recon or fall back on full SCM DB snapshot in failure scenarios like while applying delta updates from SCM metadata DB.
> There are some gaps where Recon may not know of updates in SCM metadata. E.g. Recon may be down and during downtime, some containers may get created and deleted in SCM, so Recon will never get to know about such containers even when it is started again.
> If we implement delta updates and during Recon downtime, this sync task will try to sync SCM DB delta updates since its rocks DB last sequence number and apply all delta updates to various SCM tasks in Recon.
> Reporter: Devesh Kumar Singh
> Assignee: Devesh Kumar Singh
> Priority: Major
>
> For Recon to have an accurate and updated information related to various states of containers, blocks pending for deletion or to process any other SCM metadata information in Recon, it is important to have full SCM DB snapshot sync at startup of Recon or fall back on full SCM DB snapshot in failure scenarios like while applying delta updates from SCM metadata DB.
> There are some gaps where Recon may not know of updates in SCM metadata. E.g. Recon may be down and during downtime, some containers may get created and deleted in SCM, so Recon will never get to know about such containers even when it is started again.
> If we implement delta updates and during Recon downtime, this sync task will try to sync SCM DB delta updates since its rocks DB last sequence number and apply all delta updates to various SCM tasks in Recon.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org