You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Devesh Kumar Singh (Jira)" <ji...@apache.org> on 2025/03/05 09:03:00 UTC
[jira] [Created] (HDDS-12475) Ozone Recon - Handle auto recovery of data during error in reinitialising of all OM tasks when fetching full snapshot of OM DB
Devesh Kumar Singh created HDDS-12475:
-----------------------------------------
Summary: Ozone Recon - Handle auto recovery of data during error in reinitialising of all OM tasks when fetching full snapshot of OM DB
Key: HDDS-12475
URL: https://issues.apache.org/jira/browse/HDDS-12475
Project: Apache Ozone
Issue Type: Task
Components: Ozone Recon
Reporter: Devesh Kumar Singh
Assignee: Devesh Kumar Singh
Ozone Recon - Handle auto recovery of data during error in reinitialising of all OM tasks when fetching full snapshot of OM DB
There could be few edge cases:
* If Recon was stopped for sometime and when it come back online, OM DB compaction, during that downtime, may force recon to fetch full snapshot and reinitialise all OM based recon tasks and in this flow if any of the OM tasks reinitialisation fails, lastRunTaskStatus will confirm failure, but in next run of sync OM iteration, failed task may go for delta updates but had missed OM DB updates in last run of full snapshot completely.
* Even if Recon is up and running in cluster continuously, there is a possibility that Recon may start lagging over a period of time if OM DB write TPS is very high in cluster. In such a case, recon has a mechanism to fall back on full snapshot and reinitialise all OM based recon tasks and in this flow if any of the OM tasks reinitialisation fails, lastRunTaskStatus will confirm failure, but in next run of sync OM iteration, failed task may go for delta updates but had missed OM DB updates in last run of full snapshot completely.
And in both above edge cases, above failures may completely go silent and unnoticed and even existing metrics like lastRunTaskStatus which recorded failure may be overridden with next delta run status which may be success.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org