[jira] [Created] (HDDS-11741) Recon - Implement dynamic scaling in Recon for processing of full OM snapshot by background tasks

["Devesh Kumar Singh (Jira)" <ji...@apache.org>]

Devesh Kumar Singh created HDDS-11741:
-----------------------------------------

             Summary: Recon - Implement dynamic scaling in Recon for processing of full OM snapshot by background tasks
                 Key: HDDS-11741
                 URL: https://issues.apache.org/jira/browse/HDDS-11741
             Project: Apache Ozone
          Issue Type: Task
          Components: Ozone Recon
            Reporter: Devesh Kumar Singh


This scaling can be done only for full snapshot case, not when Recon is receiving incremental set of OM update events in every periodic sync, because incremental events in every sync can also be in huge numbers in high TPS throughput at OM due to high TP write/update workload and all those incremental events has to be processed in sequence only (already mentioned in design doc), We cannot make it concurrent processing of those incremental events and this processing by each task will take time. We cannot speed up the processing of incremental sync events.
Scaling and speed up can only be done in case of full snapshot processing. And this may improve the speed significantly when Recon is trying to catch up with OM after long pause or it was down or during initial catch up or any other case when Recon has to fall back on full snapshot processing...



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org