You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "devmadhuu (via GitHub)" <gi...@apache.org> on 2025/02/04 14:57:26 UTC
[PR] HDDS-11953. Ozone Recon - Improve Recon OM sync process based on continuous pull of OM data. [ozone]
devmadhuu opened a new pull request, #7810:
URL: https://github.com/apache/ozone/pull/7810
## What changes were proposed in this pull request?
This PR change is to get rid of interval delay configs for Recon OM DB snapshot sync with active OM DB because these configs may not make much sense when OM write TPS is very high in the range of more than 5k.
Currently Recon sync with OM based on following configs:
ozone.recon.om.snapshot.task.interval.delay -> 5s
recon.om.delta.update.limit -> 50000
recon.om.delta.update.loop.limit -> 50
Above are recommended and default configs for high write TPS workload in the range of approx 5k to achieve near real time sync between Recon and OM data. However in future, TPS may achieve higher targets in OM and Recon needs to pull data continuously to keep lag and latency minimum.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-11953
## How was this patch tested?
Tested with junit and integration tests of recon OM DB sync.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11953. Improve Recon OM sync process based on continuous pull of OM data. [ozone]
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on code in PR #7810:
URL: https://github.com/apache/ozone/pull/7810#discussion_r1948970268
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java:
##########
@@ -311,7 +310,7 @@ public boolean triggerSyncDataFromOMImmediately() {
startSyncDataFromOM(0L);
return true;
} else {
- LOG.debug("OM DB sync is already running.");
+ LOG.info("OM DB sync is already running when trying to trigger OM DB sync manually.");
Review Comment:
Not CLI tool, but we have Rest API to trigger above manually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11953. Improve Recon OM sync process based on continuous pull of OM data. [ozone]
Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl commented on code in PR #7810:
URL: https://github.com/apache/ozone/pull/7810#discussion_r1948850529
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java:
##########
@@ -311,7 +310,7 @@ public boolean triggerSyncDataFromOMImmediately() {
startSyncDataFromOM(0L);
return true;
} else {
- LOG.debug("OM DB sync is already running.");
+ LOG.info("OM DB sync is already running when trying to trigger OM DB sync manually.");
Review Comment:
do we have any CLI tool to trigger Manually ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11953. Improve Recon OM sync process based on continuous pull of OM data. [ozone]
Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl merged PR #7810:
URL: https://github.com/apache/ozone/pull/7810
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org