You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "aryangupta1998 (via GitHub)" <gi...@apache.org> on 2026/08/13 08:40:01 UTC
[PR] HDDS-12208. Give `ozone debug replicas verify` options for OM container location cache behavior. [ozone]
aryangupta1998 opened a new pull request, #11015:
URL: https://github.com/apache/ozone/pull/11015
## What changes were proposed in this pull request?
This change adds explicit ozone debug replicas verify options to control OM container location cache behavior during key verification.
Adds --refresh-container-locations-from-scm to force OM to refresh container locations from SCM before returning key info.
Adds a shorter alias --refresh-from-scm for the same behavior.
Keeps default behavior unchanged (no flag: OM cache path remains as-is, for faster execution).
Updates debug CLI docs and integration tests to cover both the long and short flags.
Replica verification can be affected by stale OM container location cache entries in some situations.
This option gives operators a slower but potentially more reliable verification mode when they need fresh container location data.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-12206
## How was this patch tested?
Tested manually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12208. Give `ozone debug replicas verify` options for OM container location cache behavior. [ozone]
Posted by "aryangupta1998 (via GitHub)" <gi...@apache.org>.
aryangupta1998 commented on code in PR #11015:
URL: https://github.com/apache/ozone/pull/11015#discussion_r3795024457
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneDebugReplicasVerify.java:
##########
@@ -269,6 +269,25 @@ void testChecksumsWithEmptyBlockFile() {
.doesNotContain("Checksum mismatch");
}
+ @Test
+ void testVerifyWithContainerLocationRefreshFromScm() {
+ List<String> parameters = new ArrayList<>();
+ parameters.add(0, getSetConfStringFromConf(ScmConfigKeys.OZONE_SCM_CLIENT_ADDRESS_KEY));
+ parameters.add(0, getSetConfStringFromConf(OMConfigKeys.OZONE_OM_ADDRESS_KEY));
+ parameters.add("replicas");
+ parameters.add("verify");
+ parameters.add("--checksums");
+ parameters.add("--refresh-from-scm");
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12208. Give `ozone debug replicas verify` options for OM container location cache behavior. [ozone]
Posted by "aryangupta1998 (via GitHub)" <gi...@apache.org>.
aryangupta1998 merged PR #11015:
URL: https://github.com/apache/ozone/pull/11015
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12208. Give `ozone debug replicas verify` options for OM container location cache behavior. [ozone]
Posted by "aryangupta1998 (via GitHub)" <gi...@apache.org>.
aryangupta1998 commented on PR #11015:
URL: https://github.com/apache/ozone/pull/11015#issuecomment-5324431186
Thanks @ashishkumar50, @ayushtkn, and @sadanand48 for the reviews!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12208. Give `ozone debug replicas verify` options for OM container location cache behavior. [ozone]
Posted by "aryangupta1998 (via GitHub)" <gi...@apache.org>.
aryangupta1998 commented on code in PR #11015:
URL: https://github.com/apache/ozone/pull/11015#discussion_r3778361782
##########
hadoop-hdds/docs/content/tools/debug/DebugReplicas.md:
##########
@@ -109,6 +110,13 @@ failed checks.
--container-state Check the container and replica states. Containers in
[DELETING, DELETED] states, or it's replicas in
[DELETED, UNHEALTHY, INVALID] states fail the check.
+ --refresh-container-locations-from-scm
+ Force OM to refresh container locations from SCM
+ before returning key info. This can slow down
+ verification but may avoid stale location results
+ from OM cache.
+ --refresh-from-scm Short alias for
+ --refresh-container-locations-from-scm.
Review Comment:
I originally kept two names to balance:
- discoverability/self-documentation (--refresh-container-locations-from-scm)
- typing convenience (--refresh-from-scm)
But this is a debug-only command, so I agree we can keep it simpler and avoid extra surface area. I’m happy to switch to a single flag and keep the description explicit so intent is still clear.
If you’re good with it, I’ll retain only --refresh-from-scm and update help/docs accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12208. Give `ozone debug replicas verify` options for OM container location cache behavior. [ozone]
Posted by "ashishkumar50 (via GitHub)" <gi...@apache.org>.
ashishkumar50 commented on code in PR #11015:
URL: https://github.com/apache/ozone/pull/11015#discussion_r3794764741
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneDebugReplicasVerify.java:
##########
@@ -269,6 +269,25 @@ void testChecksumsWithEmptyBlockFile() {
.doesNotContain("Checksum mismatch");
}
+ @Test
+ void testVerifyWithContainerLocationRefreshFromScm() {
+ List<String> parameters = new ArrayList<>();
+ parameters.add(0, getSetConfStringFromConf(ScmConfigKeys.OZONE_SCM_CLIENT_ADDRESS_KEY));
+ parameters.add(0, getSetConfStringFromConf(OMConfigKeys.OZONE_OM_ADDRESS_KEY));
+ parameters.add("replicas");
+ parameters.add("verify");
+ parameters.add("--checksums");
+ parameters.add("--refresh-from-scm");
Review Comment:
Passing `--refresh-from-scm` true or false will not make any difference in the test. If we just want to verify whether option name is correct then its better to add in UT instead of IT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12208. Give `ozone debug replicas verify` options for OM container location cache behavior. [ozone]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #11015:
URL: https://github.com/apache/ozone/pull/11015#discussion_r3779591896
##########
hadoop-hdds/docs/content/tools/debug/DebugReplicas.md:
##########
@@ -109,6 +110,13 @@ failed checks.
--container-state Check the container and replica states. Containers in
[DELETING, DELETED] states, or it's replicas in
[DELETED, UNHEALTHY, INVALID] states fail the check.
+ --refresh-container-locations-from-scm
+ Force OM to refresh container locations from SCM
+ before returning key info. This can slow down
+ verification but may avoid stale location results
+ from OM cache.
+ --refresh-from-scm Short alias for
+ --refresh-container-locations-from-scm.
Review Comment:
Sounds good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12208. Give `ozone debug replicas verify` options for OM container location cache behavior. [ozone]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #11015:
URL: https://github.com/apache/ozone/pull/11015#discussion_r3776721598
##########
hadoop-hdds/docs/content/tools/debug/DebugReplicas.md:
##########
@@ -109,6 +110,13 @@ failed checks.
--container-state Check the container and replica states. Containers in
[DELETING, DELETED] states, or it's replicas in
[DELETED, UNHEALTHY, INVALID] states fail the check.
+ --refresh-container-locations-from-scm
+ Force OM to refresh container locations from SCM
+ before returning key info. This can slow down
+ verification but may avoid stale location results
+ from OM cache.
+ --refresh-from-scm Short alias for
+ --refresh-container-locations-from-scm.
Review Comment:
I don't catch this, why do we need two names? Why we can't just have one maybe the shorter one, make sure its defination is well, rather than having two and having a short alias for one command. That too a debug command
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org