You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2026/07/03 09:11:53 UTC
[PR] HDDS-15730. Support more StatefulService types in DBScanner [ozone]
adoroszlai opened a new pull request, #10654:
URL: https://github.com/apache/ozone/pull/10654
## What changes were proposed in this pull request?
Refactor SCM `StatefulService` to allow using its static metadata (name, parser) without instantiating the service or duplicating code in `DBScanner`.
Add support for Container Balancer and Root CA Rotation Manager services in `DBScanner`.
https://issues.apache.org/jira/browse/HDDS-15730
## How was this patch tested?
```
$ ozone admin containerbalancer start
Container Balancer started successfully.
$ ozone debug ldb --db=/data/metadata/scm.db scan --cf=statefulServiceConfig
{ "ContainerBalancer": {
"memoizedHashCode" : 0,
"memoizedSize" : -1,
"unknownFields" : {
"fields" : { }
},
"bitField0_" : 262143,
"utilizationThreshold_" : {
"hash" : 0,
"bytes" : "MTA="
},
"datanodesInvolvedMaxPercentagePerIteration_" : 20,
"sizeMovedMaxPerIteration_" : 536870912000,
"sizeEnteringTargetMax_" : 27917287424,
"sizeLeavingSourceMax_" : 27917287424,
"iterations_" : 10,
"excludeContainers_" : {
"hash" : 0,
"bytes" : ""
},
"moveTimeout_" : 3900000,
"balancingIterationInterval_" : 4200000,
"includeDatanodes_" : {
"hash" : 0,
"bytes" : ""
},
"excludeDatanodes_" : {
"hash" : 0,
"bytes" : ""
},
"moveNetworkTopologyEnable_" : false,
"triggerDuBeforeMoveEnable_" : false,
"shouldRun_" : false,
"nextIterationIndex_" : 0,
"moveReplicationTimeout_" : 3000000,
"includeContainers_" : {
"hash" : 0,
"bytes" : ""
},
"includeNonStandardContainers_" : false,
"memoizedIsInitialized" : 1
}
}
```
https://github.com/adoroszlai/ozone/actions/runs/28643243287
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15730. Support more StatefulService types in DBScanner [ozone]
Posted by "sreejasahithi (via GitHub)" <gi...@apache.org>.
sreejasahithi commented on code in PR #10654:
URL: https://github.com/apache/ozone/pull/10654#discussion_r3531470256
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogStateManagerImpl.java:
##########
@@ -54,6 +55,9 @@ public class DeletedBlockLogStateManagerImpl
private volatile Set<Long> deletingTxIDs;
public static final String SERVICE_NAME = DeletedBlockLogStateManager.class.getSimpleName();
+ public static final StatefulServiceDefinition<DeletedBlocksTransaction> SERVICE_DEFINITION =
+ new StatefulServiceDefinition<>(SERVICE_NAME, DeletedBlocksTransaction.parser());
+
Review Comment:
SERVICE_DEFINITION here should use DeletedBlocksTransactionSummary, not DeletedBlocksTransaction:
```suggestion
public static final StatefulServiceDefinition<DeletedBlocksTransactionSummary> SERVICE_DEFINITION =
new StatefulServiceDefinition<>(SERVICE_NAME, DeletedBlocksTransactionSummary.parser());
```
The old DBScanner also parsed it as DeletedBlocksTransactionSummary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15730. Support more StatefulService types in DBScanner [ozone]
Posted by "sreejasahithi (via GitHub)" <gi...@apache.org>.
sreejasahithi commented on code in PR #10654:
URL: https://github.com/apache/ozone/pull/10654#discussion_r3531470256
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogStateManagerImpl.java:
##########
@@ -54,6 +55,9 @@ public class DeletedBlockLogStateManagerImpl
private volatile Set<Long> deletingTxIDs;
public static final String SERVICE_NAME = DeletedBlockLogStateManager.class.getSimpleName();
+ public static final StatefulServiceDefinition<DeletedBlocksTransaction> SERVICE_DEFINITION =
+ new StatefulServiceDefinition<>(SERVICE_NAME, DeletedBlocksTransaction.parser());
+
Review Comment:
I think SERVICE_DEFINITION here should use DeletedBlocksTransactionSummary, not DeletedBlocksTransaction:
```suggestion
public static final StatefulServiceDefinition<DeletedBlocksTransactionSummary> SERVICE_DEFINITION =
new StatefulServiceDefinition<>(SERVICE_NAME, DeletedBlocksTransactionSummary.parser());
```
The old DBScanner also parsed it as DeletedBlocksTransactionSummary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15730. Support more StatefulService types in DBScanner [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #10654:
URL: https://github.com/apache/ozone/pull/10654#issuecomment-4954797540
Thanks @ChenSammi, @sreejasahithi for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15730. Support more StatefulService types in DBScanner [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #10654:
URL: https://github.com/apache/ozone/pull/10654#discussion_r3531861050
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogStateManagerImpl.java:
##########
@@ -54,6 +55,9 @@ public class DeletedBlockLogStateManagerImpl
private volatile Set<Long> deletingTxIDs;
public static final String SERVICE_NAME = DeletedBlockLogStateManager.class.getSimpleName();
+ public static final StatefulServiceDefinition<DeletedBlocksTransaction> SERVICE_DEFINITION =
+ new StatefulServiceDefinition<>(SERVICE_NAME, DeletedBlocksTransaction.parser());
+
Review Comment:
Thanks for spotting this. The other two definitions are used with subclasses of `StatefulService`, which ensures type safety. I don't know why this one is different, but now made some further change: use the definition in `loadDeletedBlocksSummary()`, which returns `DeletedBlocksTransactionSummary`, so that type mismatch is caught at compile time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15730. Support more StatefulService types in DBScanner [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #10654:
URL: https://github.com/apache/ozone/pull/10654#issuecomment-4895103219
@sreejasahithi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15730. Support more StatefulService types in DBScanner [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi merged PR #10654:
URL: https://github.com/apache/ozone/pull/10654
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org