You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/10/24 12:11:10 UTC
[GitHub] [ozone] kaijchen opened a new pull request, #3877: HDDS-7396. Force close non-RATIS containers in ReplicationManager
kaijchen opened a new pull request, #3877:
URL: https://github.com/apache/ozone/pull/3877
## What changes were proposed in this pull request?
Force close non-RATIS containers in ReplicationManager.
https://github.com/apache/ozone/blob/965d31cde91ba7c9973beb9e082f7c0e492e18a8/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/CloseContainerCommandHandler.java#L106-L113
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7396
## How was this patch tested?
TestClosingContainerHandler#testOpenOrClosingReplicasAreClosed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #3877: HDDS-7396. Force close non-RATIS containers in ReplicationManager
Posted by GitBox <gi...@apache.org>.
kaijchen commented on PR #3877:
URL: https://github.com/apache/ozone/pull/3877#issuecomment-1291406938
Thanks @sodonnel @siddhantsangwan @JacksonYao287 for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on a diff in pull request #3877: HDDS-7396. Force close non-RATIS containers in ReplicationManager
Posted by GitBox <gi...@apache.org>.
kaijchen commented on code in PR #3877:
URL: https://github.com/apache/ozone/pull/3877#discussion_r1004179998
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/health/TestClosingContainerHandler.java:
##########
@@ -153,32 +164,59 @@ public void testUnhealthyRatisReplicaIsNotClosed() {
/**
* Close commands should be sent for Open or Closing replicas.
*/
- @Test
- public void testOpenOrClosingReplicasAreClosed() {
+ @ParameterizedTest
+ @MethodSource("replicationConfigs")
+ public void testOpenOrClosingReplicasAreClosed(ReplicationConfig repConfig) {
Review Comment:
Yeah, it's covered by the parameterized test.
Thanks for pointing this out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on a diff in pull request #3877: HDDS-7396. Force close non-RATIS containers in ReplicationManager
Posted by GitBox <gi...@apache.org>.
kaijchen commented on code in PR #3877:
URL: https://github.com/apache/ozone/pull/3877#discussion_r1003575342
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/health/TestClosingContainerHandler.java:
##########
@@ -153,32 +163,55 @@ public void testUnhealthyRatisReplicaIsNotClosed() {
/**
* Close commands should be sent for Open or Closing replicas.
*/
- @Test
- public void testOpenOrClosingReplicasAreClosed() {
+ @ParameterizedTest
+ @MethodSource("replicationConfigs")
+ public void testOpenOrClosingReplicasAreClosed(ReplicationConfig repConfig) {
ContainerInfo containerInfo = ReplicationTestUtil.createContainerInfo(
- ecReplicationConfig, 1, CLOSING);
- Set<ContainerReplica> containerReplicas = ReplicationTestUtil
- .createReplicas(containerInfo.containerID(),
- ContainerReplicaProto.State.CLOSING, 1, 2);
- containerReplicas.add(ReplicationTestUtil.createContainerReplica(
- containerInfo.containerID(), 3,
- HddsProtos.NodeOperationalState.IN_SERVICE,
- ContainerReplicaProto.State.OPEN));
+ repConfig, 1, CLOSING);
+
+ final int replicas = repConfig.getRequiredNodes();
+ final int closing = replicas / 2 + 1;
+ final boolean force = repConfig.getReplicationType() != RATIS;
+
+ Set<ContainerReplica> containerReplicas = new HashSet<>();
+
+ // Add CLOSING container replicas with index [1, closing]
+ for (int i = 1; i <= closing; i++) {
+ containerReplicas.add(ReplicationTestUtil.createContainerReplica(
+ containerInfo.containerID(), i,
Review Comment:
I have set it to `ReplicationType == EC ? i : 0` in case of `STAND_ALONE` or `CHAINED` being passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on a diff in pull request #3877: HDDS-7396. Force close non-RATIS containers in ReplicationManager
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on code in PR #3877:
URL: https://github.com/apache/ozone/pull/3877#discussion_r1004171087
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/health/TestClosingContainerHandler.java:
##########
@@ -153,32 +164,59 @@ public void testUnhealthyRatisReplicaIsNotClosed() {
/**
* Close commands should be sent for Open or Closing replicas.
*/
- @Test
- public void testOpenOrClosingReplicasAreClosed() {
+ @ParameterizedTest
+ @MethodSource("replicationConfigs")
+ public void testOpenOrClosingReplicasAreClosed(ReplicationConfig repConfig) {
Review Comment:
Parameterizing this is a good idea. If I understand correctly, this test will now run for both EC and RATIS replication configs. If so, we can delete the next test `testOpenOrClosingRatisReplicasAreClosed()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on a diff in pull request #3877: HDDS-7396. Force close non-RATIS containers in ReplicationManager
Posted by GitBox <gi...@apache.org>.
sodonnel commented on code in PR #3877:
URL: https://github.com/apache/ozone/pull/3877#discussion_r1003551310
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/health/TestClosingContainerHandler.java:
##########
@@ -153,32 +163,55 @@ public void testUnhealthyRatisReplicaIsNotClosed() {
/**
* Close commands should be sent for Open or Closing replicas.
*/
- @Test
- public void testOpenOrClosingReplicasAreClosed() {
+ @ParameterizedTest
+ @MethodSource("replicationConfigs")
+ public void testOpenOrClosingReplicasAreClosed(ReplicationConfig repConfig) {
ContainerInfo containerInfo = ReplicationTestUtil.createContainerInfo(
- ecReplicationConfig, 1, CLOSING);
- Set<ContainerReplica> containerReplicas = ReplicationTestUtil
- .createReplicas(containerInfo.containerID(),
- ContainerReplicaProto.State.CLOSING, 1, 2);
- containerReplicas.add(ReplicationTestUtil.createContainerReplica(
- containerInfo.containerID(), 3,
- HddsProtos.NodeOperationalState.IN_SERVICE,
- ContainerReplicaProto.State.OPEN));
+ repConfig, 1, CLOSING);
+
+ final int replicas = repConfig.getRequiredNodes();
+ final int closing = replicas / 2 + 1;
+ final boolean force = repConfig.getReplicationType() != RATIS;
+
+ Set<ContainerReplica> containerReplicas = new HashSet<>();
+
+ // Add CLOSING container replicas with index [1, closing]
+ for (int i = 1; i <= closing; i++) {
+ containerReplicas.add(ReplicationTestUtil.createContainerReplica(
+ containerInfo.containerID(), i,
Review Comment:
Minor point, but just incase it causes problems in the future, we should fix it.
Ratis replicas should always have index = 0. EC replicas should always have indexes >= 1.
Based on the force flag, you could set the index to `force == true ? i : 0` in the two loops.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 merged pull request #3877: HDDS-7396. Force close non-RATIS containers in ReplicationManager
Posted by GitBox <gi...@apache.org>.
JacksonYao287 merged PR #3877:
URL: https://github.com/apache/ozone/pull/3877
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on a diff in pull request #3877: HDDS-7396. Force close non-RATIS containers in ReplicationManager
Posted by GitBox <gi...@apache.org>.
kaijchen commented on code in PR #3877:
URL: https://github.com/apache/ozone/pull/3877#discussion_r1003552545
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/health/TestClosingContainerHandler.java:
##########
@@ -153,32 +163,55 @@ public void testUnhealthyRatisReplicaIsNotClosed() {
/**
* Close commands should be sent for Open or Closing replicas.
*/
- @Test
- public void testOpenOrClosingReplicasAreClosed() {
+ @ParameterizedTest
+ @MethodSource("replicationConfigs")
+ public void testOpenOrClosingReplicasAreClosed(ReplicationConfig repConfig) {
ContainerInfo containerInfo = ReplicationTestUtil.createContainerInfo(
- ecReplicationConfig, 1, CLOSING);
- Set<ContainerReplica> containerReplicas = ReplicationTestUtil
- .createReplicas(containerInfo.containerID(),
- ContainerReplicaProto.State.CLOSING, 1, 2);
- containerReplicas.add(ReplicationTestUtil.createContainerReplica(
- containerInfo.containerID(), 3,
- HddsProtos.NodeOperationalState.IN_SERVICE,
- ContainerReplicaProto.State.OPEN));
+ repConfig, 1, CLOSING);
+
+ final int replicas = repConfig.getRequiredNodes();
+ final int closing = replicas / 2 + 1;
+ final boolean force = repConfig.getReplicationType() != RATIS;
+
+ Set<ContainerReplica> containerReplicas = new HashSet<>();
+
+ // Add CLOSING container replicas with index [1, closing]
+ for (int i = 1; i <= closing; i++) {
+ containerReplicas.add(ReplicationTestUtil.createContainerReplica(
+ containerInfo.containerID(), i,
Review Comment:
Thanks, I missed that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org