You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "sodonnel (via GitHub)" <gi...@apache.org> on 2023/01/30 12:50:45 UTC
[GitHub] [ozone] sodonnel opened a new pull request, #4220: HDDS-7844
sodonnel opened a new pull request, #4220:
URL: https://github.com/apache/ozone/pull/4220
## What changes were proposed in this pull request?
In order to allow replication commands intended to fix under and mis-replication to run with higher priority that replication commands related to the balancer, we will change the replication supervisor to be a priority queue, ordered by priority, enqueue time.
Commands sent from the balancer will have a low priority, other commands will a normal priority. The default will be NORMAL. This means that balancer commands will not run while there are other replication commands present in the queue.
This means it is not important if the balancer adds a lot of replication commands to the queue and then some nodes go down in the cluster. The replication commands related to the down nodes will automatically get to the front of the queue.
Note that this change does not modify the balancer to sent low priority commands, and all commands sent from SCM will be NORMAL priority for now. Another Jira will make changes to the balancer to send its replicate commands with low priority.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7844
## How was this patch tested?
Add tests to the supervisor to validate the priority ordering works as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kerneltime commented on pull request #4220: HDDS-7844. EC: Add normal and low priority to replication supervisor and commands
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on PR #4220:
URL: https://github.com/apache/ozone/pull/4220#issuecomment-1409007955
@swamirishi can you take a look as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a diff in pull request #4220: HDDS-7844. EC: Add normal and low priority to replication supervisor and commands
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #4220:
URL: https://github.com/apache/ozone/pull/4220#discussion_r1091176445
##########
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java:
##########
@@ -350,6 +357,121 @@ public void taskWithObsoleteTermIsDropped() {
Assert.assertEquals(0, supervisor.getReplicationSuccessCount());
}
+ @Test
+ public void testPriorityOrdering() throws InterruptedException {
+ long deadline = clock.millis() + 1000;
+ long containerId = 1;
+ long term = 1;
+ OzoneConfiguration conf = new OzoneConfiguration();
+ ReplicationServer.ReplicationConfig repConf =
+ conf.getObject(ReplicationServer.ReplicationConfig.class);
+ repConf.setReplicationMaxStreams(1);
+ ReplicationSupervisor supervisor =
+ new ReplicationSupervisor(null, repConf, clock);
+
+ final CountDownLatch indicateRunning = new CountDownLatch(1);
+ final CountDownLatch completeRunning = new CountDownLatch(1);
+ // Going to create 5 tasks below, so this counter needs to be set to 5.
+ final CountDownLatch tasksCompleteLatch = new CountDownLatch(5);
+
+ supervisor.addTask(new BlockingTask(containerId, deadline, term,
+ indicateRunning, completeRunning));
+ // Wait for the first task to block the single threaded executor
+ indicateRunning.await();
+
+ List<String> completionOrder = new ArrayList<>();
+ // Now load some tasks out of order.
+ clock.fastForward(10);
+ supervisor.addTask(new OrderedTask(containerId, deadline, term, clock,
+ LOW, "LOW_10", completionOrder, tasksCompleteLatch));
+ clock.rewind(5);
+ supervisor.addTask(new OrderedTask(containerId, deadline, term, clock,
+ LOW, "LOW_5", completionOrder, tasksCompleteLatch));
+
+ supervisor.addTask(new OrderedTask(containerId, deadline, term, clock,
+ NORMAL, "HIGH_5", completionOrder, tasksCompleteLatch));
+ clock.rewind(4);
+ supervisor.addTask(new OrderedTask(containerId, deadline, term, clock,
+ NORMAL, "HIGH_1", completionOrder, tasksCompleteLatch));
+ clock.fastForward(10);
+ supervisor.addTask(new OrderedTask(containerId, deadline, term, clock,
+ NORMAL, "HIGH_11", completionOrder, tasksCompleteLatch));
+
+ List<String> expectedOrder = new ArrayList<>();
+ expectedOrder.add("HIGH_1");
+ expectedOrder.add("HIGH_5");
Review Comment:
:hand:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel merged pull request #4220: HDDS-7844. EC: Add normal and low priority to replication supervisor and commands
Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel merged PR #4220:
URL: https://github.com/apache/ozone/pull/4220
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org