You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "sodonnel (via GitHub)" <gi...@apache.org> on 2023/02/08 16:36:40 UTC
[GitHub] [ozone] sodonnel opened a new pull request, #4260: HDDS-7928. EC: Change ContainerReplicaPendingOps to store deadline rather than scheduled time
sodonnel opened a new pull request, #4260:
URL: https://github.com/apache/ozone/pull/4260
## What changes were proposed in this pull request?
In order to facilitate a "Move Manager" for the balancer, we need to schedule pending ops on the datanodes with different deadlines.
For example, as replica scheduled due to replication would have a timeout of 10 minutes.
However the balancer schedules large batches of work each hour, so replicas scheduled by the balancer probably need a timeout of 60 minutes.
To facilitate this, we need to change containerReplicaPending ops to store the deadline rather than the scheduled time. This also fixes another issue, in that the ContainerReplicaPendingOps "expiry thread" had its own setting for the timeout for all pending ops, and it is not related to the replication manager settings.
As part of this change, some APIs into RM has been changed a little to allow the move manager to schedule replication commands via RM, so that logic is consolidated in a single place. A later PR that adds the MoveManager will use these APIs.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7928
## How was this patch tested?
Adjusted the tests for ContainerReplicaPendingOps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel merged pull request #4260: HDDS-7928. EC: Change ContainerReplicaPendingOps to store deadline rather than scheduled time
Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel merged PR #4260:
URL: https://github.com/apache/ozone/pull/4260
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a diff in pull request #4260: HDDS-7928. EC: Change ContainerReplicaPendingOps to store deadline rather than scheduled time
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #4260:
URL: https://github.com/apache/ozone/pull/4260#discussion_r1101301934
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ContainerReplicaOp.java:
##########
@@ -47,7 +47,7 @@ public ContainerReplicaOp(PendingOpType opType,
this.opType = opType;
this.target = target;
this.replicaIndex = replicaIndex;
- this.scheduledEpochMillis = scheduledTime;
+ this.deadlineEpochMillis = scheduledTime;
Review Comment:
Nit: ideally parameter name should also reflect the new meaning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org