You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/08/12 02:46:33 UTC
[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1314: HDDS-3988: DN can distinguish SCMCommand from stale leader SCM
timmylicheng commented on a change in pull request #1314:
URL: https://github.com/apache/hadoop-ozone/pull/1314#discussion_r468973335
##########
File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/StateContext.java
##########
@@ -470,6 +484,65 @@ public void execute(ExecutorService service, long time, TimeUnit unit)
}
}
+ /**
+ * After startup, datanode needs detect latest leader SCM before handling
+ * any SCMCommand, so that it won't be disturbed by stale leader SCM.
+ *
+ * The rule is: after majority SCMs are in HEARTBEAT state and has
+ * heard from leader SCMs (commandQueue is not empty), datanode will init
+ * termOfLeaderSCM with the max term found in commandQueue.
+ *
+ * The init process also works for non-HA mode. In that case, term of all
+ * SCMCommands will be 0.
+ */
+ void initTermOfLeaderSCM() {
+ // only init once
+ if (termOfLeaderSCM.isPresent()) {
+ return;
+ }
+
+ AtomicInteger scmNum = new AtomicInteger(0);
+ AtomicInteger activeScmNum = new AtomicInteger(0);
+
+ getParent().getConnectionManager().getValues()
+ .forEach(endpoint -> {
+ if (endpoint.isPassive()) {
+ return;
+ }
+ scmNum.incrementAndGet();
+ if (endpoint.getState()
+ == EndpointStateMachine.EndPointStates.HEARTBEAT) {
+ activeScmNum.incrementAndGet();
+ }
+ });
+
+ // majority SCMs should be in HEARTBEAT state.
+ if (activeScmNum.get() < scmNum.get() / 2 + 1) {
+ return;
+ }
+
+ // if commandQueue is not empty, init termOfLeaderSCM
+ // with the largest term found in commandQueue
+ commandQueue.stream()
+ .mapToLong(SCMCommand::getTerm)
+ .max()
+ .ifPresent(term -> termOfLeaderSCM = Optional.of(term));
+ }
+
+ /**
+ * monotonically increase termOfLeaderSCM.
+ * Always record the latest term that has seen.
+ */
+ void updateTermOfLeaderSCM(SCMCommand<?> command) {
Review comment:
Can be a private method
##########
File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/StateContext.java
##########
@@ -470,6 +484,65 @@ public void execute(ExecutorService service, long time, TimeUnit unit)
}
}
+ /**
+ * After startup, datanode needs detect latest leader SCM before handling
+ * any SCMCommand, so that it won't be disturbed by stale leader SCM.
+ *
+ * The rule is: after majority SCMs are in HEARTBEAT state and has
+ * heard from leader SCMs (commandQueue is not empty), datanode will init
+ * termOfLeaderSCM with the max term found in commandQueue.
+ *
+ * The init process also works for non-HA mode. In that case, term of all
+ * SCMCommands will be 0.
+ */
+ void initTermOfLeaderSCM() {
Review comment:
Can be a private method
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org