You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/12/17 07:58:57 UTC
[GitHub] [ozone] GlenGeng opened a new pull request #1721: HDDS-4567: SCMContext Phase 1 - Raft Related Info
GlenGeng opened a new pull request #1721:
URL: https://github.com/apache/ozone/pull/1721
## What changes were proposed in this pull request?
We want to provide SCMContext, which would be a single source of truth for some key information that is shared across all components within SCM.
SCMContext holds two kind of key information:
1. RaftServer related info: isLeader, term.
2. SafeMode related info: inSafeMode, preCheckComplete.
For details, please refer to
https://docs.google.com/document/d/1h_3gpC4o2EpuBlcQiJC7MMoZz9JmaMX9CxObSxWU614/edit#heading=h.4f1y2k1ojlx8
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4567
## How was this patch tested?
CI
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] GlenGeng closed pull request #1721: HDDS-4567: SCMContext Phase 1 - Raft Related Info
Posted by GitBox <gi...@apache.org>.
GlenGeng closed pull request #1721:
URL: https://github.com/apache/ozone/pull/1721
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] GlenGeng commented on a change in pull request #1721: HDDS-4567: SCMContext Phase 1 - Raft Related Info
Posted by GitBox <gi...@apache.org>.
GlenGeng commented on a change in pull request #1721:
URL: https://github.com/apache/ozone/pull/1721#discussion_r547778227
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMContext.java
##########
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p/>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p/>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.hadoop.hdds.scm.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.scm.server.StorageContainerManager;
+import org.apache.ratis.proto.RaftProtos;
+import org.apache.ratis.protocol.RaftGroupId;
+import org.apache.ratis.protocol.exceptions.NotLeaderException;
+import org.apache.ratis.server.RaftServer;
+import org.apache.ratis.server.impl.RaftServerImpl;
+import org.apache.ratis.server.impl.RaftServerProxy;
+import org.apache.ratis.util.ExitUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.OptionalLong;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantLock;
+
+/**
+ * SCMContext is the single source of truth for some key information shared
+ * across all components within SCM, including:
+ * - RaftServer related info, e.g., isLeader, term.
+ * - SafeMode related info, e.g., inSafeMode, preCheckComplete.
+ */
+public class SCMContext {
+ private static final Logger LOG = LoggerFactory.getLogger(SCMContext.class);
+
+ private static final SCMContext EMPTY_CONTEXT
+ = new SCMContext(true, 0, null);
+
+ /**
+ * Used by non-HA mode SCM, Recon and Unit Tests.
+ */
+ public static SCMContext emptyContext() {
+ return EMPTY_CONTEXT;
+ }
+
+ private boolean isLeader;
+ private OptionalLong termOpt;
+ private final StorageContainerManager scm;
Review comment:
Good point! In SCMContext phase 2, it will also handle safe mode related things, and in future, SCMContext will be a mediator of all the XxxManagers in SCM. That's why I choose holding `scm` instead of holding `SCMHAManager`. I will revisit this code in my next PR: SCMContext phase 2.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on a change in pull request #1721: HDDS-4567: SCMContext Phase 1 - Raft Related Info
Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #1721:
URL: https://github.com/apache/ozone/pull/1721#discussion_r546204324
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/CloseContainerEventHandler.java
##########
@@ -74,14 +79,13 @@ public void onMessage(ContainerID containerID, EventPublisher publisher) {
.getContainer(containerID);
// Send close command to datanodes, if the container is in CLOSING state
if (container.getState() == LifeCycleState.CLOSING) {
+ SCMCommand<?> command = new CloseContainerCommand(
+ containerID.getId(), container.getPipelineID());
+ command.setTerm(scmContext.getTerm());
Review comment:
Question:
when you do decide to throw the NotLeaderException out rather than handling immediately after `scmContext.getTerm()`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] GlenGeng commented on a change in pull request #1721: HDDS-4567: SCMContext Phase 1 - Raft Related Info
Posted by GitBox <gi...@apache.org>.
GlenGeng commented on a change in pull request #1721:
URL: https://github.com/apache/ozone/pull/1721#discussion_r547775633
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/CloseContainerEventHandler.java
##########
@@ -74,14 +79,13 @@ public void onMessage(ContainerID containerID, EventPublisher publisher) {
.getContainer(containerID);
// Send close command to datanodes, if the container is in CLOSING state
if (container.getState() == LifeCycleState.CLOSING) {
+ SCMCommand<?> command = new CloseContainerCommand(
+ containerID.getId(), container.getPipelineID());
+ command.setTerm(scmContext.getTerm());
Review comment:
`NotLeaderException` is a subclass of `IOException` which is handled in this `onMessage()` method.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on a change in pull request #1721: HDDS-4567: SCMContext Phase 1 - Raft Related Info
Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #1721:
URL: https://github.com/apache/ozone/pull/1721#discussion_r546445491
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMContext.java
##########
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p/>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p/>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.hadoop.hdds.scm.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.scm.server.StorageContainerManager;
+import org.apache.ratis.proto.RaftProtos;
+import org.apache.ratis.protocol.RaftGroupId;
+import org.apache.ratis.protocol.exceptions.NotLeaderException;
+import org.apache.ratis.server.RaftServer;
+import org.apache.ratis.server.impl.RaftServerImpl;
+import org.apache.ratis.server.impl.RaftServerProxy;
+import org.apache.ratis.util.ExitUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.OptionalLong;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantLock;
+
+/**
+ * SCMContext is the single source of truth for some key information shared
+ * across all components within SCM, including:
+ * - RaftServer related info, e.g., isLeader, term.
+ * - SafeMode related info, e.g., inSafeMode, preCheckComplete.
+ */
+public class SCMContext {
+ private static final Logger LOG = LoggerFactory.getLogger(SCMContext.class);
+
+ private static final SCMContext EMPTY_CONTEXT
+ = new SCMContext(true, 0, null);
+
+ /**
+ * Used by non-HA mode SCM, Recon and Unit Tests.
+ */
+ public static SCMContext emptyContext() {
+ return EMPTY_CONTEXT;
+ }
+
+ private boolean isLeader;
+ private OptionalLong termOpt;
+ private final StorageContainerManager scm;
Review comment:
WIll it better to only hold reference to SCMHAManager than scm?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org