You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@seata.apache.org by "funky-eyes (via GitHub)" <gi...@apache.org> on 2025/12/22 06:39:05 UTC

[PR] bugfix: raft split-brain causes incorrect cluster information [incubator-seata]

funky-eyes opened a new pull request, #7891:
URL: https://github.com/apache/incubator-seata/pull/7891

   <!--
       Licensed to the Apache Software Foundation (ASF) under one or more
       contributor license agreements.  See the NOTICE file distributed with
       this work for additional information regarding copyright ownership.
       The ASF licenses this file to You under the Apache License, Version 2.0
       (the "License"); you may not use this file except in compliance with
       the License.  You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
       
       Unless required by applicable law or agreed to in writing, software
       distributed under the License is distributed on an "AS IS" BASIS,
       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
       See the License for the specific language governing permissions and
       limitations under the License.
   -->
   <!-- Please make sure you have read and understood the contributing guidelines -->
   
   - [ ] I have read the [CONTRIBUTING.md](https://github.com/apache/incubator-seata/blob/2.x/CONTRIBUTING.md) guidelines.
   - [ ] I have registered the PR [changes](https://github.com/apache/incubator-seata/tree/2.x/changes).
   
   ### Ⅰ. Describe what this PR did
   
   When the old leader is re-elected, it sends a heartbeat to the Naming Server—targeting either itself or the (former) leader. Although the term has already changed, the heartbeat thread and the re-election thread run concurrently. As a result, the leader status hasn't been updated yet when the heartbeat is sent. Therefore, the node needs to be corrected during the next heartbeat cycle. To achieve this, we should flag the role; if the role differs, it indicates that the node has undergone a change.
   
   ### Ⅱ. Does this pull request fix one issue?
   <!-- If that, add "fixes #xxx" below in the next line, for example, fixes #97. -->
   
   
   ### Ⅲ. Why don't you add test cases (unit test/integration test)? 
   
   
   ### Ⅳ. Describe how to verify it
   
   
   ### Ⅴ. Special notes for reviews
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org


Re: [PR] bugfix: raft split-brain causes incorrect cluster information [incubator-seata]

Posted by "funky-eyes (via GitHub)" <gi...@apache.org>.
funky-eyes merged PR #7891:
URL: https://github.com/apache/incubator-seata/pull/7891


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org


Re: [PR] bugfix: raft split-brain causes incorrect cluster information [incubator-seata]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #7891:
URL: https://github.com/apache/incubator-seata/pull/7891#issuecomment-3680716865

   ## [Codecov](https://app.codecov.io/gh/apache/incubator-seata/pull/7891?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 71.09%. Comparing base ([`5219ca3`](https://app.codecov.io/gh/apache/incubator-seata/commit/5219ca3fa28297a2d66accb4253e9cbb48468ed1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`4d8c184`](https://app.codecov.io/gh/apache/incubator-seata/commit/4d8c184cbbf6f2a0704ad0d4409986eee8d3490b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/incubator-seata/pull/7891?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [...ta/server/instance/RaftServerInstanceStrategy.java](https://app.codecov.io/gh/apache/incubator-seata/pull/7891?src=pr&el=tree&filepath=server%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fseata%2Fserver%2Finstance%2FRaftServerInstanceStrategy.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zZWF0YS9zZXJ2ZXIvaW5zdGFuY2UvUmFmdFNlcnZlckluc3RhbmNlU3RyYXRlZ3kuamF2YQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/incubator-seata/pull/7891?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##                2.x    #7891      +/-   ##
   ============================================
   - Coverage     71.09%   71.09%   -0.01%     
     Complexity      797      797              
   ============================================
     Files          1300     1300              
     Lines         49602    49600       -2     
     Branches       5873     5874       +1     
   ============================================
   - Hits          35266    35262       -4     
   + Misses        11426    11423       -3     
   - Partials       2910     2915       +5     
   ```
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/incubator-seata/pull/7891?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [...common/metadata/namingserver/NamingServerNode.java](https://app.codecov.io/gh/apache/incubator-seata/pull/7891?src=pr&el=tree&filepath=common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fseata%2Fcommon%2Fmetadata%2Fnamingserver%2FNamingServerNode.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zZWF0YS9jb21tb24vbWV0YWRhdGEvbmFtaW5nc2VydmVyL05hbWluZ1NlcnZlck5vZGUuamF2YQ==) | `92.30% <100.00%> (+0.30%)` | :arrow_up: |
   | [...ta/server/instance/RaftServerInstanceStrategy.java](https://app.codecov.io/gh/apache/incubator-seata/pull/7891?src=pr&el=tree&filepath=server%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fseata%2Fserver%2Finstance%2FRaftServerInstanceStrategy.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zZWF0YS9zZXJ2ZXIvaW5zdGFuY2UvUmFmdFNlcnZlckluc3RhbmNlU3RyYXRlZ3kuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   
   ... and [7 files with indirect coverage changes](https://app.codecov.io/gh/apache/incubator-seata/pull/7891/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   
   [![Impacted file tree graph](https://app.codecov.io/gh/apache/incubator-seata/pull/7891/graphs/tree.svg?width=650&height=150&src=pr&token=tbmHt2ZfxO&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)](https://app.codecov.io/gh/apache/incubator-seata/pull/7891?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org


Re: [PR] bugfix: raft split-brain causes incorrect cluster information [incubator-seata]

Posted by "funky-eyes (via GitHub)" <gi...@apache.org>.
funky-eyes commented on code in PR #7891:
URL: https://github.com/apache/incubator-seata/pull/7891#discussion_r2639066649


##########
server/src/test/java/org/apache/seata/server/instance/RaftServerInstanceStrategyTest.java:
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.seata.server.instance;
+
+import com.alipay.sofa.jraft.entity.PeerId;
+import org.apache.seata.common.XID;
+import org.apache.seata.common.holder.ObjectHolder;
+import org.apache.seata.common.metadata.ClusterRole;
+import org.apache.seata.common.metadata.Instance;
+import org.apache.seata.common.metadata.Node;
+import org.apache.seata.common.store.SessionMode;
+import org.apache.seata.server.BaseSpringBootTest;
+import org.apache.seata.server.cluster.listener.ClusterChangeEvent;
+import org.apache.seata.server.cluster.raft.RaftServer;
+import org.apache.seata.server.cluster.raft.RaftServerManager;
+import org.apache.seata.server.cluster.raft.RaftStateMachine;
+import org.apache.seata.server.session.SessionHolder;
+import org.apache.seata.server.store.StoreConfig;
+import org.apache.seata.server.store.VGroupMappingStoreManager;
+import org.apache.seata.spring.boot.autoconfigure.properties.registry.RegistryNamingServerProperties;
+import org.apache.seata.spring.boot.autoconfigure.properties.server.raft.ServerRaftProperties;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.mockito.junit.jupiter.MockitoExtension;
+import org.springframework.boot.autoconfigure.web.ServerProperties;
+import org.springframework.core.env.ConfigurableEnvironment;
+import org.springframework.core.env.MapPropertySource;
+import org.springframework.core.env.MutablePropertySources;
+import org.springframework.core.env.StandardEnvironment;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicLong;
+
+import static org.apache.seata.common.Constants.OBJECT_KEY_SPRING_CONFIGURABLE_ENVIRONMENT;
+import static org.junit.jupiter.api.Assertions.*;
+import static org.mockito.Mockito.*;

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org


Re: [PR] bugfix: raft split-brain causes incorrect cluster information [incubator-seata]

Posted by "xingfudeshi (via GitHub)" <gi...@apache.org>.
xingfudeshi commented on code in PR #7891:
URL: https://github.com/apache/incubator-seata/pull/7891#discussion_r2639057969


##########
server/src/test/java/org/apache/seata/server/instance/RaftServerInstanceStrategyTest.java:
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.seata.server.instance;
+
+import com.alipay.sofa.jraft.entity.PeerId;
+import org.apache.seata.common.XID;
+import org.apache.seata.common.holder.ObjectHolder;
+import org.apache.seata.common.metadata.ClusterRole;
+import org.apache.seata.common.metadata.Instance;
+import org.apache.seata.common.metadata.Node;
+import org.apache.seata.common.store.SessionMode;
+import org.apache.seata.server.BaseSpringBootTest;
+import org.apache.seata.server.cluster.listener.ClusterChangeEvent;
+import org.apache.seata.server.cluster.raft.RaftServer;
+import org.apache.seata.server.cluster.raft.RaftServerManager;
+import org.apache.seata.server.cluster.raft.RaftStateMachine;
+import org.apache.seata.server.session.SessionHolder;
+import org.apache.seata.server.store.StoreConfig;
+import org.apache.seata.server.store.VGroupMappingStoreManager;
+import org.apache.seata.spring.boot.autoconfigure.properties.registry.RegistryNamingServerProperties;
+import org.apache.seata.spring.boot.autoconfigure.properties.server.raft.ServerRaftProperties;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.mockito.junit.jupiter.MockitoExtension;
+import org.springframework.boot.autoconfigure.web.ServerProperties;
+import org.springframework.core.env.ConfigurableEnvironment;
+import org.springframework.core.env.MapPropertySource;
+import org.springframework.core.env.MutablePropertySources;
+import org.springframework.core.env.StandardEnvironment;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicLong;
+
+import static org.apache.seata.common.Constants.OBJECT_KEY_SPRING_CONFIGURABLE_ENVIRONMENT;
+import static org.junit.jupiter.api.Assertions.*;
+import static org.mockito.Mockito.*;

Review Comment:
   Do not import *



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org


Re: [PR] bugfix: raft split-brain causes incorrect cluster information [incubator-seata]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #7891:
URL: https://github.com/apache/incubator-seata/pull/7891#discussion_r2638877541


##########
common/src/main/java/org/apache/seata/common/metadata/namingserver/NamingServerNode.java:
##########
@@ -78,7 +78,9 @@ public boolean isChanged(Object obj) {
         NamingServerNode otherNode = (NamingServerNode) obj;
 
         // other node is newer than me
-        return otherNode.term > term || !StringUtils.equals(otherNode.getVersion(), this.getVersion());
+        return !Objects.equals(this.getRole(), otherNode.getRole())
+                || otherNode.term > term

Review Comment:
   The role comparison logic may incorrectly treat older nodes as "changed". According to the method's intent (detecting if "other node is newer than me"), a role difference alone should not trigger a change detection if the other node has an older or equal term. The current logic `!Objects.equals(this.getRole(), otherNode.getRole()) || otherNode.term > term` will return true for any role difference, even when otherNode.term < term. Consider changing the logic to: `otherNode.term > term || (otherNode.term >= term && !Objects.equals(this.getRole(), otherNode.getRole())) || !StringUtils.equals(otherNode.getVersion(), this.getVersion())` to ensure role changes are only considered for nodes with equal or higher terms.
   ```suggestion
           return otherNode.term > term
                   || (otherNode.term >= term && !Objects.equals(this.getRole(), otherNode.getRole()))
   ```



##########
namingserver/src/main/java/org/apache/seata/namingserver/manager/NamingManager.java:
##########
@@ -192,15 +192,20 @@ public Result<String> createGroup(
             return new Result<>("400", "vGroup " + vGroup + " already exists");
         }
         // add vGroup in new cluster
-        List<Node> nodeList = getInstances(namespace, clusterName);
+        List<NamingServerNode> nodeList = getInstances(namespace, clusterName);
         if (nodeList == null || nodeList.size() == 0) {
             LOGGER.error("no instance in cluster {}", clusterName);
             return new Result<>("301", "no instance in cluster:" + clusterName);
         } else {
-            Node node = nodeList.stream()
+            List<NamingServerNode> filteredNodes = nodeList.stream()
                     .filter(n -> n.getRole() == ClusterRole.LEADER || n.getRole() == ClusterRole.MEMBER)
-                    .collect(Collectors.toList())
-                    .get(0);
+                    .sorted((o1, o2) -> Long.compare(o2.getTerm(), o1.getTerm()))
+                    .collect(Collectors.toList());
+            if (filteredNodes.isEmpty()) {
+                LOGGER.error("no suitable instance (LEADER or MEMBER) in cluster {}", clusterName);
+                return new Result<>("301", "no suitable instance in cluster:" + clusterName);
+            }
+            NamingServerNode node = filteredNodes.get(0);

Review Comment:
   The logic for selecting a node from the filtered list has changed significantly. Previously, the code used `.get(0)` directly on the stream result, which would throw an IndexOutOfBoundsException if the list was empty. Now there's proper empty list handling with an error check. However, there's a subtle issue: the sorting by term in descending order (line 202) is good for selecting the node with the highest term, but this may not always be the best choice if there are multiple nodes with the same highest term. Consider adding a secondary sorting criterion (e.g., by role preference LEADER over MEMBER) to ensure deterministic selection.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org