You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by GitBox <gi...@apache.org> on 2022/12/02 07:30:45 UTC
[GitHub] [ignite-3] rpuch opened a new pull request, #1403: IGNITE-18085 Implement logical topology events subscription and notification
rpuch opened a new pull request, #1403:
URL: https://github.com/apache/ignite-3/pull/1403
https://issues.apache.org/jira/browse/IGNITE-18085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] rpuch commented on a diff in pull request #1403: IGNITE-18085 Implement logical topology events subscription and notification
Posted by GitBox <gi...@apache.org>.
rpuch commented on code in PR #1403:
URL: https://github.com/apache/ignite-3/pull/1403#discussion_r1039581425
##########
modules/api/src/main/java/org/apache/ignite/network/LogicalTopologyService.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.network;
+
+import java.util.concurrent.CompletableFuture;
+
+/**
+ * Used for getting information about the cluster's Logical Topology.
+ *
+ * <p>There are 2 kinds of 'topologies': physical (see {@link TopologyService} and logical.
+ * <ul>
+ * <li>Physical topology consists of nodes mutually discovered by a membership protocol (like SWIM)</li>
+ * <li>
+ * Logical topology is a subset of a physical topology and only contains nodes that have successfully
+ * <a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-77%3A+Node+Join+Protocol+and+Initialization+for+Ignite+3">
+ * joined the cluster</a>
+ * </li>
+ * </ul>
+ *
+ * @see <a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-77%3A+Node+Join+Protocol+and+Initialization+for+Ignite+3">
+ * IEP-77: Node Join Protocol and Initialization for Ignite 3</a>
+ * @see TopologyService
+ */
+public interface LogicalTopologyService {
+ /**
+ * Adds a listener for logical topology events.
+ *
+ * <p>Event listeners are not guaranteed to see events they receive being consistent with the state acquired from
+ * {@link #logicalTopologyOnLeader()} (or local logical topology state obtained in other means). This means that,
+ * if you get an event with topology version N, event listener might see version M less or greater than N if it
Review Comment:
This information might be crucial for a user of listeners, so it probably should be here. On the other hand, users of `logicalTopologyOnLeader()` are not required to even know about listeners. If they use listeners, they will read this here and will be informed. So I would prefer to leave it as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] SammyVimes commented on a diff in pull request #1403: IGNITE-18085 Implement logical topology events subscription and notification
Posted by GitBox <gi...@apache.org>.
SammyVimes commented on code in PR #1403:
URL: https://github.com/apache/ignite-3/pull/1403#discussion_r1039500473
##########
modules/api/src/main/java/org/apache/ignite/network/LogicalTopologyService.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.network;
+
+import java.util.concurrent.CompletableFuture;
+
+/**
+ * Used for getting information about the cluster's Logical Topology.
+ *
+ * <p>There are 2 kinds of 'topologies': physical (see {@link TopologyService} and logical.
+ * <ul>
+ * <li>Physical topology consists of nodes mutually discovered by a membership protocol (like SWIM)</li>
+ * <li>
+ * Logical topology is a subset of a physical topology and only contains nodes that have successfully
+ * <a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-77%3A+Node+Join+Protocol+and+Initialization+for+Ignite+3">
+ * joined the cluster</a>
+ * </li>
+ * </ul>
+ *
+ * @see <a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-77%3A+Node+Join+Protocol+and+Initialization+for+Ignite+3">
+ * IEP-77: Node Join Protocol and Initialization for Ignite 3</a>
+ * @see TopologyService
+ */
+public interface LogicalTopologyService {
+ /**
+ * Adds a listener for logical topology events.
+ *
+ * <p>Event listeners are not guaranteed to see events they receive being consistent with the state acquired from
Review Comment:
probably all of this should be in the listener's docs, not in topology service
##########
modules/runner/src/integrationTest/java/org/apache/ignite/internal/AbstractClusterIntegrationTest.java:
##########
@@ -79,57 +85,113 @@ public abstract class AbstractClusterIntegrationTest extends BaseIgniteAbstractT
*/
@BeforeEach
void startNodes(TestInfo testInfo) {
- String connectNodeAddr = "\"localhost:" + BASE_PORT + '\"';
-
- List<CompletableFuture<Ignite>> futures = IntStream.range(0, nodes())
- .mapToObj(i -> {
- String nodeName = testNodeName(testInfo, i);
-
- String config = IgniteStringFormatter.format(NODE_BOOTSTRAP_CFG, BASE_PORT + i, connectNodeAddr);
-
- return IgnitionManager.start(nodeName, config, WORK_DIR.resolve(nodeName));
- })
+ List<CompletableFuture<Ignite>> futures = IntStream.range(0, initialNodes())
+ .mapToObj(i -> startNode0(i, testInfo))
.collect(toList());
- String metaStorageNodeName = testNodeName(testInfo, nodes() - 1);
+ String metaStorageNodeName = testNodeName(testInfo, initialNodes() - 1);
IgnitionManager.init(metaStorageNodeName, List.of(metaStorageNodeName), "cluster");
for (CompletableFuture<Ignite> future : futures) {
- assertThat(future, willCompleteSuccessfully());
+ assertThat(future, willSucceedIn(10, TimeUnit.SECONDS));
clusterNodes.add(future.join());
}
}
+ private static CompletableFuture<Ignite> startNode0(int nodeIndex, TestInfo testInfo) {
+ String connectNodeAddr = "\"localhost:" + BASE_PORT + '\"';
+
+ String nodeName = testNodeName(testInfo, nodeIndex);
+
+ String config = IgniteStringFormatter.format(NODE_BOOTSTRAP_CFG, BASE_PORT + nodeIndex, connectNodeAddr);
+
+ return IgnitionManager.start(nodeName, config, WORK_DIR.resolve(nodeName));
+ }
+
/**
- * Get a count of nodes in the Ignite cluster.
+ * Starts an Ignite node with the given index.
*
- * @return Count of nodes.
+ * @param nodeIndex Zero-based index (used to build node name).
+ * @param testInfo Test info (used to build node name).
+ * @return Started Ignite node.
*/
- protected int nodes() {
+ protected Ignite startNode(int nodeIndex, TestInfo testInfo) {
+ CompletableFuture<Ignite> future = startNode0(nodeIndex, testInfo);
+
+ assertThat(future, willSucceedIn(10, TimeUnit.SECONDS));
+
+ Ignite ignite = future.join();
+
+ if (nodeIndex < clusterNodes.size()) {
+ clusterNodes.set(nodeIndex, ignite);
+ } else if (nodeIndex == clusterNodes.size()) {
+ clusterNodes.add(ignite);
+ } else {
+ throw new IllegalArgumentException("Cannot start node with index " + nodeIndex + " because we only have "
+ + clusterNodes.size() + " nodes");
+ }
+
+ return ignite;
+ }
+
+ /**
+ * Returns count of nodes in the Ignite cluster started before each test.
+ *
+ * @return Count of nodes in initial cluster.
+ */
+ protected int initialNodes() {
Review Comment:
Why not `nodeCount`?
##########
modules/api/src/main/java/org/apache/ignite/network/LogicalTopologyService.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.network;
+
+import java.util.concurrent.CompletableFuture;
+
+/**
+ * Used for getting information about the cluster's Logical Topology.
+ *
+ * <p>There are 2 kinds of 'topologies': physical (see {@link TopologyService} and logical.
+ * <ul>
+ * <li>Physical topology consists of nodes mutually discovered by a membership protocol (like SWIM)</li>
+ * <li>
+ * Logical topology is a subset of a physical topology and only contains nodes that have successfully
+ * <a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-77%3A+Node+Join+Protocol+and+Initialization+for+Ignite+3">
+ * joined the cluster</a>
+ * </li>
+ * </ul>
+ *
+ * @see <a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-77%3A+Node+Join+Protocol+and+Initialization+for+Ignite+3">
+ * IEP-77: Node Join Protocol and Initialization for Ignite 3</a>
+ * @see TopologyService
+ */
+public interface LogicalTopologyService {
+ /**
+ * Adds a listener for logical topology events.
+ *
+ * <p>Event listeners are not guaranteed to see events they receive being consistent with the state acquired from
+ * {@link #logicalTopologyOnLeader()} (or local logical topology state obtained in other means). This means that,
+ * if you get an event with topology version N, event listener might see version M less or greater than N if it
Review Comment:
this, however, I'd put in the `logicalTopologyOnLeader` docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] rpuch commented on a diff in pull request #1403: IGNITE-18085 Implement logical topology events subscription and notification
Posted by GitBox <gi...@apache.org>.
rpuch commented on code in PR #1403:
URL: https://github.com/apache/ignite-3/pull/1403#discussion_r1039647983
##########
modules/api/src/main/java/org/apache/ignite/network/LogicalTopologyService.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.network;
+
+import java.util.concurrent.CompletableFuture;
+
+/**
+ * Used for getting information about the cluster's Logical Topology.
+ *
+ * <p>There are 2 kinds of 'topologies': physical (see {@link TopologyService} and logical.
+ * <ul>
+ * <li>Physical topology consists of nodes mutually discovered by a membership protocol (like SWIM)</li>
+ * <li>
+ * Logical topology is a subset of a physical topology and only contains nodes that have successfully
+ * <a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-77%3A+Node+Join+Protocol+and+Initialization+for+Ignite+3">
+ * joined the cluster</a>
+ * </li>
+ * </ul>
+ *
+ * @see <a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-77%3A+Node+Join+Protocol+and+Initialization+for+Ignite+3">
+ * IEP-77: Node Join Protocol and Initialization for Ignite 3</a>
+ * @see TopologyService
+ */
+public interface LogicalTopologyService {
+ /**
+ * Adds a listener for logical topology events.
+ *
+ * <p>Event listeners are not guaranteed to see events they receive being consistent with the state acquired from
Review Comment:
Moved the docs. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] rpuch commented on a diff in pull request #1403: IGNITE-18085 Implement logical topology events subscription and notification
Posted by GitBox <gi...@apache.org>.
rpuch commented on code in PR #1403:
URL: https://github.com/apache/ignite-3/pull/1403#discussion_r1039582808
##########
modules/runner/src/integrationTest/java/org/apache/ignite/internal/AbstractClusterIntegrationTest.java:
##########
@@ -79,57 +85,113 @@ public abstract class AbstractClusterIntegrationTest extends BaseIgniteAbstractT
*/
@BeforeEach
void startNodes(TestInfo testInfo) {
- String connectNodeAddr = "\"localhost:" + BASE_PORT + '\"';
-
- List<CompletableFuture<Ignite>> futures = IntStream.range(0, nodes())
- .mapToObj(i -> {
- String nodeName = testNodeName(testInfo, i);
-
- String config = IgniteStringFormatter.format(NODE_BOOTSTRAP_CFG, BASE_PORT + i, connectNodeAddr);
-
- return IgnitionManager.start(nodeName, config, WORK_DIR.resolve(nodeName));
- })
+ List<CompletableFuture<Ignite>> futures = IntStream.range(0, initialNodes())
+ .mapToObj(i -> startNode0(i, testInfo))
.collect(toList());
- String metaStorageNodeName = testNodeName(testInfo, nodes() - 1);
+ String metaStorageNodeName = testNodeName(testInfo, initialNodes() - 1);
IgnitionManager.init(metaStorageNodeName, List.of(metaStorageNodeName), "cluster");
for (CompletableFuture<Ignite> future : futures) {
- assertThat(future, willCompleteSuccessfully());
+ assertThat(future, willSucceedIn(10, TimeUnit.SECONDS));
clusterNodes.add(future.join());
}
}
+ private static CompletableFuture<Ignite> startNode0(int nodeIndex, TestInfo testInfo) {
+ String connectNodeAddr = "\"localhost:" + BASE_PORT + '\"';
+
+ String nodeName = testNodeName(testInfo, nodeIndex);
+
+ String config = IgniteStringFormatter.format(NODE_BOOTSTRAP_CFG, BASE_PORT + nodeIndex, connectNodeAddr);
+
+ return IgnitionManager.start(nodeName, config, WORK_DIR.resolve(nodeName));
+ }
+
/**
- * Get a count of nodes in the Ignite cluster.
+ * Starts an Ignite node with the given index.
*
- * @return Count of nodes.
+ * @param nodeIndex Zero-based index (used to build node name).
+ * @param testInfo Test info (used to build node name).
+ * @return Started Ignite node.
*/
- protected int nodes() {
+ protected Ignite startNode(int nodeIndex, TestInfo testInfo) {
+ CompletableFuture<Ignite> future = startNode0(nodeIndex, testInfo);
+
+ assertThat(future, willSucceedIn(10, TimeUnit.SECONDS));
+
+ Ignite ignite = future.join();
+
+ if (nodeIndex < clusterNodes.size()) {
+ clusterNodes.set(nodeIndex, ignite);
+ } else if (nodeIndex == clusterNodes.size()) {
+ clusterNodes.add(ignite);
+ } else {
+ throw new IllegalArgumentException("Cannot start node with index " + nodeIndex + " because we only have "
+ + clusterNodes.size() + " nodes");
+ }
+
+ return ignite;
+ }
+
+ /**
+ * Returns count of nodes in the Ignite cluster started before each test.
+ *
+ * @return Count of nodes in initial cluster.
+ */
+ protected int initialNodes() {
Review Comment:
Node count might change during the lifetime of a cluster if we add a node dynamically. This method returns the *initial* value of nodes count.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] SammyVimes merged pull request #1403: IGNITE-18085 Implement logical topology events subscription and notification
Posted by GitBox <gi...@apache.org>.
SammyVimes merged PR #1403:
URL: https://github.com/apache/ignite-3/pull/1403
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org