You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "korlov42 (via GitHub)" <gi...@apache.org> on 2024/11/11 08:28:44 UTC
[PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
korlov42 opened a new pull request, #4697:
URL: https://github.com/apache/ignite-3/pull/4697
https://issues.apache.org/jira/browse/IGNITE-22768
This patch introduces recovery for the case when node on which fragment is mapped left the cluster before the moment of sending the fragment.
To address the issue, the following has been done:
- introduced `nodeExclusionFilter` to `MappingService`: nodes which meet the predicate must be excluded from the mapping. If one or more mandatory nodes were excluded, then generic `MappingException` will be thrown
- changed `ExecutionService` interface: now `execute` method returns future that will be completed only after distributed query is completely initialized
- `MessageService` now completes future returned from `send` with `UnknownNodeException` if recipient cannot be resolved (left the cluster or never exists)
- introduced `Program`'s to query FSM, which includes error handle to recover from particular situations. See javadoc for details
-----------
Thank you for submitting the pull request.
To streamline the review process of the patch and ensure better code quality
we ask both an author and a reviewer to verify the following:
### The Review Checklist
- [ ] **Formal criteria:** TC status, codestyle, mandatory documentation. Also make sure to complete the following:
\- There is a single JIRA ticket related to the pull request.
\- The web-link to the pull request is attached to the JIRA ticket.
\- The JIRA ticket has the Patch Available state.
\- The description of the JIRA ticket explains WHAT was made, WHY and HOW.
\- The pull request title is treated as the final commit message. The following pattern must be used: IGNITE-XXXX Change summary where XXXX - number of JIRA issue.
- [ ] **Design:** new code conforms with the design principles of the components it is added to.
- [ ] **Patch quality:** patch cannot be split into smaller pieces, its size must be reasonable.
- [ ] **Code quality:** code is clean and readable, necessary developer documentation is added if needed.
- [ ] **Tests code quality:** test set covers positive/negative scenarios, happy/edge cases. Tests are effective in terms of execution time and resources.
### Notes
- [Apache Ignite Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1838112199
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/fsm/QueryExecutionProgram.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.engine.exec.fsm;
+
+import java.util.List;
+import org.apache.ignite.internal.sql.engine.AsyncSqlCursor;
+import org.apache.ignite.internal.sql.engine.InternalSqlRow;
+import org.apache.ignite.internal.sql.engine.SqlOperationContext;
+import org.apache.ignite.internal.sql.engine.message.UnknownNodeException;
+
+/**
+ * Generic execution program, which accepts query string and initialization parameters as input, and returns cursor as result.
+ */
+class QueryExecutionProgram extends Program<AsyncSqlCursor<InternalSqlRow>> {
+ private static final String PROGRAM_NAME = "QUERY_EXECUTION";
+ private static final List<Transition> TRANSITIONS = List.of(
+ new Transition(
+ ExecutionPhase.REGISTERED,
+ query -> ExecutionPhase.PARSING
+ ),
+ new Transition(
+ ExecutionPhase.PARSING,
+ query -> query.parsedResult != null
+ ? ExecutionPhase.OPTIMIZING
+ : ExecutionPhase.SCRIPT_INITIALIZATION),
+ new Transition(
+ ExecutionPhase.OPTIMIZING,
+ query -> ExecutionPhase.CURSOR_INITIALIZATION
+ ),
+ new Transition(
+ ExecutionPhase.CURSOR_INITIALIZATION,
+ query -> ExecutionPhase.EXECUTING
+ ),
+ new Transition(
+ ExecutionPhase.SCRIPT_INITIALIZATION,
+ query -> ExecutionPhase.EXECUTING
+ )
+ );
+
+ static final Program<AsyncSqlCursor<InternalSqlRow>> INSTANCE = new QueryExecutionProgram();
+
+ private QueryExecutionProgram() {
+ super(
+ PROGRAM_NAME,
+ TRANSITIONS,
+ phase -> phase == ExecutionPhase.EXECUTING,
+ query -> query.cursor,
+ QueryExecutionProgram::errorHandler
+ );
+ }
+
+ static boolean errorHandler(Query query, Throwable th) {
Review Comment:
or you mean that in future this methods will be different ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1838094639
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/fsm/QueryExecutionProgram.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.engine.exec.fsm;
+
+import java.util.List;
+import org.apache.ignite.internal.sql.engine.AsyncSqlCursor;
+import org.apache.ignite.internal.sql.engine.InternalSqlRow;
+import org.apache.ignite.internal.sql.engine.SqlOperationContext;
+import org.apache.ignite.internal.sql.engine.message.UnknownNodeException;
+
+/**
+ * Generic execution program, which accepts query string and initialization parameters as input, and returns cursor as result.
+ */
+class QueryExecutionProgram extends Program<AsyncSqlCursor<InternalSqlRow>> {
+ private static final String PROGRAM_NAME = "QUERY_EXECUTION";
+ private static final List<Transition> TRANSITIONS = List.of(
+ new Transition(
+ ExecutionPhase.REGISTERED,
+ query -> ExecutionPhase.PARSING
+ ),
+ new Transition(
+ ExecutionPhase.PARSING,
+ query -> query.parsedResult != null
+ ? ExecutionPhase.OPTIMIZING
+ : ExecutionPhase.SCRIPT_INITIALIZATION),
+ new Transition(
+ ExecutionPhase.OPTIMIZING,
+ query -> ExecutionPhase.CURSOR_INITIALIZATION
+ ),
+ new Transition(
+ ExecutionPhase.CURSOR_INITIALIZATION,
+ query -> ExecutionPhase.EXECUTING
+ ),
+ new Transition(
+ ExecutionPhase.SCRIPT_INITIALIZATION,
+ query -> ExecutionPhase.EXECUTING
+ )
+ );
+
+ static final Program<AsyncSqlCursor<InternalSqlRow>> INSTANCE = new QueryExecutionProgram();
+
+ private QueryExecutionProgram() {
+ super(
+ PROGRAM_NAME,
+ TRANSITIONS,
+ phase -> phase == ExecutionPhase.EXECUTING,
+ query -> query.cursor,
+ QueryExecutionProgram::errorHandler
+ );
+ }
+
+ static boolean errorHandler(Query query, Throwable th) {
Review Comment:
i think it become more clear if "errorHandler" and "canRecover" will be moved into Program ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
Posted by "korlov42 (via GitHub)" <gi...@apache.org>.
korlov42 commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1838148653
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/fsm/QueryExecutionProgram.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.engine.exec.fsm;
+
+import java.util.List;
+import org.apache.ignite.internal.sql.engine.AsyncSqlCursor;
+import org.apache.ignite.internal.sql.engine.InternalSqlRow;
+import org.apache.ignite.internal.sql.engine.SqlOperationContext;
+import org.apache.ignite.internal.sql.engine.message.UnknownNodeException;
+
+/**
+ * Generic execution program, which accepts query string and initialization parameters as input, and returns cursor as result.
+ */
+class QueryExecutionProgram extends Program<AsyncSqlCursor<InternalSqlRow>> {
+ private static final String PROGRAM_NAME = "QUERY_EXECUTION";
+ private static final List<Transition> TRANSITIONS = List.of(
+ new Transition(
+ ExecutionPhase.REGISTERED,
+ query -> ExecutionPhase.PARSING
+ ),
+ new Transition(
+ ExecutionPhase.PARSING,
+ query -> query.parsedResult != null
+ ? ExecutionPhase.OPTIMIZING
+ : ExecutionPhase.SCRIPT_INITIALIZATION),
+ new Transition(
+ ExecutionPhase.OPTIMIZING,
+ query -> ExecutionPhase.CURSOR_INITIALIZATION
+ ),
+ new Transition(
+ ExecutionPhase.CURSOR_INITIALIZATION,
+ query -> ExecutionPhase.EXECUTING
+ ),
+ new Transition(
+ ExecutionPhase.SCRIPT_INITIALIZATION,
+ query -> ExecutionPhase.EXECUTING
+ )
+ );
+
+ static final Program<AsyncSqlCursor<InternalSqlRow>> INSTANCE = new QueryExecutionProgram();
+
+ private QueryExecutionProgram() {
+ super(
+ PROGRAM_NAME,
+ TRANSITIONS,
+ phase -> phase == ExecutionPhase.EXECUTING,
+ query -> query.cursor,
+ QueryExecutionProgram::errorHandler
+ );
+ }
+
+ static boolean errorHandler(Query query, Throwable th) {
Review Comment:
Every program is supposed to have its own recovery rules.
For instance, to recover on CURSOR_INITIALISATION phase you need to make sure that data cursor is closed which appeared only on this very phase. Another phase may require different action to clean up state.
The same with exceptions which can be recovered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
Posted by "AMashenkov (via GitHub)" <gi...@apache.org>.
AMashenkov commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1841963156
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/mapping/MappingServiceImpl.java:
##########
@@ -220,7 +231,9 @@ private CompletableFuture<MappedFragments> mapFragments(
return res.thenApply(assignments -> {
Int2ObjectMap<ExecutionTarget> targetsById = new Int2ObjectOpenHashMap<>();
- MappingContext context = new MappingContext(localNodeName, new ArrayList<>(assignments.nodes()), template.cluster);
+ MappingContext context = new MappingContext(
+ localNodeName, new ArrayList<>(assignments.nodes(nodeExclusionFilter)), template.cluster
Review Comment:
Do we need a copy to ArrayList here if filter is null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
Posted by "AMashenkov (via GitHub)" <gi...@apache.org>.
AMashenkov commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1841979730
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/mapping/MappingServiceImpl.java:
##########
@@ -220,7 +231,9 @@ private CompletableFuture<MappedFragments> mapFragments(
return res.thenApply(assignments -> {
Int2ObjectMap<ExecutionTarget> targetsById = new Int2ObjectOpenHashMap<>();
- MappingContext context = new MappingContext(localNodeName, new ArrayList<>(assignments.nodes()), template.cluster);
+ MappingContext context = new MappingContext(
+ localNodeName, new ArrayList<>(assignments.nodes(nodeExclusionFilter)), template.cluster
Review Comment:
If we can't avoid copying, then the filter can be not null (@Nullable not needed) and null value replaced with "always false" predicate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
Posted by "korlov42 (via GitHub)" <gi...@apache.org>.
korlov42 commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1842057692
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/mapping/MappingServiceImpl.java:
##########
@@ -220,7 +231,9 @@ private CompletableFuture<MappedFragments> mapFragments(
return res.thenApply(assignments -> {
Int2ObjectMap<ExecutionTarget> targetsById = new Int2ObjectOpenHashMap<>();
- MappingContext context = new MappingContext(localNodeName, new ArrayList<>(assignments.nodes()), template.cluster);
+ MappingContext context = new MappingContext(
+ localNodeName, new ArrayList<>(assignments.nodes(nodeExclusionFilter)), template.cluster
Review Comment:
Change {{DistributionHolder}} to return list instead of set. Now the collection is copied only once when filter is null, otherwise we create filtered list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
Posted by "korlov42 (via GitHub)" <gi...@apache.org>.
korlov42 merged PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org