You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "korlov42 (via GitHub)" <gi...@apache.org> on 2024/05/21 13:07:54 UTC
[PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
korlov42 opened a new pull request, #3798:
URL: https://github.com/apache/ignite-3/pull/3798
Thank you for submitting the pull request.
To streamline the review process of the patch and ensure better code quality
we ask both an author and a reviewer to verify the following:
### The Review Checklist
- [ ] **Formal criteria:** TC status, codestyle, mandatory documentation. Also make sure to complete the following:
\- There is a single JIRA ticket related to the pull request.
\- The web-link to the pull request is attached to the JIRA ticket.
\- The JIRA ticket has the Patch Available state.
\- The description of the JIRA ticket explains WHAT was made, WHY and HOW.
\- The pull request title is treated as the final commit message. The following pattern must be used: IGNITE-XXXX Change summary where XXXX - number of JIRA issue.
- [ ] **Design:** new code conforms with the design principles of the components it is added to.
- [ ] **Patch quality:** patch cannot be split into smaller pieces, its size must be reasonable.
- [ ] **Code quality:** code is clean and readable, necessary developer documentation is added if needed.
- [ ] **Tests code quality:** test set covers positive/negative scenarios, happy/edge cases. Tests are effective in terms of execution time and resources.
### Notes
- [Apache Ignite Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "korlov42 (via GitHub)" <gi...@apache.org>.
korlov42 commented on code in PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798#discussion_r1611168277
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java:
##########
@@ -122,20 +124,19 @@
import org.apache.ignite.internal.systemview.api.SystemViewManager;
import org.apache.ignite.internal.table.distributed.TableManager;
import org.apache.ignite.internal.table.distributed.schema.SchemaSyncService;
+import org.apache.ignite.internal.tx.HybridTimestampTracker;
import org.apache.ignite.internal.tx.InternalTransaction;
+import org.apache.ignite.internal.tx.TxManager;
import org.apache.ignite.internal.tx.impl.TransactionInflights;
-import org.apache.ignite.internal.util.AsyncCursor;
-import org.apache.ignite.internal.util.AsyncWrapper;
import org.apache.ignite.internal.util.ExceptionUtils;
import org.apache.ignite.internal.util.IgniteSpinBusyLock;
import org.apache.ignite.lang.ErrorGroups.Sql;
import org.apache.ignite.sql.ResultSetMetadata;
import org.apache.ignite.sql.SqlException;
-import org.apache.ignite.tx.IgniteTransactions;
-import org.apache.ignite.tx.TransactionException;
import org.jetbrains.annotations.Nullable;
import org.jetbrains.annotations.TestOnly;
+
Review Comment:
fixed, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "korlov42 (via GitHub)" <gi...@apache.org>.
korlov42 commented on code in PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798#discussion_r1611169873
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/AsyncDataCursor.java:
##########
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.engine.exec;
+
+import java.util.concurrent.CompletableFuture;
+import org.apache.ignite.internal.util.AsyncCursor;
+
+/**
+ * Asynchronous cursor that introduce ability to listen for certain events of cursor's lifecycle.
+ *
+ * @param <T> Type of the items.
+ */
+public interface AsyncDataCursor<T> extends AsyncCursor<T> {
+ /**
+ * Returns {@link CompletableFuture} that will be completed when cursor is closed.
+ *
+ * <p>The future will be completed with {@link null} if cursor was completed successfully,
+ * or with an exception otherwise.
+ *
+ * @return A future representing result of operation.
+ */
+ CompletableFuture<Void> onClose();
+
+ /**
+ * Returns {@link CompletableFuture} that will be completed when first page is fetched and ready
+ * to be returned to user.
+ *
+ * @return A future representing result of operation.
+ */
+ CompletableFuture<Void> onFirstPageReady();
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "korlov42 (via GitHub)" <gi...@apache.org>.
korlov42 commented on code in PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798#discussion_r1611168774
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java:
##########
@@ -500,9 +501,13 @@ public CompletableFuture<AsyncSqlCursor<InternalSqlRow>> queryAsync(
SqlProperties properties0 = SqlPropertiesHelper.chain(properties, DEFAULT_PROPERTIES);
if (Commons.isMultiStatementQueryAllowed(properties0)) {
- return queryScript(properties0, new QueryTransactionContext(transactions, transaction), qry, params);
+ return queryScript(
+ properties0, new QueryTransactionContextImpl(txManager, observableTimeTracker, transaction), qry, params
+ );
} else {
- return querySingle(properties0, new QueryTransactionContext(transactions, transaction), qry, params);
+ return querySingle(
+ properties0, new QueryTransactionContextImpl(txManager, observableTimeTracker, transaction), qry, params
+ );
Review Comment:
good point, fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "korlov42 (via GitHub)" <gi...@apache.org>.
korlov42 commented on code in PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798#discussion_r1610005605
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java:
##########
@@ -564,16 +569,7 @@ private CompletableFuture<AsyncSqlCursor<InternalSqlRow>> querySingle(
validateParsedStatement(properties, result);
validateDynamicParameters(result.dynamicParamsCount(), params, true);
- QueryTransactionWrapper txWrapper = txCtx.getOrStartImplicit(result.queryType());
-
- InternalTransaction tx = txWrapper.unwrap();
-
- // Adding inflights only for read-only transactions.
- if (tx.isReadOnly() && !transactionInflights.addInflight(tx.id(), tx.isReadOnly())) {
Review Comment:
this will be removed in https://issues.apache.org/jira/browse/IGNITE-21868 anyway
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java:
##########
@@ -582,21 +578,11 @@ TX_ALREADY_FINISHED_ERR, format("Transaction is already finished [tx={}]", tx)
.parameters(params)
.timeZoneId(timeZoneId)
.defaultSchemaName(schemaName)
- .operationTime(tx.startTimestamp())
+ .operationTime(operationTime)
+ .txContext(txContext)
.build();
- return executeParsedStatement(operationContext, result, txWrapper, null)
- .handle((executionResult, e) -> {
Review Comment:
this will be removed in https://issues.apache.org/jira/browse/IGNITE-21868 anyway
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798#discussion_r1611082984
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/AsyncDataCursor.java:
##########
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.engine.exec;
+
+import java.util.concurrent.CompletableFuture;
+import org.apache.ignite.internal.util.AsyncCursor;
+
+/**
+ * Asynchronous cursor that introduce ability to listen for certain events of cursor's lifecycle.
+ *
+ * @param <T> Type of the items.
+ */
+public interface AsyncDataCursor<T> extends AsyncCursor<T> {
+ /**
+ * Returns {@link CompletableFuture} that will be completed when cursor is closed.
+ *
+ * <p>The future will be completed with {@link null} if cursor was completed successfully,
+ * or with an exception otherwise.
+ *
+ * @return A future representing result of operation.
+ */
+ CompletableFuture<Void> onClose();
+
+ /**
+ * Returns {@link CompletableFuture} that will be completed when first page is fetched and ready
+ * to be returned to user.
+ *
+ * @return A future representing result of operation.
+ */
+ CompletableFuture<Void> onFirstPageReady();
Review Comment:
Looks like it make sense to make `AsyncSqlCursor` extend `AsyncDataCursor` (instead of `AsyncCursor`) and remove duplicate methods from `AsyncSqlCursor`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798#discussion_r1610154057
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java:
##########
@@ -500,9 +501,13 @@ public CompletableFuture<AsyncSqlCursor<InternalSqlRow>> queryAsync(
SqlProperties properties0 = SqlPropertiesHelper.chain(properties, DEFAULT_PROPERTIES);
if (Commons.isMultiStatementQueryAllowed(properties0)) {
- return queryScript(properties0, new QueryTransactionContext(transactions, transaction), qry, params);
+ return queryScript(
+ properties0, new QueryTransactionContextImpl(txManager, observableTimeTracker, transaction), qry, params
+ );
} else {
- return querySingle(properties0, new QueryTransactionContext(transactions, transaction), qry, params);
+ return querySingle(
+ properties0, new QueryTransactionContextImpl(txManager, observableTimeTracker, transaction), qry, params
+ );
Review Comment:
It seems better to introduce a variable to create the tx context (otherwise it's harder to understand that there is no difference in the arguments).
Something like this
```
SqlProperties properties0 = SqlPropertiesHelper.chain(properties, DEFAULT_PROPERTIES);
QueryTransactionContext txCtx = new QueryTransactionContextImpl(txManager, observableTimeTracker, transaction);
if (Commons.isMultiStatementQueryAllowed(properties0)) {
return queryScript(properties0, txCtx, qry, params);
} else {
return querySingle(properties0, txCtx, qry, params);
}
```
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java:
##########
@@ -367,14 +393,36 @@ private AsyncCursor<InternalSqlRow> executeExecutablePlan(
DUMMY_DESCRIPTION,
handler,
Commons.parametersMap(operationContext.parameters()),
- TxAttributes.fromTx(tx),
+ TxAttributes.dummy(),
Review Comment:
why not just pass `null`?
or add constructor without tx :thinking:
to me this looks clearer and less error prone
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java:
##########
@@ -122,20 +124,19 @@
import org.apache.ignite.internal.systemview.api.SystemViewManager;
import org.apache.ignite.internal.table.distributed.TableManager;
import org.apache.ignite.internal.table.distributed.schema.SchemaSyncService;
+import org.apache.ignite.internal.tx.HybridTimestampTracker;
import org.apache.ignite.internal.tx.InternalTransaction;
+import org.apache.ignite.internal.tx.TxManager;
import org.apache.ignite.internal.tx.impl.TransactionInflights;
-import org.apache.ignite.internal.util.AsyncCursor;
-import org.apache.ignite.internal.util.AsyncWrapper;
import org.apache.ignite.internal.util.ExceptionUtils;
import org.apache.ignite.internal.util.IgniteSpinBusyLock;
import org.apache.ignite.lang.ErrorGroups.Sql;
import org.apache.ignite.sql.ResultSetMetadata;
import org.apache.ignite.sql.SqlException;
-import org.apache.ignite.tx.IgniteTransactions;
-import org.apache.ignite.tx.TransactionException;
import org.jetbrains.annotations.Nullable;
import org.jetbrains.annotations.TestOnly;
+
Review Comment:
```suggestion
```
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/AsyncDataCursor.java:
##########
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.engine.exec;
+
+import java.util.concurrent.CompletableFuture;
+import org.apache.ignite.internal.util.AsyncCursor;
+
+/**
+ * Asynchronous cursor that introduce ability to listen for certain events of cursor's lifecycle.
+ *
+ * @param <T> Type of the items.
+ */
+public interface AsyncDataCursor<T> extends AsyncCursor<T> {
+ /**
+ * Returns {@link CompletableFuture} that will be completed when cursor is closed.
+ *
+ * <p>The future will be completed with {@link null} if cursor was completed successfully,
+ * or with an exception otherwise.
+ *
+ * @return A future representing result of operation.
+ */
+ CompletableFuture<Void> onClose();
+
+ /**
+ * Returns {@link CompletableFuture} that will be completed when first page is fetched and ready
+ * to be returned to user.
+ *
+ * @return A future representing result of operation.
+ */
+ CompletableFuture<Void> onFirstPageReady();
Review Comment:
Looks like it make sense to make `AsyncSqlCursor` extend `AsyncDataCursor` (instead of `AsyncCursor`) and remove duplicate methods from `AsyncSqlCursor`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "korlov42 (via GitHub)" <gi...@apache.org>.
korlov42 commented on code in PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798#discussion_r1611179102
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java:
##########
@@ -367,14 +393,36 @@ private AsyncCursor<InternalSqlRow> executeExecutablePlan(
DUMMY_DESCRIPTION,
handler,
Commons.parametersMap(operationContext.parameters()),
- TxAttributes.fromTx(tx),
+ TxAttributes.dummy(),
Review Comment:
> why not just pass null?
well, we can pass null and just suppress the warning, if you insist. But I would prefer to rework this part so this won't be a problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "korlov42 (via GitHub)" <gi...@apache.org>.
korlov42 commented on code in PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798#discussion_r1611164182
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java:
##########
@@ -367,14 +393,36 @@ private AsyncCursor<InternalSqlRow> executeExecutablePlan(
DUMMY_DESCRIPTION,
handler,
Commons.parametersMap(operationContext.parameters()),
- TxAttributes.fromTx(tx),
+ TxAttributes.dummy(),
Review Comment:
Because KV operation don't really need execution context. They need only dynamic parameters to build key/value tuple properly, while execution context is part of multistep query execution. I think, it's better to create fake object for this particular case rather than handle potential NPE (or guard it every time with `assert attribute != null` like I did with numerous attributes of operationContext within this patch) everywhere within multistep execution path. Moreover there is already fake fragment description, so one more fake object won't make it worse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "ygerzhedovich (via GitHub)" <gi...@apache.org>.
ygerzhedovich merged PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-22263 Sql. Avoid starting transaction for KV operation [ignite-3]
Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #3798:
URL: https://github.com/apache/ignite-3/pull/3798#discussion_r1611644937
##########
modules/jdbc/src/integrationTest/java/org/apache/ignite/jdbc/ItJdbcStatementSelfTest.java:
##########
@@ -69,6 +69,10 @@ public void beforeEach() throws Exception {
@AfterEach
public void afterEach() throws Exception {
+ if (stmt != null) {
Review Comment:
already implemented in AbstractJdbcSelfTest
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java:
##########
@@ -246,7 +245,8 @@ public SqlQueryProcessor(
PlacementDriver placementDriver,
SqlDistributedConfiguration clusterCfg,
SqlLocalConfiguration nodeCfg,
- TransactionInflights transactionInflights
+ TransactionInflights transactionInflights,
Review Comment:
remove it plz
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlOperationContext.java:
##########
@@ -124,21 +137,17 @@ public HybridTimestamp operationTime() {
*/
@SuppressWarnings("PublicInnerClass")
public static class Builder {
- private QueryCancel cancel = new QueryCancel();
-
private UUID queryId;
-
- private Object[] parameters = ArrayUtils.OBJECT_EMPTY_ARRAY;
-
private ZoneId timeZoneId;
-
- private @Nullable PrefetchCallback prefetchCallback;
-
+ private Object[] parameters = ArrayUtils.OBJECT_EMPTY_ARRAY;
private HybridTimestamp operationTime;
+ private QueryTransactionContext txContext;
- private String defaultSchemaName;
+ private @Nullable QueryCancel cancel;
+ private @Nullable String defaultSchemaName;
+ private @Nullable PrefetchCallback prefetchCallback;
- public Builder cancel(QueryCancel cancel) {
+ public Builder cancel(@Nullable QueryCancel cancel) {
Review Comment:
why you append @Nullable here ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org