You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Alexey Scherbakov (Jira)" <ji...@apache.org> on 2023/12/13 14:59:00 UTC
[jira] [Updated] (IGNITE-21077) Unify transactions for SQL and KV layers
[ https://issues.apache.org/jira/browse/IGNITE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexey Scherbakov updated IGNITE-21077:
---------------------------------------
Description:
Currently KV and SQL engines use different approach to handling transactions, which leads to different execution paths. As a consequence, performance improvements for KV does not apply to SQL.
The problem here is due to distributed nature of SQL execution, in contrary to KV API, which can be coordinated from a single txn coordinator.
The idea is to introduce nested transactions, which works in the following way:
# Query initiator starts parent transaction.
# SQL engine should always use KV api to access data for further SQL execution.
# If SQL query is mapped to multiple execution nodes, each node accesses data using KV API and starts nested transaction having the ID of a parent transaction.
# Nested transaction can'be directly committed or rolled back.
# If at least one nested transaction is created, we need additional PREPARE step. On PREPARE step query initiator sends prepare request to each node containing nested transaction, which is ACKED when a nested txn is safe to commit.
# On receiving all acks parent transaction sends commit request to commit partition.
# PREPARE step introduces additional latency, so it's always OK to map query to a single execution node. For example, SQL transaction containing only unicast operations, can be always executed from a initiator, so PREPARE step is not required.
The same approach can be used to implement things like transactional Compute.
was:
Currently KV and SQL engines use different approach to handling transactions, which leads to different execution paths. As a consequence, performance improvements for KV does not apply to SQL.
The problem here is due to distributed nature of SQL execution, in contrary to KV API, which can be coordinated from a single txn coordinator.
The idea is to introduce nested transactions, which works in the following way:
# Query initiator starts parent transaction.
# SQL engine should always use KV api to access data for further SQL execution.
# If SQL query is mapped to multiple execution nodes, each node accesses data using KV API and starts nested transaction having the ID of a parent transaction.
# Nested transaction can'be directly committed or rolled back.
# If at least one nested transaction is created, we need additional PREPARE step. On PREPARE step query initiator sends prepare request to each node containing nested transaction, which is ACKED when a nested txn is safe to commit.
# On receiving all acks parent transaction sends commit request to commit partition.
# PREPARE step introduces additional latency, so it's always OK to map query to a single execution node. For example, SQL transaction containing only unicast operations, can be always executed from a initiator, so PREPARE step is not required.
> Unify transactions for SQL and KV layers
> ----------------------------------------
>
> Key: IGNITE-21077
> URL: https://issues.apache.org/jira/browse/IGNITE-21077
> Project: Ignite
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Alexey Scherbakov
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0
>
>
> Currently KV and SQL engines use different approach to handling transactions, which leads to different execution paths. As a consequence, performance improvements for KV does not apply to SQL.
> The problem here is due to distributed nature of SQL execution, in contrary to KV API, which can be coordinated from a single txn coordinator.
> The idea is to introduce nested transactions, which works in the following way:
> # Query initiator starts parent transaction.
> # SQL engine should always use KV api to access data for further SQL execution.
> # If SQL query is mapped to multiple execution nodes, each node accesses data using KV API and starts nested transaction having the ID of a parent transaction.
> # Nested transaction can'be directly committed or rolled back.
> # If at least one nested transaction is created, we need additional PREPARE step. On PREPARE step query initiator sends prepare request to each node containing nested transaction, which is ACKED when a nested txn is safe to commit.
> # On receiving all acks parent transaction sends commit request to commit partition.
> # PREPARE step introduces additional latency, so it's always OK to map query to a single execution node. For example, SQL transaction containing only unicast operations, can be always executed from a initiator, so PREPARE step is not required.
> The same approach can be used to implement things like transactional Compute.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)