You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "CTTY (via GitHub)" <gi...@apache.org> on 2026/09/22 21:50:31 UTC
[PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
CTTY opened a new pull request, #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265
Rework the TransactionAction trait per the stateful transaction RFC (milestone 1): each action declares an associated retry-persistent State type created via new_state(), commit() borrows the action immutably and its state mutably per attempt, and a consuming cleanup() plus CommitStatus define terminal-cleanup semantics (not yet invoked; wired in a later milestone).
Transactions now store one entry per action, pairing immutable intent with exclusively-owned execution state behind an object-safe ErasedActionEntry adapter. Cloning a transaction creates a new execution of the action plan: actions are duplicated and paired with fresh state, so retry state is never shared between executions.
All existing actions migrate with State = () and no-op cleanup.
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
## AI Disclosure
<!--
https://iceberg.apache.org/contribute/#guidelines-for-ai-assisted-contributions
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4252328680
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
Review Comment:
API-wise I think forcing `Box::new(entry.action).cleanup(...)` will be kinda weird. I'm thinking if this is really worth it or if there is an even better solution
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4252313470
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
Review Comment:
Makes sense to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4252308871
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
Review Comment:
The same discussion happened when discussing RFC. I think we should just leave it empty and make this explicit, otherwise this will be error-prone
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4126678117
##########
crates/iceberg/src/transaction/append.rs:
##########
@@ -87,7 +88,13 @@ impl FastAppendAction {
#[async_trait]
impl TransactionAction for FastAppendAction {
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit> {
+ // TODO: replace with a persistent `SimpleSnapshotProducer` once snapshot
Review Comment:
lets create a github issue and refer to it in the comments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "JanKaul (via GitHub)" <gi...@apache.org>.
JanKaul commented on PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#issuecomment-5827897423
@Stefan-Dienst , I think the `StatelessAction` trait is a great idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4126934946
##########
crates/iceberg/src/transaction/append.rs:
##########
@@ -87,7 +88,13 @@ impl FastAppendAction {
#[async_trait]
impl TransactionAction for FastAppendAction {
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit> {
+ // TODO: replace with a persistent `SimpleSnapshotProducer` once snapshot
Review Comment:
I'm currently using #3266 to track the entire work, will likely finish the TODO here very soon after the current PR
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
Review Comment:
No, CommitStatus is a terminal status, and pending is not really an end state of an op
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4252368400
##########
crates/iceberg/src/transaction/update_location.rs:
##########
@@ -56,7 +55,11 @@ impl UpdateLocationAction {
#[async_trait]
impl TransactionAction for UpdateLocationAction {
- async fn commit(self: Arc<Self>, _table: &Table) -> Result<ActionCommit> {
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
+
+ async fn commit(&self, _state: &mut (), _table: &Table) -> Result<ActionCommit> {
let updates: Vec<TableUpdate>;
if let Some(location) = self.location.clone() {
Review Comment:
This makes sense
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
Review Comment:
Noted in #3417
##########
crates/iceberg/src/transaction/mod.rs:
##########
@@ -84,10 +84,25 @@ pub use crate::transaction::upgrade_format_version::UpgradeFormatVersionAction;
use crate::{Catalog, TableCommit, TableRequirement, TableUpdate};
/// Table transaction.
-#[derive(Clone)]
pub struct Transaction {
table: Table,
- actions: Vec<BoxedTransactionAction>,
+ actions: Vec<TransactionActionEntry>,
+}
+
+impl Clone for Transaction {
Review Comment:
This is to satisfy the existing `Clone` trait of `Transaction`. I'm ok with dropping the `Clone` directly
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
Review Comment:
Will change this to `#[allow(dead_code)]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4253232346
##########
crates/iceberg/public-api.txt:
##########
@@ -3238,25 +3238,33 @@ pub struct iceberg::transaction::AddColumn
impl iceberg::transaction::AddColumn
pub fn iceberg::transaction::AddColumn::optional(name: impl alloc::string::ToString, field_type: iceberg::spec::Type) -> Self
pub fn iceberg::transaction::AddColumn::required(name: impl alloc::string::ToString, field_type: iceberg::spec::Type, initial_default: iceberg::spec::Literal) -> Self
+impl core::clone::Clone for iceberg::transaction::AddColumn
Review Comment:
I'm thinking if we actually need to have `fresh_clone` or `Clone` on actions. Looks like the original intention was allow third-party retry. But I no longer see the point of keeping them cloneable with retry already implemented now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4153186875
##########
crates/iceberg/public-api.txt:
##########
@@ -3238,25 +3238,33 @@ pub struct iceberg::transaction::AddColumn
impl iceberg::transaction::AddColumn
pub fn iceberg::transaction::AddColumn::optional(name: impl alloc::string::ToString, field_type: iceberg::spec::Type) -> Self
pub fn iceberg::transaction::AddColumn::required(name: impl alloc::string::ToString, field_type: iceberg::spec::Type, initial_default: iceberg::spec::Literal) -> Self
+impl core::clone::Clone for iceberg::transaction::AddColumn
Review Comment:
Deriving `Clone` on all eight actions is needed for `fresh_clone`, but it also lands `Clone` in the public API for each — semver surface we can't easily walk back, and it lets a caller clone a half-built action and `apply` both copies (a `with_check_duplicate(false)` append cloned and applied twice silently doubles data files). I'd at least doc-comment that these `Clone` impls exist for transaction retry infrastructure, not user re-application.
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
Review Comment:
`cleanup` being required means all eight actions carry an identical empty body, and the compiler can't tell a deliberate no-op from a forgotten one. I'd give it a default empty impl on the trait so stateless actions inherit it and only stateful ones override.
That also lines up with the `StatelessAction` direction @Stefan-Dienst floated — a blanket `TransactionAction` impl over a `StatelessAction: Clone + Send + Sync + 'static` that only defines `commit` would collapse the `State`/`new_state`/`cleanup` boilerplate for the eight stateless actions down to a single method. Since the trait is `pub(crate)`, the blanket impl has no public-API cost.
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
Review Comment:
While we're here — the `Unknown` doc says cleanup must delete nothing, but the other half of the guarantee is that retry must be suppressed: if a submitted-but-unconfirmed commit is retried, the original may have landed and the second attempt duplicates it. Worth stating here so whoever wires milestone 3 classifies submitted-but-unverifiable errors as `Unknown` and keeps them out of the retry predicate (`when(|e| e.retryable())` has no such exclusion today).
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
Review Comment:
`ActionEntry<A>` stores `action: Box<A>` but is itself always held behind `Box<dyn ErasedActionEntry>`, so every entry is two allocations and the hot `do_commit` loop derefs through the inner box. The inner box only earns its keep at `cleanup`, which needs `self: Box<Self>` on the action. I'd store `action: A` and box once at that cold cleanup call (`Box::new(entry.action).cleanup(...)`); if the pre-boxing is deliberate, a one-line comment saying why would stop the next reader from "simplifying" it and breaking cleanup forwarding.
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -159,14 +298,30 @@ mod tests {
let tx = Transaction::new(&table);
let updated_tx = action.apply(tx).unwrap();
- // There should be one action in the transaction now
+ // There should be one action entry in the transaction now
assert_eq!(updated_tx.actions.len(), 1);
(*updated_tx.actions[0])
- .downcast_ref::<TestAction>()
+ .downcast_ref::<ActionEntry<TestAction>>()
.expect("TestAction was not applied to Transaction!");
}
+ #[test]
+ fn test_transaction_clone_creates_fresh_entries() {
Review Comment:
This test can't verify the invariant it names: with `TestAction::State = ()`, a fresh-state clone and a state-sharing clone are indistinguishable, so the milestone's central claim — retained state across retries, fresh state across clones — is exercised in neither direction.
I'd add a test action with non-trivial state (say `State = u32` bumped on each `commit`) and assert both halves: `fresh_clone` resets the counter to zero, and driving `do_commit` twice on the same entry accumulates. That's the one thing this milestone most needs to pin down.
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
+ /// One replay attempt against the current transaction-local table.
+ async fn commit(&mut self, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort terminal cleanup, consuming the entry.
+ // TODO: invoke from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, table: &Table, status: CommitStatus);
+
+ /// Creates a new entry for a new logical execution of the same action:
+ /// the action (intent) is cloned, and it is paired with fresh state from
+ /// [`TransactionAction::new_state`]. Accrued retry state is never carried
+ /// over to the new entry.
+ fn fresh_clone(&self) -> TransactionActionEntry;
+}
+
+#[async_trait]
+impl<A: TransactionAction> ErasedActionEntry for ActionEntry<A> {
+ async fn commit(&mut self, table: &Table) -> Result<ActionCommit> {
+ self.action.commit(&mut self.state, table).await
+ }
+
+ async fn cleanup(self: Box<Self>, table: &Table, status: CommitStatus) {
+ let entry = *self;
Review Comment:
This unbox-and-forward path is the most novel code in the PR and nothing exercises it — it's `#[allow(dead_code)]` and no test constructs a `CommitStatus` or calls `cleanup`. I'd add a two-line unit test that boxes an `ActionEntry` and calls `.cleanup(&table, CommitStatus::Committed).await`, so the forwarding chain is validated now rather than discovered when milestone 3 wires it up.
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
+ /// One replay attempt against the current transaction-local table.
+ async fn commit(&mut self, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort terminal cleanup, consuming the entry.
+ // TODO: invoke from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, table: &Table, status: CommitStatus);
+
+ /// Creates a new entry for a new logical execution of the same action:
+ /// the action (intent) is cloned, and it is paired with fresh state from
+ /// [`TransactionAction::new_state`]. Accrued retry state is never carried
+ /// over to the new entry.
+ fn fresh_clone(&self) -> TransactionActionEntry;
Review Comment:
Minor: `fresh_clone` reads like the clone is fresh, but it's the *state* that's fresh — the action intent is cloned as-is. Something like `clone_with_fresh_state` or `new_execution_entry` would say what it does. Not blocking.
##########
crates/iceberg/src/transaction/update_location.rs:
##########
@@ -56,7 +55,11 @@ impl UpdateLocationAction {
#[async_trait]
impl TransactionAction for UpdateLocationAction {
- async fn commit(self: Arc<Self>, _table: &Table) -> Result<ActionCommit> {
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
+
+ async fn commit(&self, _state: &mut (), _table: &Table) -> Result<ActionCommit> {
let updates: Vec<TableUpdate>;
if let Some(location) = self.location.clone() {
Review Comment:
Small thing: `self.location.clone()` clones the whole `Option<String>` just to destructure it. `as_deref` avoids the clone:
```rust
if let Some(location) = self.location.as_deref() {
updates = vec![TableUpdate::SetLocation { location: location.to_owned() }];
}
```
##########
crates/iceberg/src/transaction/append.rs:
##########
@@ -87,7 +88,13 @@ impl FastAppendAction {
#[async_trait]
impl TransactionAction for FastAppendAction {
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit> {
+ // TODO: replace with a persistent `SimpleSnapshotProducer` once snapshot
+ // production is migrated (stateful transaction RFC, milestone 2).
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
+
+ async fn commit(&self, _state: &mut (), table: &Table) -> Result<ActionCommit> {
let snapshot_producer = SnapshotProducer::new(
table,
self.commit_uuid.unwrap_or_else(Uuid::now_v7),
Review Comment:
Not for this PR — `commit_uuid` defaults to a fresh `Uuid::now_v7()` on every `commit` call, so once retries drive `commit` more than once, each attempt writes manifests under a new UUID and the no-op cleanup never reclaims the orphans. It's pre-existing behavior, but this is the right spot to make the milestone-2 TODO explicit: moving the `SnapshotProducer` (and its UUID) into `new_state` is what makes retries reuse the same UUID and already-written manifests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4263733351
##########
crates/iceberg/src/transaction/append.rs:
##########
@@ -87,7 +88,13 @@ impl FastAppendAction {
#[async_trait]
impl TransactionAction for FastAppendAction {
Review Comment:
Because FastAppend will still need a snapshot producer at some point to persist metadata rather than just generating `ActionCommit`. We will need to wire that after converting the existing the SnapshotProducer into a TransactionState
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "JanKaul (via GitHub)" <gi...@apache.org>.
JanKaul commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4081962925
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
Review Comment:
Is the inner `Box<A>` here necessary? Since the entry is already stored as `Box<dyn ErasedActionEntry>`, this looks like a second nested allocation — and as far as I can tell it only exists to satisfy the `self: Box<Self>` receiver on `TransactionAction::cleanup`.
Given that `TransactionAction` is never used as a trait object (erasure happens at the `ErasedActionEntry` layer), could `cleanup` take `self` by value instead of `Box<Self>`? That would let `ActionEntry` hold `action: A` directly and drop the extra allocation:
```
async fn cleanup(self, state: Self::State, table: &Table, status: CommitStatus);
```
Or was the `Box<Self>` receiver a deliberate choice for something later in the RFC?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "blackmwk (via GitHub)" <gi...@apache.org>.
blackmwk commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4217114738
##########
crates/iceberg/src/transaction/mod.rs:
##########
@@ -84,10 +84,25 @@ pub use crate::transaction::upgrade_format_version::UpgradeFormatVersionAction;
use crate::{Catalog, TableCommit, TableRequirement, TableUpdate};
/// Table transaction.
-#[derive(Clone)]
pub struct Transaction {
table: Table,
- actions: Vec<BoxedTransactionAction>,
+ actions: Vec<TransactionActionEntry>,
+}
+
+impl Clone for Transaction {
Review Comment:
Why we need this? If you want to retry a tx, we should change the table.
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
Review Comment:
Create a tracking issue to remove this attribute.
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
Review Comment:
I don't want to add this, I think it's fine to add this function.
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
Review Comment:
I'm not a big fine of this approach, it's odd to make `ErasedActionEntry` a trait. Why not do following:
```
pub(crate) trait DynAction {
...
}
impl<T: Action> DynAction for T {
....
}
```
and store a Box<dyn DynAction> in ActionEntry?
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -103,26 +221,45 @@ impl ActionCommit {
}
}
+/// Runs a single commit attempt of an action with fresh state.
+///
+/// Test-only convenience mirroring how a transaction drives one attempt.
+#[cfg(test)]
+pub(crate) async fn commit_with_fresh_state<A: TransactionAction>(
Review Comment:
As we did in other mod, put the test util methods in test_util mod.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "blackmwk (via GitHub)" <gi...@apache.org>.
blackmwk merged PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4264337566
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,175 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
Review Comment:
nit: `Transaction` isn't `Clone` anymore, so this bit's just wrong now. I'd drop it:
```suggestion
/// attempts of one logical execution but is never shared between executions.
```
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,175 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions implement [`StatelessAction`] instead, which
+ /// supplies `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: ownership makes any further use
+ /// of this action after cleanup a compile error.
+ /// Cleanup must not change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Convenience trait for actions that carry no per-attempt state.
+///
+/// Implementing `StatelessAction` yields a [`TransactionAction`] impl with
+/// `State = ()`, a no-op [`TransactionAction::new_state`], a no-op
+/// [`TransactionAction::cleanup`], and a [`TransactionAction::commit`] that
+/// forwards to [`StatelessAction::commit`]. Actions that need retry-persistent
+/// state (e.g. a persistent snapshot producer) implement `TransactionAction`
+/// directly instead.
+#[async_trait]
+pub(crate) trait StatelessAction: Clone + Send + Sync + 'static {
+ /// One commit attempt against the current transaction-local table.
+ ///
+ /// Same contract as [`TransactionAction::commit`], minus the state parameter.
+ async fn commit(&self, table: &Table) -> Result<ActionCommit>;
+}
+
+#[async_trait]
+impl<T: StatelessAction> TransactionAction for T {
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
+
+ async fn commit(&self, _state: &mut (), table: &Table) -> Result<ActionCommit> {
+ StatelessAction::commit(self, table).await
+ }
+
+ async fn cleanup(self, _state: (), _table: &Table, _status: CommitStatus) {}
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing and this action won't be retryable.
Review Comment:
nit: retry is a transaction-level thing, not an action property.
```suggestion
/// delete nothing, and the transaction must not be retried.
```
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -159,14 +349,69 @@ mod tests {
let tx = Transaction::new(&table);
let updated_tx = action.apply(tx).unwrap();
- // There should be one action in the transaction now
+ // There should be one action entry in the transaction now
assert_eq!(updated_tx.actions.len(), 1);
(*updated_tx.actions[0])
- .downcast_ref::<TestAction>()
+ .downcast_ref::<ActionEntry<TestAction>>()
.expect("TestAction was not applied to Transaction!");
}
+ #[tokio::test]
+ async fn test_entry_retains_state_across_commits() {
+ let table = make_v2_table();
+ let mut tx = CountingAction.apply(Transaction::new(&table)).unwrap();
+
+ let entry = &mut tx.actions[0];
+ entry.commit(&table).await.unwrap();
+ entry.commit(&table).await.unwrap();
+
+ let entry = (**entry)
+ .downcast_ref::<ActionEntry<CountingAction>>()
+ .unwrap();
+ assert_eq!(entry.state, 2);
+ }
+
+ #[tokio::test]
+ async fn test_clone_with_new_state_resets_state() {
+ let table = make_v2_table();
+ let mut tx = CountingAction.apply(Transaction::new(&table)).unwrap();
+ tx.actions[0].commit(&table).await.unwrap();
+
+ let cloned = tx.actions[0].clone_with_new_state();
+
+ let original = (*tx.actions[0])
+ .downcast_ref::<ActionEntry<CountingAction>>()
+ .unwrap();
+ let cloned = (*cloned)
+ .downcast_ref::<ActionEntry<CountingAction>>()
+ .unwrap();
+ assert_eq!(original.state, 1);
+ assert_eq!(cloned.state, 0);
+ }
+
+ #[tokio::test]
+ async fn test_stateless_action_blanket_impl() {
+ let table = make_v2_table();
+
+ // commit goes through the blanket TransactionAction impl
+ let mut commit = commit_with_fresh_state(StatelessTestAction, &table)
+ .await
+ .unwrap();
+ assert_eq!(commit.take_updates(), vec![TableUpdate::SetLocation {
+ location: String::from("s3://bucket/prefix/stateless/"),
+ }]);
+
+ // ApplyTransactionAction is available through the blanket impl too
+ let tx = StatelessTestAction.apply(Transaction::new(&table)).unwrap();
+ assert_eq!(tx.actions.len(), 1);
+
+ // and the erased cleanup forwarding chain compiles and runs
+ let entry: Box<ActionEntry<StatelessTestAction>> =
+ Box::new(ActionEntry::new(StatelessTestAction));
+ entry.cleanup(&table, CommitStatus::Committed).await;
Review Comment:
This shows the unbox runs, not that state/status actually reach `cleanup`, since the body's empty. Fine to leave for milestone 3 as you said. If you want it now, a `CountingAction` that records `(state, status)` into a test-held sink would do it. Not blocking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "blackmwk (via GitHub)" <gi...@apache.org>.
blackmwk commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4256305861
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,175 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions implement [`StatelessAction`] instead, which
+ /// supplies `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: ownership makes any further use
+ /// of this action after cleanup a compile error.
+ /// Cleanup must not change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Convenience trait for actions that carry no per-attempt state.
+///
+/// Implementing `StatelessAction` yields a [`TransactionAction`] impl with
+/// `State = ()`, a no-op [`TransactionAction::new_state`], a no-op
+/// [`TransactionAction::cleanup`], and a [`TransactionAction::commit`] that
+/// forwards to [`StatelessAction::commit`]. Actions that need retry-persistent
+/// state (e.g. a persistent snapshot producer) implement `TransactionAction`
+/// directly instead.
+#[async_trait]
+pub(crate) trait StatelessAction: Clone + Send + Sync + 'static {
+ /// One commit attempt against the current transaction-local table.
+ ///
+ /// Same contract as [`TransactionAction::commit`], minus the state parameter.
+ async fn commit(&self, table: &Table) -> Result<ActionCommit>;
+}
+
+#[async_trait]
+impl<T: StatelessAction> TransactionAction for T {
Review Comment:
I love this.
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,175 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions implement [`StatelessAction`] instead, which
+ /// supplies `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: ownership makes any further use
+ /// of this action after cleanup a compile error.
+ /// Cleanup must not change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Convenience trait for actions that carry no per-attempt state.
+///
+/// Implementing `StatelessAction` yields a [`TransactionAction`] impl with
+/// `State = ()`, a no-op [`TransactionAction::new_state`], a no-op
+/// [`TransactionAction::cleanup`], and a [`TransactionAction::commit`] that
+/// forwards to [`StatelessAction::commit`]. Actions that need retry-persistent
+/// state (e.g. a persistent snapshot producer) implement `TransactionAction`
+/// directly instead.
+#[async_trait]
+pub(crate) trait StatelessAction: Clone + Send + Sync + 'static {
+ /// One commit attempt against the current transaction-local table.
+ ///
+ /// Same contract as [`TransactionAction::commit`], minus the state parameter.
+ async fn commit(&self, table: &Table) -> Result<ActionCommit>;
+}
+
+#[async_trait]
+impl<T: StatelessAction> TransactionAction for T {
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
Review Comment:
```suggestion
fn new_state(&self) -> () {}
```
A little easier to read.
##########
crates/iceberg/src/transaction/append.rs:
##########
@@ -87,7 +88,13 @@ impl FastAppendAction {
#[async_trait]
impl TransactionAction for FastAppendAction {
Review Comment:
WHy not implement StatelessAction?
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,175 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions implement [`StatelessAction`] instead, which
+ /// supplies `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: ownership makes any further use
+ /// of this action after cleanup a compile error.
+ /// Cleanup must not change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Convenience trait for actions that carry no per-attempt state.
+///
+/// Implementing `StatelessAction` yields a [`TransactionAction`] impl with
+/// `State = ()`, a no-op [`TransactionAction::new_state`], a no-op
+/// [`TransactionAction::cleanup`], and a [`TransactionAction::commit`] that
+/// forwards to [`StatelessAction::commit`]. Actions that need retry-persistent
+/// state (e.g. a persistent snapshot producer) implement `TransactionAction`
+/// directly instead.
+#[async_trait]
+pub(crate) trait StatelessAction: Clone + Send + Sync + 'static {
+ /// One commit attempt against the current transaction-local table.
+ ///
+ /// Same contract as [`TransactionAction::commit`], minus the state parameter.
+ async fn commit(&self, table: &Table) -> Result<ActionCommit>;
+}
+
+#[async_trait]
+impl<T: StatelessAction> TransactionAction for T {
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
+
+ async fn commit(&self, _state: &mut (), table: &Table) -> Result<ActionCommit> {
+ StatelessAction::commit(self, table).await
+ }
+
+ async fn cleanup(self, _state: (), _table: &Table, _status: CommitStatus) {}
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing and this action won't be retryable.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: A,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self { action, state }
+ }
+
+ /// The action (intent) held by this entry.
+ #[allow(dead_code)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
Review Comment:
```suggestion
pub(crate) trait DynActionEntry: AsAny + Send + Sync {
```
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
Review Comment:
This approach also sounds reaonsable to me
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,175 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions implement [`StatelessAction`] instead, which
+ /// supplies `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: ownership makes any further use
+ /// of this action after cleanup a compile error.
+ /// Cleanup must not change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Convenience trait for actions that carry no per-attempt state.
+///
+/// Implementing `StatelessAction` yields a [`TransactionAction`] impl with
+/// `State = ()`, a no-op [`TransactionAction::new_state`], a no-op
+/// [`TransactionAction::cleanup`], and a [`TransactionAction::commit`] that
+/// forwards to [`StatelessAction::commit`]. Actions that need retry-persistent
+/// state (e.g. a persistent snapshot producer) implement `TransactionAction`
+/// directly instead.
+#[async_trait]
+pub(crate) trait StatelessAction: Clone + Send + Sync + 'static {
+ /// One commit attempt against the current transaction-local table.
+ ///
+ /// Same contract as [`TransactionAction::commit`], minus the state parameter.
+ async fn commit(&self, table: &Table) -> Result<ActionCommit>;
+}
+
+#[async_trait]
+impl<T: StatelessAction> TransactionAction for T {
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
+
+ async fn commit(&self, _state: &mut (), table: &Table) -> Result<ActionCommit> {
+ StatelessAction::commit(self, table).await
+ }
+
+ async fn cleanup(self, _state: (), _table: &Table, _status: CommitStatus) {}
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing and this action won't be retryable.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: A,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self { action, state }
+ }
+
+ /// The action (intent) held by this entry.
+ #[allow(dead_code)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
Review Comment:
A little more close to rust convention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4126748728
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
Review Comment:
is it expected to see a `CommitStatus::Unknown` if commit op is pending?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "JanKaul (via GitHub)" <gi...@apache.org>.
JanKaul commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4247746571
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
Review Comment:
Are you proposing to also store the `State` in `DynAction` then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4263780248
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,175 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions implement [`StatelessAction`] instead, which
+ /// supplies `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: ownership makes any further use
+ /// of this action after cleanup a compile error.
+ /// Cleanup must not change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Convenience trait for actions that carry no per-attempt state.
+///
+/// Implementing `StatelessAction` yields a [`TransactionAction`] impl with
+/// `State = ()`, a no-op [`TransactionAction::new_state`], a no-op
+/// [`TransactionAction::cleanup`], and a [`TransactionAction::commit`] that
+/// forwards to [`StatelessAction::commit`]. Actions that need retry-persistent
+/// state (e.g. a persistent snapshot producer) implement `TransactionAction`
+/// directly instead.
+#[async_trait]
+pub(crate) trait StatelessAction: Clone + Send + Sync + 'static {
+ /// One commit attempt against the current transaction-local table.
+ ///
+ /// Same contract as [`TransactionAction::commit`], minus the state parameter.
+ async fn commit(&self, table: &Table) -> Result<ActionCommit>;
+}
+
+#[async_trait]
+impl<T: StatelessAction> TransactionAction for T {
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
+
+ async fn commit(&self, _state: &mut (), table: &Table) -> Result<ActionCommit> {
+ StatelessAction::commit(self, table).await
+ }
+
+ async fn cleanup(self, _state: (), _table: &Table, _status: CommitStatus) {}
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing and this action won't be retryable.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: A,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self { action, state }
+ }
+
+ /// The action (intent) held by this entry.
+ #[allow(dead_code)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
+ /// One replay attempt against the current transaction-local table.
+ async fn commit(&mut self, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort terminal cleanup, consuming the entry.
+ // TODO: invoke from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, table: &Table, status: CommitStatus);
+
+ /// Creates a new entry for a new logical execution of the same action:
+ /// the action (intent) is cloned, and it is paired with fresh state from
+ /// [`TransactionAction::new_state`]. Accrued retry state is never carried
+ /// over to the new entry.
+ #[allow(dead_code)]
+ fn clone_with_new_state(&self) -> TransactionActionEntry;
Review Comment:
I'm leaving this here for now but marked in the clean up issue. If this turns out to be useless then we can remove this later: https://github.com/apache/iceberg-rust/issues/3439
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4253378250
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -159,14 +298,30 @@ mod tests {
let tx = Transaction::new(&table);
let updated_tx = action.apply(tx).unwrap();
- // There should be one action in the transaction now
+ // There should be one action entry in the transaction now
assert_eq!(updated_tx.actions.len(), 1);
(*updated_tx.actions[0])
- .downcast_ref::<TestAction>()
+ .downcast_ref::<ActionEntry<TestAction>>()
.expect("TestAction was not applied to Transaction!");
}
+ #[test]
+ fn test_transaction_clone_creates_fresh_entries() {
Review Comment:
Good point! I added `test_entry_retains_state_across_commits` and `test_clone_with_new_state_resets_state` to actually exercise this logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4253432337
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
+ /// One replay attempt against the current transaction-local table.
+ async fn commit(&mut self, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort terminal cleanup, consuming the entry.
+ // TODO: invoke from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, table: &Table, status: CommitStatus);
+
+ /// Creates a new entry for a new logical execution of the same action:
+ /// the action (intent) is cloned, and it is paired with fresh state from
+ /// [`TransactionAction::new_state`]. Accrued retry state is never carried
+ /// over to the new entry.
+ fn fresh_clone(&self) -> TransactionActionEntry;
+}
+
+#[async_trait]
+impl<A: TransactionAction> ErasedActionEntry for ActionEntry<A> {
+ async fn commit(&mut self, table: &Table) -> Result<ActionCommit> {
+ self.action.commit(&mut self.state, table).await
+ }
+
+ async fn cleanup(self: Box<Self>, table: &Table, status: CommitStatus) {
+ let entry = *self;
Review Comment:
the tests will be added later after this is actually wired up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4253272622
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,175 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions implement [`StatelessAction`] instead, which
+ /// supplies `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: ownership makes any further use
+ /// of this action after cleanup a compile error.
+ /// Cleanup must not change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Convenience trait for actions that carry no per-attempt state.
+///
+/// Implementing `StatelessAction` yields a [`TransactionAction`] impl with
+/// `State = ()`, a no-op [`TransactionAction::new_state`], a no-op
+/// [`TransactionAction::cleanup`], and a [`TransactionAction::commit`] that
+/// forwards to [`StatelessAction::commit`]. Actions that need retry-persistent
+/// state (e.g. a persistent snapshot producer) implement `TransactionAction`
+/// directly instead.
+#[async_trait]
+pub(crate) trait StatelessAction: Clone + Send + Sync + 'static {
+ /// One commit attempt against the current transaction-local table.
+ ///
+ /// Same contract as [`TransactionAction::commit`], minus the state parameter.
+ async fn commit(&self, table: &Table) -> Result<ActionCommit>;
+}
+
+#[async_trait]
+impl<T: StatelessAction> TransactionAction for T {
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
+
+ async fn commit(&self, _state: &mut (), table: &Table) -> Result<ActionCommit> {
+ StatelessAction::commit(self, table).await
+ }
+
+ async fn cleanup(self, _state: (), _table: &Table, _status: CommitStatus) {}
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing and this action won't be retryable.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: A,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self { action, state }
+ }
+
+ /// The action (intent) held by this entry.
+ #[allow(dead_code)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
+ /// One replay attempt against the current transaction-local table.
+ async fn commit(&mut self, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort terminal cleanup, consuming the entry.
+ // TODO: invoke from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, table: &Table, status: CommitStatus);
+
+ /// Creates a new entry for a new logical execution of the same action:
+ /// the action (intent) is cloned, and it is paired with fresh state from
+ /// [`TransactionAction::new_state`]. Accrued retry state is never carried
+ /// over to the new entry.
+ #[allow(dead_code)]
+ fn clone_with_new_state(&self) -> TransactionActionEntry;
Review Comment:
I'm thinking if we actually need this. `Transaction` used to be `Clone` because we want to allow thirdparty retry, but now if `Transaction` itself is no longer cloneable, then it has no point to just clone actions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4263682029
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,175 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions implement [`StatelessAction`] instead, which
+ /// supplies `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: ownership makes any further use
+ /// of this action after cleanup a compile error.
+ /// Cleanup must not change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Convenience trait for actions that carry no per-attempt state.
+///
+/// Implementing `StatelessAction` yields a [`TransactionAction`] impl with
+/// `State = ()`, a no-op [`TransactionAction::new_state`], a no-op
+/// [`TransactionAction::cleanup`], and a [`TransactionAction::commit`] that
+/// forwards to [`StatelessAction::commit`]. Actions that need retry-persistent
+/// state (e.g. a persistent snapshot producer) implement `TransactionAction`
+/// directly instead.
+#[async_trait]
+pub(crate) trait StatelessAction: Clone + Send + Sync + 'static {
+ /// One commit attempt against the current transaction-local table.
+ ///
+ /// Same contract as [`TransactionAction::commit`], minus the state parameter.
+ async fn commit(&self, table: &Table) -> Result<ActionCommit>;
+}
+
+#[async_trait]
+impl<T: StatelessAction> TransactionAction for T {
Review Comment:
Credit to @Stefan-Dienst !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "Stefan-Dienst (via GitHub)" <gi...@apache.org>.
Stefan-Dienst commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4097038906
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
Review Comment:
Nit: make wording more explicit, e.g. "Consumes the action and its state: ownership makes any further use of this action after cleanup a compile error."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4252334471
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
+ /// One replay attempt against the current transaction-local table.
+ async fn commit(&mut self, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort terminal cleanup, consuming the entry.
+ // TODO: invoke from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, table: &Table, status: CommitStatus);
+
+ /// Creates a new entry for a new logical execution of the same action:
+ /// the action (intent) is cloned, and it is paired with fresh state from
+ /// [`TransactionAction::new_state`]. Accrued retry state is never carried
+ /// over to the new entry.
+ fn fresh_clone(&self) -> TransactionActionEntry;
Review Comment:
Naming is hard, this makes sense :D I'll think more about the naming
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4252906215
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
+ state: A::State,
+}
+
+impl<A: TransactionAction> ActionEntry<A> {
+ fn new(action: A) -> Self {
+ let state = action.new_state();
+ Self {
+ action: Box::new(action),
+ state,
+ }
+ }
+
+ /// The action (intent) held by this entry.
+ #[cfg(test)]
+ pub(crate) fn action(&self) -> &A {
+ &self.action
+ }
+}
+
+/// Object-safe adapter over [`ActionEntry`], allowing a transaction to store
+/// heterogeneous entries while keeping each action paired with its own state.
+#[async_trait]
+pub(crate) trait ErasedActionEntry: AsAny + Send + Sync {
Review Comment:
I've thought about adding DynAction + DynState but I didn't see the obvious advantage, plus it seems a bit off to "decouple" state from action in the action entry level
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4253176505
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
Review Comment:
I don't think we intended to have double-boxing here. I have replaced `Box<A>` with just `A`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4253381964
##########
crates/iceberg/public-api.txt:
##########
@@ -3238,25 +3238,33 @@ pub struct iceberg::transaction::AddColumn
impl iceberg::transaction::AddColumn
pub fn iceberg::transaction::AddColumn::optional(name: impl alloc::string::ToString, field_type: iceberg::spec::Type) -> Self
pub fn iceberg::transaction::AddColumn::required(name: impl alloc::string::ToString, field_type: iceberg::spec::Type, initial_default: iceberg::spec::Literal) -> Self
+impl core::clone::Clone for iceberg::transaction::AddColumn
Review Comment:
we can discuss here: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4253272622 closing this one
##########
crates/iceberg/public-api.txt:
##########
@@ -3238,25 +3238,33 @@ pub struct iceberg::transaction::AddColumn
impl iceberg::transaction::AddColumn
pub fn iceberg::transaction::AddColumn::optional(name: impl alloc::string::ToString, field_type: iceberg::spec::Type) -> Self
pub fn iceberg::transaction::AddColumn::required(name: impl alloc::string::ToString, field_type: iceberg::spec::Type, initial_default: iceberg::spec::Literal) -> Self
+impl core::clone::Clone for iceberg::transaction::AddColumn
Review Comment:
we can discuss here: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4253272622 resolving this one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4127652120
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
Review Comment:
thanks @CTTY
I was thinking more of TransactionStatus, which can be pending and not yet committed/rolledback. For CommitStatus it makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4252548117
##########
crates/iceberg/src/transaction/append.rs:
##########
@@ -87,7 +88,13 @@ impl FastAppendAction {
#[async_trait]
impl TransactionAction for FastAppendAction {
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit> {
+ // TODO: replace with a persistent `SimpleSnapshotProducer` once snapshot
+ // production is migrated (stateful transaction RFC, milestone 2).
+ type State = ();
+
+ fn new_state(&self) -> Self::State {}
+
+ async fn commit(&self, _state: &mut (), table: &Table) -> Result<ActionCommit> {
let snapshot_producer = SnapshotProducer::new(
table,
self.commit_uuid.unwrap_or_else(Uuid::now_v7),
Review Comment:
Yes, thanks for pointing this out, I was planning to address this later. I've noted this in #3418
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4252768892
##########
crates/iceberg/src/transaction/mod.rs:
##########
@@ -84,10 +84,25 @@ pub use crate::transaction::upgrade_format_version::UpgradeFormatVersionAction;
use crate::{Catalog, TableCommit, TableRequirement, TableUpdate};
/// Table transaction.
-#[derive(Clone)]
pub struct Transaction {
table: Table,
- actions: Vec<BoxedTransactionAction>,
+ actions: Vec<TransactionActionEntry>,
+}
+
+impl Clone for Transaction {
Review Comment:
Looks like we have discussed this in #1474 , `Clone` was added because it allows consumer to retry transactions. Since now we are supporting retry within transactions, I agree that Clone can be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4252950990
##########
crates/iceberg/src/transaction/append.rs:
##########
@@ -87,7 +88,13 @@ impl FastAppendAction {
#[async_trait]
impl TransactionAction for FastAppendAction {
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit> {
+ // TODO: replace with a persistent `SimpleSnapshotProducer` once snapshot
Review Comment:
Created issues for the rest of the milestones linked in https://github.com/apache/iceberg-rust/issues/3266
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Introduce stateful TransactionAction trait [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #3265:
URL: https://github.com/apache/iceberg-rust/pull/3265#discussion_r4253183407
##########
crates/iceberg/src/transaction/action.rs:
##########
@@ -25,28 +24,147 @@ use crate::table::Table;
use crate::transaction::Transaction;
use crate::{Result, TableRequirement, TableUpdate};
-/// A boxed, thread-safe reference to a `TransactionAction`.
-pub(crate) type BoxedTransactionAction = Arc<dyn TransactionAction>;
+/// A boxed entry pairing a transaction action with its retry-persistent state.
+pub(crate) type TransactionActionEntry = Box<dyn ErasedActionEntry>;
/// A trait representing an atomic action that can be part of a transaction.
///
/// Implementors of this trait define how a specific action is committed to a table.
/// Each action is responsible for generating the updates and requirements needed
/// to modify the table metadata.
+///
+/// An action's intent is immutable once applied to a transaction. Retry-persistent
+/// state lives in the associated [`TransactionAction::State`], which survives replay
+/// attempts of one logical execution but is never shared between executions
+/// (cloning a transaction creates fresh state via [`TransactionAction::new_state`]).
#[async_trait]
-pub(crate) trait TransactionAction: AsAny + Sync + Send {
+pub(crate) trait TransactionAction: Clone + Send + Sync + 'static {
+ /// Retry-persistent state exclusively owned by one logical execution of this
+ /// action. Stateless actions use `State = ()`.
+ type State: Send + Sync + 'static;
+
+ /// Creates fresh state for one logical execution.
+ ///
+ /// This is infallible and table-independent; table-dependent initialization
+ /// happens during [`TransactionAction::commit`].
+ fn new_state(&self) -> Self::State;
+
/// Commits this action against the provided table and returns the resulting updates.
/// NOTE: This function is intended for internal use only and should not be called directly by users.
///
+ /// One replay attempt: the action (intent) is borrowed immutably, its execution
+ /// state mutably, and the table reflects the current transaction-local base.
+ ///
/// # Arguments
///
+ /// * `state` - The retry-persistent state for this logical execution.
/// * `table` - The current state of the table this action should apply to.
///
/// # Returns
///
/// An `ActionCommit` containing table updates and table requirements,
/// or an error if the commit fails.
- async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit>;
+ async fn commit(&self, state: &mut Self::State, table: &Table) -> Result<ActionCommit>;
+
+ /// Best-effort cleanup after the transaction reaches a terminal result.
+ ///
+ /// Consumes the action and its state: the type system guarantees that no
+ /// further attempt can run for this entry after cleanup. Cleanup must not
+ /// change the already-determined transaction result.
+ ///
+ /// See [`CommitStatus`] for what each terminal status allows cleanup to do.
+ // TODO: invoke this from `Transaction::commit` once terminal status
+ // classification is wired up (stateful transaction RFC, milestone 3).
+ #[allow(dead_code)]
+ async fn cleanup(self: Box<Self>, state: Self::State, table: &Table, status: CommitStatus);
+}
+
+/// Classification of a transaction's terminal result, consumed by
+/// [`TransactionAction::cleanup`] to determine what is safe to delete.
+///
+/// The transaction/catalog layer determines this classification; actions
+/// consume it rather than independently interpreting catalog errors.
+// TODO: produce this classification in `Transaction::commit` once terminal
+// cleanup is wired up (stateful transaction RFC, milestone 3).
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum CommitStatus {
+ /// The catalog confirmed that the commit was applied. Cleanup may remove
+ /// owned artifacts not retained by the committed result.
+ Committed,
+ /// The transaction definitively did not commit and will not retry.
+ /// Owned artifacts are deletable.
+ Failed,
+ /// The commit request was submitted, but its outcome could not be
+ /// resolved: the catalog may or may not have applied it. Cleanup must
+ /// delete nothing.
+ ///
+ /// Example: every action executed and validated successfully, but the
+ /// connection failed while awaiting the catalog's response to
+ /// `update_table`.
+ Unknown,
+}
+
+/// An entry in a transaction, pairing an action's immutable intent with the
+/// retry-persistent state of one logical execution.
+///
+/// The pairing is preserved by construction: the entry is created with fresh
+/// state and owns both exclusively, so terminal cleanup can consume them together.
+pub(crate) struct ActionEntry<A: TransactionAction> {
+ action: Box<A>,
Review Comment:
I have removed Box in the newest revision
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org