You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "CTTY (via GitHub)" <gi...@apache.org> on 2025/06/14 00:29:54 UTC
[PR] feat(transaction): Implement TransactionAction for ReplaceSortOrderAction [iceberg-rust]
CTTY opened a new pull request, #1441:
URL: https://github.com/apache/iceberg-rust/pull/1441
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Implement TransactionAction for ReplaceSortOrderAction [iceberg-rust]
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on code in PR #1441:
URL: https://github.com/apache/iceberg-rust/pull/1441#discussion_r2147042449
##########
crates/iceberg/src/transaction/sort_order.rs:
##########
@@ -15,32 +15,97 @@
// specific language governing permissions and limitations
// under the License.
+use std::sync::Arc;
+
+use async_trait::async_trait;
+
use crate::error::Result;
use crate::spec::{NullOrder, SortDirection, SortField, SortOrder, Transform};
-use crate::transaction::Transaction;
+use crate::table::Table;
+use crate::transaction::{ActionCommit, TransactionAction};
use crate::{Error, ErrorKind, TableRequirement, TableUpdate};
+#[derive(Debug, PartialEq, Eq, Clone)]
+struct PendingSortField {
+ name: String,
+ direction: SortDirection,
+ null_order: NullOrder,
+}
+
Review Comment:
Maybe add this as comment for PendingSortField
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Implement TransactionAction for ReplaceSortOrderAction [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #1441:
URL: https://github.com/apache/iceberg-rust/pull/1441#discussion_r2146439072
##########
crates/iceberg/src/transaction/sort_order.rs:
##########
@@ -15,32 +15,97 @@
// specific language governing permissions and limitations
// under the License.
+use std::sync::Arc;
+
+use async_trait::async_trait;
+
use crate::error::Result;
use crate::spec::{NullOrder, SortDirection, SortField, SortOrder, Transform};
-use crate::transaction::Transaction;
+use crate::table::Table;
+use crate::transaction::{ActionCommit, TransactionAction};
use crate::{Error, ErrorKind, TableRequirement, TableUpdate};
+#[derive(Debug, PartialEq, Eq, Clone)]
+struct PendingSortField {
+ name: String,
+ direction: SortDirection,
+ null_order: NullOrder,
+}
+
Review Comment:
`PendingSortField` is added to defer the construction of `Vec<SortField>` from `asc`/`desc` to `commit` to avoid passing `Table` to methods like `asc`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Implement TransactionAction for ReplaceSortOrderAction [iceberg-rust]
Posted by "liurenjie1024 (via GitHub)" <gi...@apache.org>.
liurenjie1024 commented on code in PR #1441:
URL: https://github.com/apache/iceberg-rust/pull/1441#discussion_r2149562680
##########
crates/iceberg/src/transaction/sort_order.rs:
##########
@@ -15,32 +15,97 @@
// specific language governing permissions and limitations
// under the License.
+use std::sync::Arc;
+
+use async_trait::async_trait;
+
use crate::error::Result;
use crate::spec::{NullOrder, SortDirection, SortField, SortOrder, Transform};
-use crate::transaction::Transaction;
+use crate::table::Table;
+use crate::transaction::{ActionCommit, TransactionAction};
use crate::{Error, ErrorKind, TableRequirement, TableUpdate};
+#[derive(Debug, PartialEq, Eq, Clone)]
+struct PendingSortField {
+ name: String,
+ direction: SortDirection,
+ null_order: NullOrder,
+}
+
/// Transaction action for replacing sort order.
pub struct ReplaceSortOrderAction {
- pub tx: Transaction,
- pub sort_fields: Vec<SortField>,
+ pending_sort_fields: Vec<PendingSortField>,
}
impl ReplaceSortOrderAction {
+ pub fn new() -> Self {
+ ReplaceSortOrderAction {
+ pending_sort_fields: vec![],
+ }
+ }
+
/// Adds a field for sorting in ascending order.
- pub fn asc(self, name: &str, null_order: NullOrder) -> Result<Self> {
+ pub fn asc(self, name: &str, null_order: NullOrder) -> Self {
self.add_sort_field(name, SortDirection::Ascending, null_order)
}
/// Adds a field for sorting in descending order.
- pub fn desc(self, name: &str, null_order: NullOrder) -> Result<Self> {
+ pub fn desc(self, name: &str, null_order: NullOrder) -> Self {
self.add_sort_field(name, SortDirection::Descending, null_order)
}
- /// Finished building the action and apply it to the transaction.
- pub fn apply(mut self) -> Result<Transaction> {
+ fn add_sort_field(
+ mut self,
+ name: &str,
+ sort_direction: SortDirection,
+ null_order: NullOrder,
+ ) -> Self {
+ self.pending_sort_fields.push(PendingSortField {
+ name: name.to_string(),
+ direction: sort_direction,
+ null_order,
+ });
+
+ self
+ }
+}
+
+impl Default for ReplaceSortOrderAction {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+#[async_trait]
+impl TransactionAction for ReplaceSortOrderAction {
+ async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit> {
+ let pending_sort_fields = self.pending_sort_fields.clone();
Review Comment:
We don't need to clone whole vec here?
##########
crates/iceberg/src/transaction/sort_order.rs:
##########
@@ -15,32 +15,97 @@
// specific language governing permissions and limitations
// under the License.
+use std::sync::Arc;
+
+use async_trait::async_trait;
+
use crate::error::Result;
use crate::spec::{NullOrder, SortDirection, SortField, SortOrder, Transform};
-use crate::transaction::Transaction;
+use crate::table::Table;
+use crate::transaction::{ActionCommit, TransactionAction};
use crate::{Error, ErrorKind, TableRequirement, TableUpdate};
+#[derive(Debug, PartialEq, Eq, Clone)]
+struct PendingSortField {
+ name: String,
+ direction: SortDirection,
+ null_order: NullOrder,
+}
+
/// Transaction action for replacing sort order.
pub struct ReplaceSortOrderAction {
- pub tx: Transaction,
- pub sort_fields: Vec<SortField>,
+ pending_sort_fields: Vec<PendingSortField>,
}
impl ReplaceSortOrderAction {
+ pub fn new() -> Self {
+ ReplaceSortOrderAction {
+ pending_sort_fields: vec![],
+ }
+ }
+
/// Adds a field for sorting in ascending order.
- pub fn asc(self, name: &str, null_order: NullOrder) -> Result<Self> {
+ pub fn asc(self, name: &str, null_order: NullOrder) -> Self {
self.add_sort_field(name, SortDirection::Ascending, null_order)
}
/// Adds a field for sorting in descending order.
- pub fn desc(self, name: &str, null_order: NullOrder) -> Result<Self> {
+ pub fn desc(self, name: &str, null_order: NullOrder) -> Self {
self.add_sort_field(name, SortDirection::Descending, null_order)
}
- /// Finished building the action and apply it to the transaction.
- pub fn apply(mut self) -> Result<Transaction> {
+ fn add_sort_field(
+ mut self,
+ name: &str,
+ sort_direction: SortDirection,
+ null_order: NullOrder,
+ ) -> Self {
+ self.pending_sort_fields.push(PendingSortField {
+ name: name.to_string(),
+ direction: sort_direction,
+ null_order,
+ });
+
+ self
+ }
+}
+
+impl Default for ReplaceSortOrderAction {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+#[async_trait]
+impl TransactionAction for ReplaceSortOrderAction {
+ async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit> {
+ let pending_sort_fields = self.pending_sort_fields.clone();
+
+ let sort_fields: Result<Vec<SortField>> = pending_sort_fields
+ .iter()
+ .map(|p| {
+ let field_id = table
+ .metadata()
+ .current_schema()
+ .field_id_by_name(p.name.as_str())
+ .ok_or_else(|| {
+ Error::new(
+ ErrorKind::DataInvalid,
+ format!("Cannot find field {} in table schema", p.name),
+ )
+ })?;
+
+ Ok(SortField::builder()
+ .source_id(field_id)
+ .transform(Transform::Identity)
+ .direction(p.direction)
+ .null_order(p.null_order)
+ .build())
+ })
+ .collect();
+
let unbound_sort_order = SortOrder::builder()
- .with_fields(self.sort_fields)
+ .with_fields(sort_fields?)
.build_unbound()?;
Review Comment:
We should use `build_bound` to resue the verification logic in `SortOrder` builder.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Implement TransactionAction for ReplaceSortOrderAction [iceberg-rust]
Posted by "liurenjie1024 (via GitHub)" <gi...@apache.org>.
liurenjie1024 merged PR #1441:
URL: https://github.com/apache/iceberg-rust/pull/1441
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Implement TransactionAction for ReplaceSortOrderAction [iceberg-rust]
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on code in PR #1441:
URL: https://github.com/apache/iceberg-rust/pull/1441#discussion_r2147038811
##########
crates/iceberg/src/transaction/sort_order.rs:
##########
@@ -15,32 +15,97 @@
// specific language governing permissions and limitations
// under the License.
+use std::sync::Arc;
+
+use async_trait::async_trait;
+
use crate::error::Result;
use crate::spec::{NullOrder, SortDirection, SortField, SortOrder, Transform};
-use crate::transaction::Transaction;
+use crate::table::Table;
+use crate::transaction::{ActionCommit, TransactionAction};
use crate::{Error, ErrorKind, TableRequirement, TableUpdate};
+#[derive(Debug, PartialEq, Eq, Clone)]
+struct PendingSortField {
+ name: String,
+ direction: SortDirection,
+ null_order: NullOrder,
+}
+
/// Transaction action for replacing sort order.
pub struct ReplaceSortOrderAction {
- pub tx: Transaction,
- pub sort_fields: Vec<SortField>,
+ pending_sort_fields: Vec<PendingSortField>,
}
impl ReplaceSortOrderAction {
+ pub fn new() -> Self {
+ ReplaceSortOrderAction {
+ pending_sort_fields: vec![],
+ }
+ }
+
/// Adds a field for sorting in ascending order.
- pub fn asc(self, name: &str, null_order: NullOrder) -> Result<Self> {
+ pub fn asc(self, name: &str, null_order: NullOrder) -> Self {
self.add_sort_field(name, SortDirection::Ascending, null_order)
}
/// Adds a field for sorting in descending order.
- pub fn desc(self, name: &str, null_order: NullOrder) -> Result<Self> {
+ pub fn desc(self, name: &str, null_order: NullOrder) -> Self {
self.add_sort_field(name, SortDirection::Descending, null_order)
}
- /// Finished building the action and apply it to the transaction.
- pub fn apply(mut self) -> Result<Transaction> {
+ fn add_sort_field(
+ mut self,
+ name: &str,
+ sort_direction: SortDirection,
+ null_order: NullOrder,
+ ) -> Self {
+ self.pending_sort_fields.push(PendingSortField {
+ name: name.to_string(),
+ direction: sort_direction,
+ null_order,
+ });
+
+ self
+ }
+}
+
+impl Default for ReplaceSortOrderAction {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+#[async_trait]
+impl TransactionAction for ReplaceSortOrderAction {
+ async fn commit(self: Arc<Self>, table: &Table) -> Result<ActionCommit> {
+ let pending_sort_fields = self.pending_sort_fields.clone();
+
+ let sort_fields: Result<Vec<SortField>> = pending_sort_fields
+ .iter()
+ .map(|p| {
+ let field_id = table
Review Comment:
how about convert code here as a function of PendingSortField, e.g. PendingSortField::to_sort_field
##########
crates/iceberg/src/transaction/sort_order.rs:
##########
@@ -15,32 +15,97 @@
// specific language governing permissions and limitations
// under the License.
+use std::sync::Arc;
+
+use async_trait::async_trait;
+
use crate::error::Result;
use crate::spec::{NullOrder, SortDirection, SortField, SortOrder, Transform};
-use crate::transaction::Transaction;
+use crate::table::Table;
+use crate::transaction::{ActionCommit, TransactionAction};
use crate::{Error, ErrorKind, TableRequirement, TableUpdate};
+#[derive(Debug, PartialEq, Eq, Clone)]
+struct PendingSortField {
+ name: String,
+ direction: SortDirection,
+ null_order: NullOrder,
+}
+
/// Transaction action for replacing sort order.
pub struct ReplaceSortOrderAction {
- pub tx: Transaction,
- pub sort_fields: Vec<SortField>,
+ pending_sort_fields: Vec<PendingSortField>,
}
impl ReplaceSortOrderAction {
+ pub fn new() -> Self {
+ ReplaceSortOrderAction {
+ pending_sort_fields: vec![],
+ }
+ }
+
/// Adds a field for sorting in ascending order.
- pub fn asc(self, name: &str, null_order: NullOrder) -> Result<Self> {
+ pub fn asc(self, name: &str, null_order: NullOrder) -> Self {
Review Comment:
I noticed that in iceberg-java user can provide an expression here to indicates the transform of the sort field. We can do this here or open issue to do in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Implement TransactionAction for ReplaceSortOrderAction [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #1441:
URL: https://github.com/apache/iceberg-rust/pull/1441#discussion_r2146439072
##########
crates/iceberg/src/transaction/sort_order.rs:
##########
@@ -15,32 +15,97 @@
// specific language governing permissions and limitations
// under the License.
+use std::sync::Arc;
+
+use async_trait::async_trait;
+
use crate::error::Result;
use crate::spec::{NullOrder, SortDirection, SortField, SortOrder, Transform};
-use crate::transaction::Transaction;
+use crate::table::Table;
+use crate::transaction::{ActionCommit, TransactionAction};
use crate::{Error, ErrorKind, TableRequirement, TableUpdate};
+#[derive(Debug, PartialEq, Eq, Clone)]
+struct PendingSortField {
+ name: String,
+ direction: SortDirection,
+ null_order: NullOrder,
+}
+
Review Comment:
This is added to defer the construction of `Vec<SortField>` from `asc`/`desc` to `commit` to avoid passing `Table` to methods like `asc`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): Implement TransactionAction for ReplaceSortOrderAction [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #1441:
URL: https://github.com/apache/iceberg-rust/pull/1441#discussion_r2147267709
##########
crates/iceberg/src/transaction/sort_order.rs:
##########
@@ -15,32 +15,97 @@
// specific language governing permissions and limitations
// under the License.
+use std::sync::Arc;
+
+use async_trait::async_trait;
+
use crate::error::Result;
use crate::spec::{NullOrder, SortDirection, SortField, SortOrder, Transform};
-use crate::transaction::Transaction;
+use crate::table::Table;
+use crate::transaction::{ActionCommit, TransactionAction};
use crate::{Error, ErrorKind, TableRequirement, TableUpdate};
+#[derive(Debug, PartialEq, Eq, Clone)]
+struct PendingSortField {
+ name: String,
+ direction: SortDirection,
+ null_order: NullOrder,
+}
+
/// Transaction action for replacing sort order.
pub struct ReplaceSortOrderAction {
- pub tx: Transaction,
- pub sort_fields: Vec<SortField>,
+ pending_sort_fields: Vec<PendingSortField>,
}
impl ReplaceSortOrderAction {
+ pub fn new() -> Self {
+ ReplaceSortOrderAction {
+ pending_sort_fields: vec![],
+ }
+ }
+
/// Adds a field for sorting in ascending order.
- pub fn asc(self, name: &str, null_order: NullOrder) -> Result<Self> {
+ pub fn asc(self, name: &str, null_order: NullOrder) -> Self {
Review Comment:
Good point! I think supporting `Transform` based sorting in `ReplaceSortOrderAction` is a bigger effort and should be treated separately. I've created an issue here to track: https://github.com/apache/iceberg-rust/issues/1443
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org