You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "dentiny (via GitHub)" <gi...@apache.org> on 2025/05/08 08:49:50 UTC

[PR] chore: Add assertion for empty data files for append action [iceberg-rust]

dentiny opened a new pull request, #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301

   ## What changes are included in this PR?
   
   The general usage to append data files is
   ```rust
   let mut action = txn.fast_append(/*commit_uuid=*/ None, /*key_metadata=*/ vec![])?;
   action.add_data_files(new_data_files)?;
   let txn = action.apply().await?;
   let table = txn.commit(catalog).await?
   ```
   I found if ` new_data_files` is empty by accident, a new manifest file will still be created and tracked by manifest list file, which doesn't make sense to me (let me if team thinks it's a valid and ideal use case).
   
   In this PR, I did a few changes:
   - Add a new invalid argument error code;
   - Add assertion to fail empty data files append, and empty action commit use cases.
   
   ## Are these changes tested?
   
   Existing unit tests pass, and add a validation test for invalid data file append.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#issuecomment-2868361247

   The failed CI test doesn't seem to be related to my change:
   ```
   ailures:
   
   ---- test_drop_table stdout ----
   Error: Unexpected => Failure in doing io operation
   
   Source: ConfigInvalid (persistent) at write, context: { uri: http://172.18.0.3:9000/warehouse/hive/metadata/00000-b7e8f769-acc7-4051-9af9-ea3c2eb05962.metadata.json, response: Parts { status: 404, version: HTTP/1.1, headers: {"accept-ranges": "bytes", "content-length": "470", "content-type": "application/xml", "server": "MinIO", "strict-transport-security": "max-age=31536000; includeSubDomains", "vary": "Origin", "vary": "Accept-Encoding", "x-amz-id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8", "x-amz-request-id": "183E1001819BE226", "x-content-type-options": "nosniff", "x-xss-protection": "1; mode=block", "date": "Sat, 10 May 2025 04:28:24 GMT"} }, service: s3, path: hive/metadata/00000-b7e8f769-acc7-4051-9af9-ea3c2eb05962.metadata.json, written: 533 } => S3Error { code: "NoSuchBucket", message: "The specified bucket does not exist", resource: "/warehouse/hive/metadata/00000-b7e8f769-acc7-4051-9af9-ea3c2eb05962.metadata.json", request_id: "183E1001819BE2
 26" }
   
   
   ---- test_load_table stdout ----
   Error: Unexpected => Failure in doing io operation
   
   Source: ConfigInvalid (persistent) at write, context: { uri: http://172.18.0.3:9000/warehouse/hive/metadata/00000-852999b5-f3a5-4f69-b338-d3c817900f0b.metadata.json, response: Parts { status: 404, version: HTTP/1.1, headers: {"accept-ranges": "bytes", "content-length": "470", "content-type": "application/xml", "server": "MinIO", "strict-transport-security": "max-age=31536000; includeSubDomains", "vary": "Origin", "vary": "Accept-Encoding", "x-amz-id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8", "x-amz-request-id": "183E1001861B3ED1", "x-content-type-options": "nosniff", "x-xss-protection": "1; mode=block", "date": "Sat, 10 May 2025 04:28:24 GMT"} }, service: s3, path: hive/metadata/00000-852999b5-f3a5-4f69-b338-d3c817900f0b.metadata.json, written: 533 } => S3Error { code: "NoSuchBucket", message: "The specified bucket does not exist", resource: "/warehouse/hive/metadata/00000-852999b5-f3a5-4f69-b338-d3c817900f0b.metadata.json", request_id: "183E1001861B3E
 D1" }
   
   
   ---- test_create_table stdout ----
   Error: Unexpected => Failure in doing io operation
   
   Source: ConfigInvalid (persistent) at write, context: { uri: http://172.18.0.3:9000/warehouse/hive/metadata/00000-7c0aecbb-50e3-4173-af7c-4cac0cba4113.metadata.json, response: Parts { status: 404, version: HTTP/1.1, headers: {"accept-ranges": "bytes", "content-length": "470", "content-type": "application/xml", "server": "MinIO", "strict-transport-security": "max-age=31536000; includeSubDomains", "vary": "Origin", "vary": "Accept-Encoding", "x-amz-id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8", "x-amz-request-id": "183E1001819BE232", "x-content-type-options": "nosniff", "x-xss-protection": "1; mode=block", "date": "Sat, 10 May 2025 04:28:24 GMT"} }, service: s3, path: hive/metadata/00000-7c0aecbb-50e3-4173-af7c-4cac0cba4113.metadata.json, written: 533 } => S3Error { code: "NoSuchBucket", message: "The specified bucket does not exist", resource: "/warehouse/hive/metadata/00000-7c0aecbb-50e3-4173-af7c-4cac0cba4113.metadata.json", request_id: "183E1001819BE2
 32" }
   
   
   ---- test_rename_table stdout ----
   Error: Unexpected => Failure in doing io operation
   
   Source: ConfigInvalid (persistent) at write, context: { uri: http://172.18.0.3:9000/warehouse/hive/metadata/00000-8aacea18-eeaf-4461-9787-cb5fa133a7b0.metadata.json, response: Parts { status: 404, version: HTTP/1.1, headers: {"accept-ranges": "bytes", "content-length": "470", "content-type": "application/xml", "server": "MinIO", "strict-transport-security": "max-age=31536000; includeSubDomains", "vary": "Origin", "vary": "Accept-Encoding", "x-amz-id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8", "x-amz-request-id": "183E10018BC5FA17", "x-content-type-options": "nosniff", "x-xss-protection": "1; mode=block", "date": "Sat, 10 May 2025 04:28:24 GMT"} }, service: s3, path: hive/metadata/00000-8aacea18-eeaf-4461-9787-cb5fa133a7b0.metadata.json, written: 533 } => S3Error { code: "NoSuchBucket", message: "The specified bucket does not exist", resource: "/warehouse/hive/metadata/00000-8aacea18-eeaf-4461-9787-cb5fa133a7b0.metadata.json", request_id: "183E10018BC5FA
 17" }
   
   
   ---- test_table_exists stdout ----
   Error: Unexpected => Failure in doing io operation
   
   Source: ConfigInvalid (persistent) at write, context: { uri: http://172.18.0.3:9000/warehouse/hive/metadata/00000-c25734ce-2f8e-4bdd-8578-4942bf85027d.metadata.json, response: Parts { status: 404, version: HTTP/1.1, headers: {"accept-ranges": "bytes", "content-length": "470", "content-type": "application/xml", "server": "MinIO", "strict-transport-security": "max-age=31536000; includeSubDomains", "vary": "Origin", "vary": "Accept-Encoding", "x-amz-id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8", "x-amz-request-id": "183E10018CA7299B", "x-content-type-options": "nosniff", "x-xss-protection": "1; mode=block", "date": "Sat, 10 May 2025 04:28:24 GMT"} }, service: s3, path: hive/metadata/00000-c25734ce-2f8e-4bdd-8578-4942bf85027d.metadata.json, written: 533 } => S3Error { code: "NoSuchBucket", message: "The specified bucket does not exist", resource: "/warehouse/hive/metadata/00000-c25734ce-2f8e-4bdd-8578-4942bf85027d.metadata.json", request_id: "183E10018CA729
 9B" }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082749568


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {

Review Comment:
   If the given iterator is empty, we will end up with a manifest file with no manifest entries, with no warning / error happening, this is something I would like to avoid.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082750070


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {

Review Comment:
   In other words, I think having an empty manifest file doesn't make too much sense.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082607971


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {

Review Comment:
   The [error description](https://github.com/apache/iceberg-rust/blob/513376cb04b899edf457918c3852a786a292071c/crates/iceberg/src/error.rs#L36-L42) reads to me as "data corruption".



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082752488


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {

Review Comment:
   Additionally, I want to note that `add_data_files` can be called multiple times and is not directly mapped to a manifest file.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082760864


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -172,6 +179,13 @@ impl<'a> SnapshotProduceAction<'a> {
     // Write manifest file for added data files and return the ManifestFile for ManifestList.
     async fn write_added_manifest(&mut self) -> Result<ManifestFile> {
         let added_data_files = std::mem::take(&mut self.added_data_files);
+        if added_data_files.is_empty() {
+            return Err(Error::new(
+                ErrorKind::Unexpected,

Review Comment:
   Sounds good! I have removed change against input argument, and use `FailedPrecondition` as the error kind for empty data entry.
   
   The error name is borrowed from grpc status code: https://grpc.io/docs/guides/status-codes/#the-full-list-of-status-codes



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082615963


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {

Review Comment:
   Yes that makes sense, `InvalidArgument` or maybe `InvalidInput` is more intuitive.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082748345


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {

Review Comment:
   It seems safe to just return `Ok(())` while input iterator is empty. Returning error here seems no contribute for users' experrence. Users can't take actions on this error. 
   
   They have to add something like:
   
   ```rust
   if input_data.is_empty() {
       return Ok(())
   }
   xxx.add_data_files(input_data).await?;
   ```
   
   So how about we just check this and early return it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2083000330


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -172,6 +179,13 @@ impl<'a> SnapshotProduceAction<'a> {
     // Write manifest file for added data files and return the ManifestFile for ManifestList.
     async fn write_added_manifest(&mut self) -> Result<ManifestFile> {
         let added_data_files = std::mem::take(&mut self.added_data_files);
+        if added_data_files.is_empty() {
+            return Err(Error::new(
+                ErrorKind::Unexpected,

Review Comment:
   Ah sorry I didn't realize it, thanks for the heads up! Updated.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082611227


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {
+            return Err(Error::new(
+                ErrorKind::InvalidArgument,
+                "Doesn't add any data files when fast append",

Review Comment:
   Thank you! I think it makes sense, updated.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082752305


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {

Review Comment:
   > If the given iterator is empty, we will end up with a manifest file with no manifest entries, with no warning / error happening (it's exactly the same behavior you described), this is something I would like to avoid.
   
   This is valid and I do agree that we should not generate an empty manifest file. 
   
   However, the `add_data_files` function itself should be able to handle empty input safely, as users might use it in loops or pipelines with filters, where encountering an empty iterator is possible.
   
   Therefore, I prefer to perform this check when committing the manifest file, rather than within `add_data_files`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082993259


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -172,6 +179,13 @@ impl<'a> SnapshotProduceAction<'a> {
     // Write manifest file for added data files and return the ManifestFile for ManifestList.
     async fn write_added_manifest(&mut self) -> Result<ManifestFile> {
         let added_data_files = std::mem::take(&mut self.added_data_files);
+        if added_data_files.is_empty() {
+            return Err(Error::new(
+                ErrorKind::Unexpected,

Review Comment:
   Hi, it's better to align with our existing error kind naming style that put the noun first: `PreconditionFailed`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082750204


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -172,6 +179,13 @@ impl<'a> SnapshotProduceAction<'a> {
     // Write manifest file for added data files and return the ManifestFile for ManifestList.
     async fn write_added_manifest(&mut self) -> Result<ManifestFile> {
         let added_data_files = std::mem::take(&mut self.added_data_files);
+        if added_data_files.is_empty() {
+            return Err(Error::new(
+                ErrorKind::Unexpected,

Review Comment:
   Maybe we can add new error code like `PreconditionFailed` for cases where we have a precondition for this operation but users failed to meet it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082749568


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {

Review Comment:
   If the given iterator is empty, we will end up with a manifest file with no manifest entries, with no warning / error happening (it's exactly the same behavior you described), this is something I would like to avoid.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082761721


##########
crates/iceberg/src/error.rs:
##########
@@ -28,6 +28,9 @@ pub type Result<T> = std::result::Result<T, Error>;
 #[derive(Clone, Copy, Debug, PartialEq, Eq)]
 #[non_exhaustive]
 pub enum ErrorKind {
+    /// The operation was rejected because the system is not in a state required for the operation’s execution.

Review Comment:
   The description from error status is borrowed from https://grpc.io/docs/guides/status-codes/#the-full-list-of-status-codes



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] chore: Add assertion for empty data files for append action [iceberg-rust]

Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on code in PR #1301:
URL: https://github.com/apache/iceberg-rust/pull/1301#discussion_r2082494488


##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {
+            return Err(Error::new(
+                ErrorKind::InvalidArgument,
+                "Doesn't add any data files when fast append",

Review Comment:
   ```suggestion
                   "No data files were provided when adding data files to snapshot,
   ```
   This is not only for fast append



##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -129,6 +129,13 @@ impl<'a> SnapshotProduceAction<'a> {
         data_files: impl IntoIterator<Item = DataFile>,
     ) -> Result<&mut Self> {
         let data_files: Vec<DataFile> = data_files.into_iter().collect();
+        if data_files.is_empty() {

Review Comment:
   I think returning a `DataInvalid` error is good enough here, I don't know if invalid argument is needed. cc @Xuanwo 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org