You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "xanderbailey (via GitHub)" <gi...@apache.org> on 2026/07/30 14:15:08 UTC

[PR] Introduce invalid data error macro [iceberg-rust]

xanderbailey opened a new pull request, #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928

   ## Which issue does this PR close?
   
   <!--
   We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
   -->
   
   Today we construct `DataInvalid` like so: which ends up being very verbose in my opinion
   ```rust
   Error::new(
       ErrorKind::DataInvalid,
       "Partition column is not a StructArray",
   )
   ```
   
   
   ## What changes are included in this PR?
   
   Introduce `invalid_data!` macro as a shorthand for this error.
   
   <!--
   Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates.
   -->
   
   ## Are these changes tested?
   
   <!--
   Specify what test covers (unit test, integration test, etc.).
   
   If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
   -->
   
   ## AI Disclosure
   
   Claude wrote the script to do this migration but I have reviewed manually.
   <!--
   https://iceberg.apache.org/contribute/#guidelines-for-ai-assisted-contributions
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r3966656397


##########
crates/iceberg/src/avro/schema.rs:
##########
@@ -604,9 +587,8 @@ pub(crate) fn avro_schema_to_schema(avro_schema: &AvroSchema) -> Result<Schema>
             ))
         }
     } else {
-        Err(Error::new(
-            ErrorKind::DataInvalid,
-            "Can't convert non record avro schema to iceberg schema: {avro_schema}",
+        Err(invalid_data!(
+            "Can't convert non record avro schema to iceberg schema: {avro_schema}"

Review Comment:
   This is the one spot where the rewrite changes behavior. Before, `{avro_schema}` sat inside a plain `&str`, so the braces printed verbatim and the substitution never happened — a latent bug. The macro's literal arm wraps the string in `format!`, so now it actually interpolates `avro_schema` via `Display`.
   
   It's a better message, and it only compiles because `AvroSchema: Display` holds, so nothing's broken. But it's a silent semantic change on a cleanup that's meant to be behavior-preserving. I'd make it deliberate — note it in the PR description, or switch to `{avro_schema:?}` — so it reads as an intentional fix rather than an accident.



##########
crates/iceberg/src/arrow/record_batch_projector.rs:
##########
@@ -97,12 +97,9 @@ impl RecordBatchProjector {
         let field_id_fetch_func = |field: &Field| -> Result<Option<i64>> {
             if let Some(value) = field.metadata().get(PARQUET_FIELD_ID_META_KEY) {
                 let field_id = value.parse::<i32>().map_err(|e| {
-                    Error::new(
-                        ErrorKind::DataInvalid,
-                        "Failed to parse field id".to_string(),
-                    )
-                    .with_context("value", value)
-                    .with_source(e)
+                    invalid_data!("Failed to parse field id".to_string())

Review Comment:
   The `.to_string()` is redundant here — the literal arm already runs the message through `format!`, so `invalid_data!("Failed to parse field id")` produces the exact same allocation. Dropping the suffix keeps it on the more idiomatic literal arm.
   
   Same shape in about six other spots: `arrow/schema.rs` (the field-id and decimal-type branches), `spec/manifest/mod.rs` (both the serialize and deserialize sites), `spec/values/datum.rs` (the AboveMax/BelowMin arm), and `spec/schema/prune_columns.rs`.



##########
crates/iceberg/src/arrow/schema.rs:
##########
@@ -851,7 +806,7 @@ pub(crate) fn get_arrow_datum(datum: &Datum) -> Result<Arc<dyn ArrowDatum + Send
         }
         (PrimitiveType::Fixed(_), PrimitiveLiteral::Binary(value)) => {
             let array = FixedSizeBinaryArray::try_from_iter(std::iter::once(value.as_slice()))
-                .map_err(|e| Error::new(ErrorKind::DataInvalid, e.to_string()))?;
+                .map_err(|e| invalid_data!(e.to_string()))?;

Review Comment:
   `invalid_data!(e.to_string())` stringifies the source and drops the structured chain, so `err.source()` comes back `None`. Not a regression — the old code did the same — but since we're on the line, `invalid_data!("{e}").with_source(e)` keeps the chain and matches the many sites that already do `.with_source`. Same story for the other `e.to_string()` spots that don't chain the source.



##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,42 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// # Examples
+///
+///
+/// ```ignore
+/// use crate::error::invalid_data;
+///
+/// // As an expression
+/// let err = invalid_data!("unexpected value: {value}");
+///
+/// // With `.ok_or_else`
+/// let field = fields.get(id).ok_or_else(|| invalid_data!("missing field {id}"))?;
+///
+/// // Attaching a source error
+/// let n: i32 = s.parse().map_err(|e| invalid_data!("not an int: {s}").with_source(e))?;
+/// ```
+macro_rules! invalid_data {
+    ($fmt: literal $(, $($arg:tt)*)?) => {

Review Comment:
   One refinement on the literal arm: `invalid_data!("static message")` expands to `format!("static message")`, which clippy's `useless_format` can flag — and with `-D warnings` in the Makefile that could bite CI depending on toolchain. Worth confirming CI is green here, since clippy's firing span through macros varies by version.
   
   If it does fire, a dedicated no-arg arm sidesteps it and doubles as the canonical form for the bare-literal sites:
   
   ```rust
   ($fmt: literal) => {
       $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, $fmt)
   };
   ($fmt: literal, $($arg:tt)*) => {
       $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, format!($fmt, $($arg)*))
   };
   ($msg: expr $(,)?) => { /* unchanged */ };
   ```
   
   That way `invalid_data!("...")` skips `format!` entirely, and the leftover `.to_string()` calls just become bare literals. wdyt?



##########
crates/iceberg/src/error.rs:
##########
@@ -532,6 +568,25 @@ Source: networking error
         )
     }
 
+    #[test]
+    fn test_invalid_data_macro() {

Review Comment:
   The test covers the literal arm nicely but not the `expr` arm directly — it's only exercised via the migrated call sites compiling. A quick `let s = String::from("computed"); assert_eq!(invalid_data!(s).message(), "computed");` would pin it down.



##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,42 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// # Examples
+///
+///
+/// ```ignore
+/// use crate::error::invalid_data;
+///
+/// // As an expression
+/// let err = invalid_data!("unexpected value: {value}");
+///
+/// // With `.ok_or_else`
+/// let field = fields.get(id).ok_or_else(|| invalid_data!("missing field {id}"))?;
+///
+/// // Attaching a source error
+/// let n: i32 = s.parse().map_err(|e| invalid_data!("not an int: {s}").with_source(e))?;
+/// ```
+macro_rules! invalid_data {
+    ($fmt: literal $(, $($arg:tt)*)?) => {
+        $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, format!($fmt $(, $($arg)*)?))
+    };
+    ($msg: expr $(,)?) => {
+        $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, $msg)
+    };
+}
+
+// Crate-internal macro: re-exported so other modules can `use crate::error::invalid_data;`.
+pub(crate) use invalid_data;

Review Comment:
   Small thing while we're here: `ensure_data_valid!` just above is `#[macro_export]` but `invalid_data!` is `pub(crate)`. That's a defensible split, but nothing says so, and a future contributor could `#[macro_export]` this without realizing it'd leak into the public API. A one-line comment on the intent — plus a note in the doc example that the `use crate::error::invalid_data` path is crate-internal — would lock it in.



##########
crates/iceberg/src/spec/manifest/entry.rs:
##########
@@ -170,15 +170,13 @@ impl TryFrom<i32> for ManifestStatus {
             0 => Ok(ManifestStatus::Existing),
             1 => Ok(ManifestStatus::Added),
             2 => Ok(ManifestStatus::Deleted),
-            _ => Err(Error::new(
-                ErrorKind::DataInvalid,
-                format!("manifest status {v} is invalid"),
-            )),
+            _ => Err(invalid_data!("manifest status {v} is invalid")),
         }
     }
 }
 
 use super::DataFileFormat;
+use crate::error::invalid_data;

Review Comment:
   This `use crate::error::invalid_data;` landed mid-file next to the `static STATUS` block rather than in the top import group. Worth moving up with the other `use crate::` lines.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "blackmwk (via GitHub)" <gi...@apache.org>.
blackmwk commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r4069467657


##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,47 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// Unlike the public [`ensure_data_valid!`], this macro is deliberately
+/// crate-internal — adding `#[macro_export]` would commit it to the public API.
+///
+/// # Examples
+///
+/// The `use` path below is crate-internal and only resolves inside this crate.
+///
+/// ```ignore
+/// use crate::error::invalid_data;
+///
+/// // As an expression
+/// let err = invalid_data!("unexpected value: {value}");
+///
+/// // With `.ok_or_else`
+/// let field = fields.get(id).ok_or_else(|| invalid_data!("missing field {id}"))?;
+///
+/// // Attaching a source error
+/// let n: i32 = s.parse().map_err(|e| invalid_data!("not an int: {s}").with_source(e))?;
+/// ```
+macro_rules! invalid_data {

Review Comment:
   Yes, sth similar.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "xanderbailey (via GitHub)" <gi...@apache.org>.
xanderbailey commented on PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#issuecomment-5730266664

   CI blocked on https://github.com/apache/iceberg-rust/pull/3249


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "xanderbailey (via GitHub)" <gi...@apache.org>.
xanderbailey commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r4005446738


##########
crates/iceberg/src/avro/schema.rs:
##########
@@ -604,9 +587,8 @@ pub(crate) fn avro_schema_to_schema(avro_schema: &AvroSchema) -> Result<Schema>
             ))
         }
     } else {
-        Err(Error::new(
-            ErrorKind::DataInvalid,
-            "Can't convert non record avro schema to iceberg schema: {avro_schema}",
+        Err(invalid_data!(
+            "Can't convert non record avro schema to iceberg schema: {avro_schema}"

Review Comment:
   I've update the PR description, hope that works for you!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r4004814640


##########
crates/iceberg/src/arrow/schema.rs:
##########
@@ -851,7 +805,7 @@ pub(crate) fn get_arrow_datum(datum: &Datum) -> Result<Arc<dyn ArrowDatum + Send
         }
         (PrimitiveType::Fixed(_), PrimitiveLiteral::Binary(value)) => {
             let array = FixedSizeBinaryArray::try_from_iter(std::iter::once(value.as_slice()))
-                .map_err(|e| Error::new(ErrorKind::DataInvalid, e.to_string()))?;
+                .map_err(|e| invalid_data!("{e}").with_source(e))?;

Review Comment:
   Small one — `{e}` formats the Arrow error into the message and then `.with_source(e)` attaches it again, so the same text shows up twice in the Display output. Not wrong, and it's actually more than the pre-PR code carried (which dropped the source entirely), but the decimal conversions just above use the brief-message-plus-source form. I'd match them:
   
   ```suggestion
                   .map_err(|e| invalid_data!("FixedSizeBinary conversion failed").with_source(e))?;
   ```



##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,47 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// Unlike the public [`ensure_data_valid!`], this macro is deliberately
+/// crate-internal — adding `#[macro_export]` would commit it to the public API.
+///
+/// # Examples
+///
+/// The `use` path below is crate-internal and only resolves inside this crate.
+///
+/// ```ignore
+/// use crate::error::invalid_data;
+///
+/// // As an expression
+/// let err = invalid_data!("unexpected value: {value}");
+///
+/// // With `.ok_or_else`
+/// let field = fields.get(id).ok_or_else(|| invalid_data!("missing field {id}"))?;
+///
+/// // Attaching a source error
+/// let n: i32 = s.parse().map_err(|e| invalid_data!("not an int: {s}").with_source(e))?;
+/// ```
+macro_rules! invalid_data {
+    // Bare literals stay in `format!` so inline captures like `{id}` still interpolate.
+    ($fmt: literal $(, $($arg:tt)*)?) => {
+        $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, format!($fmt $(, $($arg)*)?))

Review Comment:
   This is the one new thing I'd want to nail down before merge. The literal arm always routes through `format!`, so every zero-arg call — `invalid_data!("File already closed")` and the ~30 others like it — expands to `format!("File already closed")`, which is exactly what `clippy::useless_format` fires on. CI runs clippy with `-D warnings`, so if it surfaces from the expansion the build goes red.
   
   Whether it actually surfaces through a `macro_rules!` expansion is version-dependent, so the real question is just: is clippy green on this PR? If it is, I'm happy to leave the single-arm design as-is. If not, I'd add a zero-arg arm ahead of the variadic one:
   
   ```rust
       ($msg: literal) => {
           $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, $msg.to_owned())
       };
   ```
   
   (the trade-off being that inline-capture calls like `invalid_data!("text {var}")` would then need the explicit `invalid_data!("text {}", var)` form). wdyt?



##########
crates/iceberg/src/avro/schema.rs:
##########
@@ -604,9 +587,8 @@ pub(crate) fn avro_schema_to_schema(avro_schema: &AvroSchema) -> Result<Schema>
             ))
         }
     } else {
-        Err(Error::new(
-            ErrorKind::DataInvalid,
-            "Can't convert non record avro schema to iceberg schema: {avro_schema}",
+        Err(invalid_data!(
+            "Can't convert non record avro schema to iceberg schema: {avro_schema}"

Review Comment:
   This is the interpolation from last round — still riding along silently. `{avro_schema}` printed verbatim before; through the macro's `format!` arm it now actually interpolates, and it only compiles because `AvroSchema: Display` happens to hold.
   
   I'm not asking to revert it, it's a strictly better message. Just make it deliberate — a line in the description, or `{avro_schema:?}` — so it's on the record rather than an accident. wdyt?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#issuecomment-5472233364

   This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "xanderbailey (via GitHub)" <gi...@apache.org>.
xanderbailey commented on PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#issuecomment-5522314591

   @blackmwk just wanted to tag you for thoughts before this goes stale, if we're happy then I'll resolve conflicts 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "blackmwk (via GitHub)" <gi...@apache.org>.
blackmwk commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r4014494079


##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,47 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// Unlike the public [`ensure_data_valid!`], this macro is deliberately
+/// crate-internal — adding `#[macro_export]` would commit it to the public API.
+///
+/// # Examples
+///
+/// The `use` path below is crate-internal and only resolves inside this crate.
+///
+/// ```ignore
+/// use crate::error::invalid_data;
+///
+/// // As an expression
+/// let err = invalid_data!("unexpected value: {value}");
+///
+/// // With `.ok_or_else`
+/// let field = fields.get(id).ok_or_else(|| invalid_data!("missing field {id}"))?;
+///
+/// // Attaching a source error
+/// let n: i32 = s.parse().map_err(|e| invalid_data!("not an int: {s}").with_source(e))?;
+/// ```
+macro_rules! invalid_data {

Review Comment:
   nit, should we have an arm which includes exception so that we don't need to write `with_source`?



##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,47 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// Unlike the public [`ensure_data_valid!`], this macro is deliberately
+/// crate-internal — adding `#[macro_export]` would commit it to the public API.
+///
+/// # Examples
+///
+/// The `use` path below is crate-internal and only resolves inside this crate.
+///
+/// ```ignore
+/// use crate::error::invalid_data;
+///
+/// // As an expression
+/// let err = invalid_data!("unexpected value: {value}");
+///
+/// // With `.ok_or_else`
+/// let field = fields.get(id).ok_or_else(|| invalid_data!("missing field {id}"))?;
+///
+/// // Attaching a source error
+/// let n: i32 = s.parse().map_err(|e| invalid_data!("not an int: {s}").with_source(e))?;
+/// ```
+macro_rules! invalid_data {
+    // Bare literals stay in `format!` so inline captures like `{id}` still interpolate.
+    ($fmt: literal $(, $($arg:tt)*)?) => {
+        $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, format!($fmt $(, $($arg)*)?))
+    };
+    ($msg: expr $(,)?) => {
+        $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, $msg)
+    };
+}
+

Review Comment:
   I see a lot of places has repeated `return Err(invalid_data!())`, should we have a macro for it as well?



##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,47 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// Unlike the public [`ensure_data_valid!`], this macro is deliberately
+/// crate-internal — adding `#[macro_export]` would commit it to the public API.
+///
+/// # Examples
+///
+/// The `use` path below is crate-internal and only resolves inside this crate.
+///
+/// ```ignore

Review Comment:
   We should not ignore this, we should compile it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "xanderbailey (via GitHub)" <gi...@apache.org>.
xanderbailey commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r4015078421


##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,47 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// Unlike the public [`ensure_data_valid!`], this macro is deliberately
+/// crate-internal — adding `#[macro_export]` would commit it to the public API.
+///
+/// # Examples
+///
+/// The `use` path below is crate-internal and only resolves inside this crate.
+///
+/// ```ignore
+/// use crate::error::invalid_data;
+///
+/// // As an expression
+/// let err = invalid_data!("unexpected value: {value}");
+///
+/// // With `.ok_or_else`
+/// let field = fields.get(id).ok_or_else(|| invalid_data!("missing field {id}"))?;
+///
+/// // Attaching a source error
+/// let n: i32 = s.parse().map_err(|e| invalid_data!("not an int: {s}").with_source(e))?;
+/// ```
+macro_rules! invalid_data {
+    // Bare literals stay in `format!` so inline captures like `{id}` still interpolate.
+    ($fmt: literal $(, $($arg:tt)*)?) => {
+        $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, format!($fmt $(, $($arg)*)?))
+    };
+    ($msg: expr $(,)?) => {
+        $crate::error::Error::new($crate::error::ErrorKind::DataInvalid, $msg)
+    };
+}
+

Review Comment:
   Could we do this kind of change as a follow up? It would be a reasonable change on top of this PR I think



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r4046607858


##########
crates/iceberg/src/arrow/schema.rs:
##########
@@ -851,7 +805,7 @@ pub(crate) fn get_arrow_datum(datum: &Datum) -> Result<Arc<dyn ArrowDatum + Send
         }
         (PrimitiveType::Fixed(_), PrimitiveLiteral::Binary(value)) => {
             let array = FixedSizeBinaryArray::try_from_iter(std::iter::once(value.as_slice()))
-                .map_err(|e| Error::new(ErrorKind::DataInvalid, e.to_string()))?;
+                .map_err(|e| invalid_data!("FixedSizeBinary conversion failed").with_source(e))?;

Review Comment:
   this one quietly changes the visible message — it used to surface arrow's own error text as `.message()`, now it's the static `"FixedSizeBinary conversion failed"` with the arrow error tucked into `.with_source(e)`.
   
   I actually think the new shape is better since the source chain keeps everything, it's just the one spot in an otherwise mechanical pass that changes what a caller sees. I'd add a line to the PR description calling it out alongside the avro fix so it's not a silent behaviour change. wdyt?



##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,47 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// Unlike the public [`ensure_data_valid!`], this macro is deliberately
+/// crate-internal — adding `#[macro_export]` would commit it to the public API.
+///
+/// # Examples
+///
+/// The `use` path below is crate-internal and only resolves inside this crate.
+///
+/// ```ignore

Review Comment:
   re: the open thread about compiling this instead of `ignore` — I think `ignore` is right here. doctests compile as their own external crate, so they can't resolve a `pub(crate)` item no matter the toolchain, and making the macro `pub` just to green the doctest would contradict the "deliberately crate-internal" line right above it. I'd leave it as-is and close that thread. wdyt?



##########
crates/iceberg/src/avro/schema.rs:
##########
@@ -604,9 +587,8 @@ pub(crate) fn avro_schema_to_schema(avro_schema: &AvroSchema) -> Result<Schema>
             ))
         }
     } else {
-        Err(Error::new(
-            ErrorKind::DataInvalid,
-            "Can't convert non record avro schema to iceberg schema: {avro_schema}",
+        Err(invalid_data!(
+            "Can't convert non record avro schema to iceberg schema: {avro_schema}"

Review Comment:
   this is the interpolation fix from the description landing — `{avro_schema}` was riding along as literal text before and now actually renders through the macro's `format!` arm.
   
   since it's the one real behaviour change in the PR, could we add a small test that runs `avro_schema_to_schema` on a non-record schema (a bare `AvroSchema::Boolean` would do) and asserts the message contains the schema's Display rather than the literal `{avro_schema}`? that locks the fix in so it can't quietly regress. not blocking.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "xanderbailey (via GitHub)" <gi...@apache.org>.
xanderbailey commented on PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#issuecomment-5132126188

   Not sure what folks think about this change but I personally think it improves readability.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "xanderbailey (via GitHub)" <gi...@apache.org>.
xanderbailey commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r4015064005


##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,47 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// Unlike the public [`ensure_data_valid!`], this macro is deliberately
+/// crate-internal — adding `#[macro_export]` would commit it to the public API.
+///
+/// # Examples
+///
+/// The `use` path below is crate-internal and only resolves inside this crate.
+///
+/// ```ignore

Review Comment:
   The macro is pub(crate) use, so a doctest — which compiles as an external crate — can't resolve it. Should we make it pub?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "xanderbailey (via GitHub)" <gi...@apache.org>.
xanderbailey commented on PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#issuecomment-5730263586

   I think that's all sorted now, thanks for the reviews!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "anoopj (via GitHub)" <gi...@apache.org>.
anoopj commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r3687805541


##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,42 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// # Examples
+///
+///
+/// ```ignore
+/// use crate::error::invalid_data;
+///
+/// // As an expression
+/// let err = invalid_data!("unexpected value: {value}");
+///
+/// // With `.ok_or_else`
+/// let field = fields.get(id).ok_or_else(|| invalid_data!("missing field {id}"))?;
+///
+/// // Attaching a source error
+/// let n: i32 = s.parse().map_err(|e| invalid_data!("not an int: {s}").with_source(e))?;
+/// ```
+macro_rules! invalid_data {

Review Comment:
    Just one design thought: the `expr` arm accepts any expression, so if somone writes `invalid_data!(format!(...))` (out of habit), we will double allocate. Not worth guarding against though. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "xanderbailey (via GitHub)" <gi...@apache.org>.
xanderbailey commented on PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#issuecomment-5779643798

   @laskoviymishka or @blackmwk are you able to add this back into the merge queue? I think the semantic merge conflict fix bumped it off the auto merge.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka merged PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Introduce invalid data error macro [iceberg-rust]

Posted by "xanderbailey (via GitHub)" <gi...@apache.org>.
xanderbailey commented on code in PR #2928:
URL: https://github.com/apache/iceberg-rust/pull/2928#discussion_r4015070835


##########
crates/iceberg/src/error.rs:
##########
@@ -469,6 +469,47 @@ macro_rules! ensure_data_valid {
     };
 }
 
+/// Helper macro to construct an [`ErrorKind::DataInvalid`] error.
+///
+/// This is a shorthand for `Error::new(ErrorKind::DataInvalid, ...)`, the most
+/// common error constructed in this crate. It returns the [`Error`] value (it
+/// does *not* return from the enclosing function), so it composes with `?`,
+/// `.map_err(...)`, `.ok_or_else(...)`, and explicit `return Err(...)`.
+///
+/// The message may be a plain expression or a format string with arguments.
+///
+/// Unlike the public [`ensure_data_valid!`], this macro is deliberately
+/// crate-internal — adding `#[macro_export]` would commit it to the public API.
+///
+/// # Examples
+///
+/// The `use` path below is crate-internal and only resolves inside this crate.
+///
+/// ```ignore
+/// use crate::error::invalid_data;
+///
+/// // As an expression
+/// let err = invalid_data!("unexpected value: {value}");
+///
+/// // With `.ok_or_else`
+/// let field = fields.get(id).ok_or_else(|| invalid_data!("missing field {id}"))?;
+///
+/// // Attaching a source error
+/// let n: i32 = s.parse().map_err(|e| invalid_data!("not an int: {s}").with_source(e))?;
+/// ```
+macro_rules! invalid_data {

Review Comment:
   ```
   invalid_data!("msg").with_source(e) → invalid_data!("msg", source = e)
   ```
   Like this? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org