[I] arrow-avro: object_store error type is lost in the async reader (stringified into AvroError::General) [arrow-rs]

["ranflarion (via GitHub)" <gi...@apache.org>]

ranflarion opened a new issue, #10493:
URL: https://github.com/apache/arrow-rs/issues/10493

   **Describe the bug**
   
   `AvroObjectReader`'s `spawn` / `spawn_stream` helpers convert every error from the underlying `ObjectStore` with `AvroError::General(e.to_string())` (arrow-avro/src/reader/async_reader/store.rs), and the header fetch loop wraps its error the same way (`"Error fetching Avro header from file reader: {err}"` in arrow-avro/src/reader/async_reader/builder.rs). The typed `object_store::Error` is destroyed at that point: `err.source()` returns nothing and there is no way to downcast back to `object_store::Error`.
   
   This matters for callers implementing retry or error-classification logic on top of the reader: a permanent `NotFound`/`PermissionDenied` is indistinguishable from a transient transport fault without string matching. We hit this while adding retry-on-transient-store-fault logic around an Avro scan and could not avoid retrying 404s.
   
   `ParquetObjectReader` already handles this correctly: its identical `spawn` helper takes `E: Into<ParquetError>` and parquet provides `impl From<object_store::Error> for ParquetError { ParquetError::External(Box::new(e)) }` (parquet/src/errors.rs), so the full source chain survives.
   
   **To Reproduce**
   
   ```rust
   let store = Arc::new(InMemory::new());
   let mut reader = AvroObjectReader::new(store, Path::from("missing.avro"));
   let err = reader.get_bytes(0..10).await.unwrap_err();
   // err is AvroError::General("Object at location missing.avro not found: ...")
   // err.source() is None; no downcast to object_store::Error is possible
   ```
   
   **Expected behavior**
   
   Align with `ParquetObjectReader`: add a feature-gated `impl From<object_store::Error> for AvroError` mapping to `AvroError::External(Box::new(e))`, change the private `spawn` bound to `E: Into<AvroError>`, and let the header fetch propagate the error instead of reformatting it. `AvroError::External` already reports its inner error via `source()`, and `From<AvroError> for ArrowError` maps it through `from_external_error`, so the chain then survives all the way to `ArrowError`.
   
   **Additional context**
   
   Error `Display` text changes for store failures (`External: ...` instead of `Avro error: ...`); variant selection within the `#[non_exhaustive]` `AvroError` is not a stability contract. Happy to submit a PR.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] arrow-avro: object_store error type is lost in the async reader (stringified into AvroError::General) [arrow-rs]

["Jefffrey (via GitHub)" <gi...@apache.org>]

Jefffrey closed issue #10493: arrow-avro: object_store error type is lost in the async reader (stringified into AvroError::General)
URL: https://github.com/apache/arrow-rs/issues/10493


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org