You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "cbmixx (via GitHub)" <gi...@apache.org> on 2026/06/11 10:08:18 UTC
[PR] fix(parquet_derive): support raw identifiers as column names [arrow-rs]
cbmixx opened a new pull request, #10113:
URL: https://github.com/apache/arrow-rs/pull/10113
# Which issue does this PR close?
- Closes #10112.
# Rationale for this change
`#[derive(ParquetRecordReader)]` and `#[derive(ParquetRecordWriter)]` could not
handle a Parquet column whose name is a Rust keyword (e.g. `type`). The only way
to spell such a field in Rust is a raw identifier (`r#type`), but the derives
stringified the identifier including the `r#` prefix:
- The reader's column-index lookup used `name_to_index.get(stringify!(#field_names))`,
and `stringify!(r#type)` yields `"r#type"`, so reading failed with
`ParquetError::General("column name 'r#type' is not found in parquet file!")`.
- The writer's `Field::parquet_type()` used `self.ident.to_string()`, which keeps
the `r#` prefix, so the written schema got a column literally named `r#type`.
This made it impossible to read or write Parquet columns whose names are Rust
keywords, e.g. files produced by other Parquet writers with a column named `type`.
# What changes are included in this PR?
Unraw the identifier (via `syn::ext::IdentExt::unraw`, already available through
the existing `syn` dependency) wherever it is used as a column name, while keeping
the raw identifier for field access in the generated code:
- `parquet_derive/src/lib.rs`: the reader derive builds a parallel list of unrawed
field-name strings for the `name_to_index` lookup and its error message.
- `parquet_derive/src/parquet_field.rs`: `Field::parquet_type()` uses
`self.ident.unraw().to_string()` for the schema column name.
# Are these changes tested?
Yes. Added a unit test (`test_parquet_type_with_raw_identifier`) and an
integration round-trip test (`test_parquet_derive_raw_identifiers`) covering a
struct with a raw-identifier field (`r#type`) alongside a normal field, asserting
the schema columns are named `type`/`count`. I verified both tests fail without
the fix (the writer emits a column named `r#type`) and pass with it.
# Are there any user-facing changes?
Structs with raw-identifier fields now read and write columns named without the
`r#` prefix. This is a bug fix; there are no public API changes. Code that somehow
relied on the previous `r#`-prefixed column names would change behavior, but such
names could not be produced by any other Parquet writer.
---
*AI disclosure (per CONTRIBUTING.md): this change was developed with the
assistance of an AI coding tool. I reviewed every line, verified the fix against
the failing/passing tests described above, and own the change.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(parquet_derive): support raw identifiers as column names [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey merged PR #10113:
URL: https://github.com/apache/arrow-rs/pull/10113
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org