You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "Jefffrey (via GitHub)" <gi...@apache.org> on 2026/07/29 00:40:51 UTC

[I] Avoid re-encoding dictionaries when writing to Parquet [arrow-rs]

Jefffrey opened a new issue, #10465:
URL: https://github.com/apache/arrow-rs/issues/10465

   
   ### Discussed in https://github.com/apache/arrow-rs/discussions/8300
   
   <div type='discussions-op-text'>
   
   <sup>Originally posted by **duarten** September  9, 2025</sup>
   It seems we always [re-encode dictionaries](https://github.com/apache/arrow-rs/blob/main/parquet/src/arrow/arrow_writer/byte_array.rs#L342) when writing row groups to Parquet. 
   
   Would it make sense to avoid this for cases where all/most dictionary values appear in all row groups?</div>
   
   Potentially related:
   
   - https://github.com/apache/arrow-rs/pull/9520
   - https://github.com/apache/arrow-rs/pull/2322


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org