You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "the80srobot (via GitHub)" <gi...@apache.org> on 2025/01/31 10:51:06 UTC
[I] Let arrow schemas specify the ENUM type from parquet schemas [arrow-rs]
the80srobot opened a new issue, #7050:
URL: https://github.com/apache/arrow-rs/issues/7050
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
I am exporting data from an application using an Arrow schema - the data is then written into a parquet file. Some fields are logically enums, and I would like to export them as the ENUM type in Parquet, but there is currently no way to specify that.
**Describe the solution you'd like**
In parquet, ENUM is an annotation of a BYTE_ARRAY, similar to STRING. Unlike String, it's parametrized with the allowable values.
It sounds like it might be an extension type, as in #5822 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Let arrow schemas specify the ENUM type from parquet schemas [arrow-rs]
Posted by "the80srobot (via GitHub)" <gi...@apache.org>.
the80srobot commented on issue #7050:
URL: https://github.com/apache/arrow-rs/issues/7050#issuecomment-2683080821
Sorry for the delay. I checked the C++ and Python versions and neither one of them provides a similar API.
I still think it might be useful to provide this, but as I learned more about the representation of enums in Parquet, I can see how it might be superfluous.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Let arrow schemas specify the ENUM type from parquet schemas [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #7050:
URL: https://github.com/apache/arrow-rs/issues/7050#issuecomment-2627497404
Does pyarrow or similar support this, as that might provide inspiration for how to support this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Let arrow schemas specify the ENUM type from parquet schemas [arrow-rs]
Posted by "jhorstmann (via GitHub)" <gi...@apache.org>.
jhorstmann commented on issue #7050:
URL: https://github.com/apache/arrow-rs/issues/7050#issuecomment-2628979408
> Unlike String, it's parametrized with the allowable values.
I don't think that is the case inside the parquet format itself, but it might be a useful part of the api to ensure the values get dictionary encoded
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org