You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2024/02/15 17:44:57 UTC
[I] [C++][Parquet] Expand ParquetVersion enum values [arrow]
pitrou opened a new issue, #40096:
URL: https://github.com/apache/arrow/issues/40096
### Describe the enhancement requested
The latest released Parquet format version is 2.10.0, but our ParquetVersion enum only goes up to 2.6.0. We should fill in the missing values. For example, 2.8.0 adds the BYTE_STREAM_SPLIT encoding for floats.
### Component(s)
C++, Parquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Expand ParquetVersion enum values [arrow]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on issue #40096:
URL: https://github.com/apache/arrow/issues/40096#issuecomment-3336841990
@diego-ciciani01 Feel free to create a PR :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Expand ParquetVersion enum values [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #40096:
URL: https://github.com/apache/arrow/issues/40096#issuecomment-3352654045
I have an [in-progress draft PR](https://github.com/apache/arrow/pull/47618/files) up for this already, as I had assumed from the `good-first-issue` label that we just needed to add the version numbers in. It sounds like it's more complex than that, so if that's the case, feel free to take it over in a new PR, as I don't have the capacity to complete those extra bits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Expand ParquetVersion enum values [arrow]
Posted by "diego-ciciani01 (via GitHub)" <gi...@apache.org>.
diego-ciciani01 commented on issue #40096:
URL: https://github.com/apache/arrow/issues/40096#issuecomment-3339777166
Thanks for the feedback above. I’ve been digging a bit deeper into the issue, and I now understand why simply adding new values to the `ParquetVersion` enum might not be straightforward.
I think we should start by researching the exact features introduced in each version (2.7-2.10) from the Parquet spec changelog. For the version mismatch issue, we could consider adding a validation step to ensure written files declare the minimum required version for their actual features used, or something like that.
Let me know if I understood what you meant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Expand ParquetVersion enum values [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #40096:
URL: https://github.com/apache/arrow/issues/40096#issuecomment-1946739192
cc @jorisvandenbossche @mapleFU @wgtmac
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Expand ParquetVersion enum values [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #40096:
URL: https://github.com/apache/arrow/issues/40096#issuecomment-3336874132
Personally I think this would be a bit tricky, what would you plan to be in 2.10?
And some 2.10 files might be written with 2.6.0 now? 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Expand ParquetVersion enum values [arrow]
Posted by "diego-ciciani01 (via GitHub)" <gi...@apache.org>.
diego-ciciani01 commented on issue #40096:
URL: https://github.com/apache/arrow/issues/40096#issuecomment-3333691064
Hi, is it still necessary to continue with this issue? If so, I can help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Expand ParquetVersion enum values [arrow]
Posted by "imeghar2408-max (via GitHub)" <gi...@apache.org>.
imeghar2408-max commented on issue #40096:
URL: https://github.com/apache/arrow/issues/40096#issuecomment-5297891445
Hi @pitrou, I'd like to work on this issue. Is it okay if I take it up?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org