You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "asfimport (via GitHub)" <gi...@apache.org> on 2024/06/23 06:33:37 UTC
[I] [C++][Parquet] Support INT96 and FIXED_LEN_BYTE_ARRAY types [arrow]
asfimport opened a new issue, #42267:
URL: https://github.com/apache/arrow/issues/42267
I would like to add support for INT96 and FIXED_LEN_BYTE_ARRAY parquet types.
Hive data types DATE and TIMESTAMP get mapped to INT96 parquet type.
Hive DECIMAL gets mapped to parquet FIXED_LEN_BYTE_ARRAY type.
**Reporter**: [Deepak Majeti](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mdeepak) / @majetideepak
**Assignee**: Deepak Majeti](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mdeepak) / @majetideepak
<sub>**Note**: *This issue was originally created as [PARQUET-428](https://issues.apache.org/jira/browse/PARQUET-428). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Support INT96 and FIXED_LEN_BYTE_ARRAY types [arrow]
Posted by "asfimport (via GitHub)" <gi...@apache.org>.
asfimport commented on issue #42267:
URL: https://github.com/apache/arrow/issues/42267#issuecomment-2184204438
[Ryan Blue](https://issues.apache.org/jira/browse/PARQUET-428?#comment-15102254) / @rdblue:
@majetideepak, thanks for taking this on! You're right about the Hive mappings so you'll need this to support DECIMAL and the current TIMESTAMP. A couple of things to keep in mind on that eventual goal: first, while Hive maps decimal to an underlying fixed, there are several primitive types allowed by the spec that need to be supported when it comes time. I'm happy to help when it comes to ensuring compatibility so just open up a JIRA and ping me. Second, the int96 timestamp type has a lot of problems that I'm trying to fix in various processing engines. Again, when it comes time to implement we can help you through the compatibility story.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Support INT96 and FIXED_LEN_BYTE_ARRAY types [arrow]
Posted by "asfimport (via GitHub)" <gi...@apache.org>.
asfimport commented on issue #42267:
URL: https://github.com/apache/arrow/issues/42267#issuecomment-2184204440
[Deepak Majeti](https://issues.apache.org/jira/browse/PARQUET-428?#comment-15106895) / @majetideepak:
@rdblue, My plan is to first extend the parquet reader to read bits of all the Parquet types (as of today only INT96 and FIXED_LEN_BYTE_ARRAY are missing). Once we complete this, we can support other primitive and complex types (of hive) by just re-interpreting these bits.
I have already started implementing support for INT96 in the reader. I will be happy to discuss the compatibility story at this point. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Support INT96 and FIXED_LEN_BYTE_ARRAY types [arrow]
Posted by "asfimport (via GitHub)" <gi...@apache.org>.
asfimport commented on issue #42267:
URL: https://github.com/apache/arrow/issues/42267#issuecomment-2184204442
[Julien Le Dem](https://issues.apache.org/jira/browse/PARQUET-428?#comment-15122703) / @julienledem:
Issue resolved by pull request 27
<https://github.com/apache/parquet-cpp/pull/27>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org