You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Fokko (via GitHub)" <gi...@apache.org> on 2023/04/24 13:05:09 UTC
[GitHub] [arrow] Fokko opened a new issue, #35304: [Python] Read ORC metadata
Fokko opened a new issue, #35304:
URL: https://github.com/apache/arrow/issues/35304
### Describe the enhancement requested
When reading an ORC schema, the metadata of the fields isn't exposed. For more information check: https://github.com/apache/iceberg/issues/6973#issuecomment-1465090060
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #35304: [Python] Read ORC metadata
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #35304:
URL: https://github.com/apache/arrow/issues/35304#issuecomment-1523012332
In C++, there is a `ORCFileReader::ReadMetadata()`, which is exposed as the `metadata` property in the Python `ORCReader`. Does that already give the relevant information, or is this a different set of metadata? (I know that arrow and parquet also both have file-level metadata as field-level metadata, so the same might be the case here)
I see that the ORC C++ `orc::Type` has a `getAttributeKeys`, and the proto spec has an `attributes` field for a Type (https://github.com/apache/orc/blob/cd79029e41e9cfe434529f820e73900e6a72797e/proto/orc_proto.proto#L226). Is this the information that you looking for?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed issue #35304: [Python] Read ORC metadata
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #35304: [Python] Read ORC metadata
URL: https://github.com/apache/arrow/issues/35304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wgtmac commented on issue #35304: [Python] Read ORC metadata
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on issue #35304:
URL: https://github.com/apache/arrow/issues/35304#issuecomment-1538554351
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wgtmac commented on issue #35304: [Python] Read ORC metadata
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on issue #35304:
URL: https://github.com/apache/arrow/issues/35304#issuecomment-1538516563
> In C++, there is a `ORCFileReader::ReadMetadata()`, which is exposed as the `metadata` property in the Python `ORCReader`. Does that already give the relevant information, or is this a different set of metadata? (I know that arrow and parquet also both have file-level metadata as field-level metadata, so the same might be the case here)
>
> I see that the ORC C++ `orc::Type` has a `getAttributeKeys`, and the proto spec has an `attributes` field for a Type (https://github.com/apache/orc/blob/cd79029e41e9cfe434529f820e73900e6a72797e/proto/orc_proto.proto#L226). Is this the information that you looking for?
I think @Fokko is looking for a way to expose `attributes` from ORC. `attributes` store the information about column field-id defined by Apache Iceberg.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org