You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "mapleFU (via GitHub)" <gi...@apache.org> on 2025/05/28 16:48:53 UTC
[I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
mapleFU opened a new issue, #46631:
URL: https://github.com/apache/arrow/issues/46631
### Describe the enhancement requested
Currently, we've:
1. `store_schema`, which "suggest" a arrow read schema
2. `binary_type`, which set a arrow schema
So, does these two configs duplicate? And "Cast" might be applied if these two types is different( cast might be lightweight).
### Component(s)
C++, Parquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #46631:
URL: https://github.com/apache/arrow/issues/46631#issuecomment-2918590685
Let me add some comment about this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on issue #46631:
URL: https://github.com/apache/arrow/issues/46631#issuecomment-2918568987
Thanks for confirming it! So I don't think we have any issue, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on issue #46631:
URL: https://github.com/apache/arrow/issues/46631#issuecomment-2918045382
Isn't `store_schema` takes a higher priority than `properties.binary_type` if I understand it correctly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU closed issue #46631: [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow
URL: https://github.com/apache/arrow/issues/46631
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #46631:
URL: https://github.com/apache/arrow/issues/46631#issuecomment-2918562209
> Isn't `store_schema` takes a higher priority than `properties.binary_type` if I understand it correctly?
Yes, it does. `properties.binary_type` is for Parquet files that don't contain a Arrow schema.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #46631:
URL: https://github.com/apache/arrow/issues/46631#issuecomment-2919051215
```
/// However, if a serialized Arrow schema is found in the Parquet metadata,
/// this setting is ignored and the Arrow schema takes precedence
/// (see ArrowWriterProperties::store_schema).
void set_binary_type(::arrow::Type::type value) { binary_type_ = value; }
```
Since we already have this, my bad!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #46631:
URL: https://github.com/apache/arrow/issues/46631#issuecomment-2916994863
@pitrou @wgtmac do you have any opinion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #46631:
URL: https://github.com/apache/arrow/issues/46631#issuecomment-2917014271
I don't know, do you suggest something specifically?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #46631:
URL: https://github.com/apache/arrow/issues/46631#issuecomment-2918570465
Well, until people ask for the reverse behavior, I don't think so indeed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Rethink arrow::schema and binary_type config when reading arrow [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #46631:
URL: https://github.com/apache/arrow/issues/46631#issuecomment-2918571146
Also, for the record, I'm working on a similar property for Parquet list columns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org