You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@parquet.apache.org by "emkornfield (via GitHub)" <gi...@apache.org> on 2025/12/19 18:28:14 UTC
[PR] Rephrase external column data. [parquet-site]
emkornfield opened a new pull request, #145:
URL: https://github.com/apache/parquet-site/pull/145
Based on support matrix I believe this was meant to capture Parquet summary files ("_metadata") and not reading external data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rephrase external column data. [parquet-site]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #145:
URL: https://github.com/apache/parquet-site/pull/145#discussion_r2660295324
##########
data/implementations/features/api-features.yaml:
##########
@@ -1,8 +1,8 @@
category_id: api-features
features:
- - id: api-external-column-data
- display_name: External column data
- note: "In parquet.thrift: ColumnChunk->file_path"
+ - id: api-parquet-summary-file
+ display_name: Parquet summary file
+ note: "Files named _metadata that consolidate footers from multiple parquet files."
Review Comment:
Yeah, I'd err on not including that a reference to file_path here.
> In fact is there anything in the spec about summary file or is that just an implementation choice some people made?
There is not https://github.com/apache/parquet-format/pull/542 has some more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rephrase external column data. [parquet-site]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #145:
URL: https://github.com/apache/parquet-site/pull/145#discussion_r2645636889
##########
data/implementations/features/api-features.yaml:
##########
@@ -1,8 +1,8 @@
category_id: api-features
features:
- - id: api-external-column-data
- display_name: External column data
- note: "In parquet.thrift: ColumnChunk->file_path"
+ - id: api-parquet-summary-file
+ display_name: Parquet summary file
+ note: "Files named _metadata that consolidate footers from multiple parquet files."
Review Comment:
I think the reference to the `file_path` field will help keep this specific. So perhaps something like
```suggestion
note: "Files named _metadata that consolidate footers from multiple parquet files, stored in ColumnChunk->file_path"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rephrase external column data. [parquet-site]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield merged PR #145:
URL: https://github.com/apache/parquet-site/pull/145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rephrase external column data. [parquet-site]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on PR #145:
URL: https://github.com/apache/parquet-site/pull/145#issuecomment-3725068668
going to merge. Thanks for the reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rephrase external column data. [parquet-site]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #145:
URL: https://github.com/apache/parquet-site/pull/145#discussion_r2635933446
##########
data/implementations/support/hyparquet.yaml:
##########
@@ -113,8 +113,8 @@ support:
status: unknown
format-data-page-v2:
status: full
- api-external-column-data:
- status: full
+ api-parquet-summary-file:
+ status: none
Review Comment:
@platypii I looked at hyparquet documentation quickly and didn't see this mentioned but please let me know if I missed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rephrase external column data. [parquet-site]
Posted by "platypii (via GitHub)" <gi...@apache.org>.
platypii commented on code in PR #145:
URL: https://github.com/apache/parquet-site/pull/145#discussion_r2646204830
##########
data/implementations/features/api-features.yaml:
##########
@@ -1,8 +1,8 @@
category_id: api-features
features:
- - id: api-external-column-data
- display_name: External column data
- note: "In parquet.thrift: ColumnChunk->file_path"
+ - id: api-parquet-summary-file
+ display_name: Parquet summary file
+ note: "Files named _metadata that consolidate footers from multiple parquet files."
Review Comment:
Is "parquet summary file" really the same as supporting `file_path`?
Maybe summary files are the only known use case of actually using `file_path`? But nothing in the spec says file_path is necessarily associated with "summary files". In fact is there anything in the spec about summary file or is that just an implementation choice some people made?
I'm not opposed to having a row for support of summary files, but that seems like a different question than `file_path` support?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org