You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@parquet.apache.org by "ArnavBalyan (via GitHub)" <gi...@apache.org> on 2025/11/21 14:07:09 UTC
[PR] Add FSST support as an active proposal [parquet-format]
ArnavBalyan opened a new pull request, #532:
URL: https://github.com/apache/parquet-format/pull/532
### Rationale for this change
- Added FSST support under active proposals
### What changes are included in this PR?
- Updated proposals
### Do these changes have PoC implementations?
- No
Closes #531
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Add FSST support as an active proposal [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield merged PR #532:
URL: https://github.com/apache/parquet-format/pull/532
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Add FSST support as an active proposal [parquet-format]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on PR #532:
URL: https://github.com/apache/parquet-format/pull/532#issuecomment-3563180163
cc @julienledem @emkornfield could you please review, just adding FSST as an active proposal thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Add FSST support as an active proposal [parquet-format]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on code in PR #532:
URL: https://github.com/apache/parquet-format/pull/532#discussion_r2553449655
##########
proposals/README.md:
##########
@@ -58,8 +58,7 @@ Once the implementation phase is finished, we can include the contribution in th
| ID | Description | Status |
|-----|--------------|---------|
-| [github issue] | adding this new encoding | POC |
-| [github issue] | add Variant type | Implementation |
+| [Issue #531](https://github.com/apache/parquet-format/issues/531) | FSST support for Parquet format | Implementation |
Review Comment:
Updated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Add FSST support as an active proposal [parquet-format]
Posted by "julienledem (via GitHub)" <gi...@apache.org>.
julienledem commented on code in PR #532:
URL: https://github.com/apache/parquet-format/pull/532#discussion_r2553441643
##########
proposals/README.md:
##########
@@ -58,8 +58,7 @@ Once the implementation phase is finished, we can include the contribution in th
| ID | Description | Status |
|-----|--------------|---------|
-| [github issue] | adding this new encoding | POC |
-| [github issue] | add Variant type | Implementation |
+| [Issue #531](https://github.com/apache/parquet-format/issues/531) | FSST support for Parquet format | Implementation |
Review Comment:
+1 let's have the status "Draft/POC"
And please add the link to your doc in the issue.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Add FSST support as an active proposal [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #532:
URL: https://github.com/apache/parquet-format/pull/532#discussion_r2553307200
##########
proposals/README.md:
##########
@@ -58,8 +58,7 @@ Once the implementation phase is finished, we can include the contribution in th
| ID | Description | Status |
|-----|--------------|---------|
-| [github issue] | adding this new encoding | POC |
-| [github issue] | add Variant type | Implementation |
+| [Issue #531](https://github.com/apache/parquet-format/issues/531) | FSST support for Parquet format | Implementation |
Review Comment:
I don't think we are at implementation yet. I think we still want to work out:
1. New page layout
2. Sharing dictionaries
The issue also doesn't have links to any of the docs produced so far?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Add FSST support as an active proposal [parquet-format]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on PR #532:
URL: https://github.com/apache/parquet-format/pull/532#issuecomment-3567248328
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Add FSST support as an active proposal [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #532:
URL: https://github.com/apache/parquet-format/pull/532#discussion_r2554060200
##########
proposals/README.md:
##########
@@ -58,8 +58,7 @@ Once the implementation phase is finished, we can include the contribution in th
| ID | Description | Status |
|-----|--------------|---------|
-| [github issue] | adding this new encoding | POC |
-| [github issue] | add Variant type | Implementation |
+| [Issue #531](https://github.com/apache/parquet-format/issues/531) | FSST support for Parquet format | Implementation |
Review Comment:
Does this include FSST-12 or just FSST-8 (we don't need to discuss here, looking forward to seeing the numbers).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Add FSST support as an active proposal [parquet-format]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on code in PR #532:
URL: https://github.com/apache/parquet-format/pull/532#discussion_r2554027728
##########
proposals/README.md:
##########
@@ -58,8 +58,7 @@ Once the implementation phase is finished, we can include the contribution in th
| ID | Description | Status |
|-----|--------------|---------|
-| [github issue] | adding this new encoding | POC |
-| [github issue] | add Variant type | Implementation |
+| [Issue #531](https://github.com/apache/parquet-format/issues/531) | FSST support for Parquet format | Implementation |
Review Comment:
Hi @emkornfield I'll raise a draft PR for this, we would not need shared dictionaries since the symbol table empirically works best on a per-page level, and only a few hundred bytes. The existing decoder holds the symbol table as the prefix similar to DELTA_BINARY_PACKED encoder, can discuss more on the pr thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Add FSST support as an active proposal [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on PR #532:
URL: https://github.com/apache/parquet-format/pull/532#issuecomment-3567935893
LGTM, merging. Thank you @ArnavBalyan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org