You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "matanper (via GitHub)" <gi...@apache.org> on 2026/06/02 09:00:39 UTC
[PR] feat(table): add variant shredded reader [iceberg-go]
matanper opened a new pull request, #1152:
URL: https://github.com/apache/iceberg-go/pull/1152
- Adds ReassembleShreddedVariant — a named entry point that reconstructs a variant.Value from one row of a shredded Parquet variant column, delegating the spec walk to arrow-go's *extensions.VariantArray.
- Adds WrapShreddedVariantReader, a RecordReader wrapper that transparently replaces shredded variant columns with their non-shredded equivalents on every batch, so the scanner always sees a uniform struct<metadata, value> layout regardless of how the Parquet file was written.
- Wires WrapShreddedVariantReader into parquet_files.go's GetRecords path.
- Vendors six Java-produced fixture files from apache/parquet-testing (pinned to a specific commit) and adds two tests — one exercising the pure per-row function, one exercising the full scan path — covering primitive shredding, flat objects, nested shredded objects, and partially-shredded objects (typed_value + residual merge).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(table): read shredded Parquet variants [iceberg-go]
Posted by "matanper (via GitHub)" <gi...@apache.org>.
matanper closed pull request #1152: feat(table): read shredded Parquet variants
URL: https://github.com/apache/iceberg-go/pull/1152
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(table): read shredded Parquet variants [iceberg-go]
Posted by "nssalian (via GitHub)" <gi...@apache.org>.
nssalian commented on PR #1152:
URL: https://github.com/apache/iceberg-go/pull/1152#issuecomment-4609604007
@matanper I was going to put out a proposal before doing this. I'll fix this once I get consensus with the community. Also, please check prior to working on an issue if already assigned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(table): read shredded Parquet variants [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #1152:
URL: https://github.com/apache/iceberg-go/pull/1152#issuecomment-4609578558
I think @nssalian is already working on this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(table): read shredded Parquet variants [iceberg-go]
Posted by "matanper (via GitHub)" <gi...@apache.org>.
matanper commented on PR #1152:
URL: https://github.com/apache/iceberg-go/pull/1152#issuecomment-4610564101
Hi @nssalian first of all thank you for your contributions to this library 🙏
You are assigned to multiple issues at once without noticeable progress, and since I need this feature the natural thing for me, instead of being blocked, is to implement and contribute back to the community in order to help the project moving forward.
At the end both of us wish for this project to succeed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(table): read shredded Parquet variants [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #1152:
URL: https://github.com/apache/iceberg-go/pull/1152#issuecomment-4610782529
@matanper thanks for the PR but this the [general guideline](https://github.com/apache/iceberg-go/blob/main/CONTRIBUTING.md#picking-up-issues)for claiming a issue before woking on the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org