You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "lxy-9602 (via GitHub)" <gi...@apache.org> on 2026/06/23 10:35:12 UTC
[PR] [common] Add shared-shredding write helpers for MAP columns [paimon]
lxy-9602 opened a new pull request, #8332:
URL: https://github.com/apache/paimon/pull/8332
### Purpose
This is a sub-PR for MAP shared-shredding write support. The full shared-shredding write path will be wired in later PRs.
- Add `MapSharedShreddingContext` to maintain cross-file shared-shredding state and compute the next physical column width `K`.
- Add `MapSharedShreddingFieldDict` to assign stable field ids for MAP keys within a shared-shredding field.
- Add `MapSharedShreddingColumnAllocator` to allocate MAP entries into physical shared columns for each row.
- This is currently a simple temporary implementation based on row order.
- A later version will support a more sophisticated LRU-style allocation strategy.
- Add `MapSharedShreddingRowConverter` to lazily wrap logical `InternalRow` records and convert shared-shredding MAP fields into their physical representation only when those fields are accessed.
Related design:
https://cwiki.apache.org/confluence/display/PAIMON/PIP-43%3A+Columnar+Storage+Optimization+for+MAP+Type+in+Paimon
### Tests
```bash
mvn -pl paimon-common -am -Pfast-build -DfailIfNoTests=false \
-Dtest=MapSharedShreddingColumnAllocatorTest,MapSharedShreddingContextTest,MapSharedShreddingFieldDictTest,MapSharedShreddingRowConverterTest test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common] Add shared-shredding write helpers for MAP columns [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8332:
URL: https://github.com/apache/paimon/pull/8332#issuecomment-4779863906
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common] Add shared-shredding write helpers for MAP columns [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #8332:
URL: https://github.com/apache/paimon/pull/8332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common] Add shared-shredding write helpers for MAP columns [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8332:
URL: https://github.com/apache/paimon/pull/8332#issuecomment-4778639648
Looks no problem, wait CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org