You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "MonsterChenzhuo (via GitHub)" <gi...@apache.org> on 2026/05/14 09:18:49 UTC
[PR] [core] Validate page size for wide write buffer rows [paimon]
MonsterChenzhuo opened a new pull request, #7854:
URL: https://github.com/apache/paimon/pull/7854
### Purpose
When writing a very wide primary-key table, the write path may fail during commit with an unclear `NegativeArraySizeException`:
```text
26/05/13 09:08:37 WARN TaskSetManager: Lost task 1.0 in stage 6.0 (TID 160) (10.166.23.92 executor 12): java.lang.NegativeArraySizeException
at org.apache.paimon.memory.MemorySegmentUtils.getBytes(MemorySegmentUtils.java:191)
at org.apache.paimon.data.BinarySection.toBytes(BinarySection.java:101)
at org.apache.paimon.format.parquet.writer.ParquetRowDataWriter$StringWriter.writeString(ParquetRowDataWriter.java:299)
at org.apache.paimon.mergetree.SortBufferWriteBuffer.forEach(SortBufferWriteBuffer.java:162)
at org.apache.paimon.mergetree.MergeTreeWriter.flushWriteBuffer(MergeTreeWriter.java:224)
at org.apache.paimon.mergetree.MergeTreeWriter.prepareCommit(MergeTreeWriter.java:253)
.....
```
One root cause is that the table schema is too wide for the configured page-size. BinaryRow requires its fixed-length part to fit into one memory segment. If the fixed-length part is larger than page-size, later write/flush code can read invalid string offsets or lengths and surface a misleading low-level exception.
### Tests
paimon-core/src/test/java/org/apache/paimon/mergetree/SortBufferWriteBufferValidationTest.java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Validate page size for wide write buffer rows [paimon]
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo closed pull request #7854: [core] Validate page size for wide write buffer rows
URL: https://github.com/apache/paimon/pull/7854
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org