You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "yugan95 (via GitHub)" <gi...@apache.org> on 2026/04/10 03:18:30 UTC
[PR] [core] Fix OOM when writing/compacting table with large records [paimon]
yugan95 opened a new pull request, #7621:
URL: https://github.com/apache/paimon/pull/7621
### Purpose
Linked issue: close #7620
Fix OOM when writing table with large records (100MB+) due to unbounded buffer growth in sort, merge and compaction paths.
Heap dump analysis identified four independent root causes:
**1. Sort path — `RowHelper` internal buffer never shrinks**
`RowHelper.reuseWriter` grows its internal `MemorySegment` list for large records, but `BinaryRowWriter.reset()` only resets the cursor without releasing oversized segments. Additionally, `InternalRowSerializer.serialize()` can exit via `EOFException` (a normal signal when the sort buffer is full), skipping any cleanup of the bloated buffer.
**2. Merge path — `BinaryRowSerializer.deserialize(reuse)` only grows, never shrinks**
Each merge channel holds a `BinaryRow` reuse instance. When a large record is deserialized, the backing `MemorySegment` grows to fit it but is never shrunk for subsequent small records. With `max-num-file-handles` (default 128) channels each retaining a 100MB+ buffer, memory usage explodes.
**3. Compaction read path — `HeapBytesVector.reserveBytes()` integer overflow**
`reserveBytes()` computes `newCapacity * 2` using plain multiplication. When `newCapacity` exceeds ~1.07 billion bytes, this overflows `Integer.MAX_VALUE`, causing `NegativeArraySizeException` or silent data corruption.
**4. Parquet write — statistics and page-size-check config not passed through**
`RowDataParquetBuilder` does not pass through `parquet.statistics.truncate.length`, `parquet.columnindex.truncate.length`, `parquet.page.size.row.check.min`, and `parquet.page.size.row.check.max`. Without these, users cannot tune Parquet behavior for large-record scenarios, leading to multi-GB pages and bloated footers.
#### Changes
1. **`RowHelper`**: add `resetIfTooLarge()` — release internal buffer when segments exceed 4MB
2. **`InternalRowSerializer`**: call `resetIfTooLarge()` in `finally` block of `serialize()` and `serializeToPages()` to handle `EOFException` exit path
3. **`BinaryRowSerializer`**: add shrink logic in `deserialize(reuse)` — reallocate when existing buffer > 4MB threshold
4. **`HeapBytesVector`**: use bit-shift (`<< 1`) instead of `* 2`, cap at `MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8`, throw clear error on overflow
5. **`RowDataParquetBuilder`**: pass through `statistics.truncate.length`, `columnindex.truncate.length`, `min-row-count-for-page-size-check`, `max-row-count-for-page-size-check` from config
### Tests
- `RowHelperTest` — validates `resetIfTooLarge()` releases oversized buffers (> 4MB) and preserves small ones
- `BinaryRowSerializerShrinkTest` — validates `deserialize(reuse)` shrinks oversized buffers and preserves small ones
- `HeapBytesVectorReserveBytesTest` — validates overflow-safe `reserveBytes()` growth and data correctness
### API and Format
N/A — no public API or format changes.
### Documentation
N/A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common][format] Fix OOM when writing/compacting table with large records [paimon]
Posted by "yugan95 (via GitHub)" <gi...@apache.org>.
yugan95 commented on PR #7621:
URL: https://github.com/apache/paimon/pull/7621#issuecomment-4447284916
@leaves12138 Thanks for the review! All three points are addressed in the updated commit:
1. Exact capacity when doubling exceeds max
When requiredCapacity > MAX_ARRAY_SIZE >> 1, the code now allocates the exact required capacity instead of MAX_ARRAY_SIZE. This avoids the unnecessary ~2GB allocation for a ~1.1GB request.
2. Long arithmetic for required capacity
reserveBytes now accepts long. The callers compute required capacity with long arithmetic:
putByteArray: (long) bytesAppended + length
fill: (long) start.length * value.length
If the result exceeds MAX_ARRAY_SIZE, a clear RuntimeException is thrown before any downcast to int, so an int overflow can never silently bypass the guard.
3. Extracted helper + edge-case tests
The capacity-growth logic is extracted into a package-private static int calculateNewBytesCapacity(long) method, tested directly with 7 new cases covering:
Normal doubling for small values
Boundary at exactly MAX_ARRAY_SIZE >> 1 (still doubles)
Just above MAX_ARRAY_SIZE >> 1 (returns exact capacity, not MAX_ARRAY_SIZE)
Exactly MAX_ARRAY_SIZE (returns exact)
Above MAX_ARRAY_SIZE (throws)
Simulated int-overflow values via long (throws)
No huge allocations needed — all edge cases are tested purely through the helper.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common][format] Fix OOM when writing/compacting table with large records [paimon]
Posted by "LsomeYeah (via GitHub)" <gi...@apache.org>.
LsomeYeah commented on PR #7621:
URL: https://github.com/apache/paimon/pull/7621#issuecomment-4517251218
Thanks for the detailed fix! The heap-dump-based root cause breakdown is very clear. The HeapBytesVector overflow fix and Parquet config pass-through both look great.
We have a few concerns about the RowHelper / BinaryRowSerializer parts though — would love to hear your thoughts:
1. 4MB threshold lacks a basis. Data patterns vary a lot across workloads — a table with uniform 1MB records, one with mixed 1KB/100MB records, and one with sustained 5MB records all behave very differently around a fixed 4MB cutoff.
2. Reuse mechanism effectively defeated near the threshold. For sustained 5–10MB records, every call triggers release-and-rebuild or shrink-and-reallocate, paying allocation cost on every record. This defeats the purpose of these reuse buffers.
3. Hot-path behavior change with no opt-out. The finally { resetIfTooLarge() } and the new shrink branch change default behavior for every user on upgrade, not only those experiencing OOM.
4. Prior art. Worth checking how Spark's UnsafeExternalSorter or similar engines handle reuse-buffer growth — there may be a more principled heuristic (hysteresis, memory-pressure-based release) that avoids the thrash pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common][format] Fix OOM when writing/compacting table with large records [paimon]
Posted by "yugan95 (via GitHub)" <gi...@apache.org>.
yugan95 commented on PR #7621:
URL: https://github.com/apache/paimon/pull/7621#issuecomment-4533293854
@JingsongLi Thanks for the detailed per-fix comments! Addressing each point:
**1. RowHelper — `getSegments().size()` semantics**
Good question. `AbstractBinaryWriter.getSegments()` returns a single `MemorySegment`
(not a list/array — the method name is slightly misleading), and `MemorySegment.size()`
returns the byte size of the underlying buffer. So the comparison
`reuseWriter.getSegments().size() > 4MB` correctly checks byte count, and the threshold
matches.
I've also added a hysteresis guard per @LsomeYeah's feedback: `resetIfTooLarge()` now
additionally checks `reuseRow.getSizeInBytes() < threshold`, so it only releases when
the last written record was small. This avoids thrashing for sustained large-record
workloads while still reclaiming memory when the workload transitions back.
**Configurability:** I think a fixed 4MB is sufficient for the initial fix. If operators
with unusual memory profiles need tuning, we can add a system property in a follow-up.
Keeping the initial change minimal reduces merge risk.
**2. BinaryRowSerializer shrink** — now improved with hysteresis. The `length <
threshold` guard means consistent large-record workloads reuse the buffer without
reallocation, addressing the "allocating every time" concern you noted.
**3. HeapBytesVector** — addressed in commit f1954ffd.
**4. Parquet config pass-through** — moved to #7956.
**5. Performance overhead** — the check is 1 null check + 2 int comparisons, all on hot
fields already in cache. For small-record workloads (buffer < 4MB), `resetIfTooLarge()`
short-circuits at the first size check — no branch taken, no allocation. This is
comparable to an array bounds check and will be inlined by the JIT. Happy to run a JMH
benchmark if the team wants hard numbers, but I don't expect a measurable delta.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common][format] Fix OOM when writing/compacting table with large records [paimon]
Posted by "yugan95 (via GitHub)" <gi...@apache.org>.
yugan95 commented on PR #7621:
URL: https://github.com/apache/paimon/pull/7621#issuecomment-4533242443
@LsomeYeah Thanks for the thoughtful review! All four points are addressed in the
updated commit:
**1. 4MB threshold basis**
The threshold is derived from the production scenario that surfaced this bug: 256
buckets × 4MB = 1GB baseline, which leaves reasonable headroom in a typical 4–8GB
TaskManager heap. 4MB is high enough to avoid interfering with normal workloads (most
records are well under 1MB) while preventing the pathological accumulation we saw in
heap dumps (256 × 100MB+ = tens of GB).
**2. Thrashing near the threshold — fixed with hysteresis**
Good catch — this was a real gap. I've updated both `RowHelper.resetIfTooLarge()` and
`BinaryRowSerializer.deserialize(reuse)` with a hysteresis guard:
- `RowHelper`: only releases when `buffer > threshold && lastRecord < threshold`
- `BinaryRowSerializer`: only shrinks when `buffer > threshold && currentRecordLength <
threshold`
This means:
- **Sustained large records (5–10MB):** buffer is retained, no thrashing
- **Occasional large record → back to small records:** buffer is released, OOM
protection kicks in
- **Normal small records (< 4MB):** buffer never exceeds threshold, the check is a
complete no-op
Tests updated to verify the hysteresis behavior in both classes.
**3. Hot-path overhead**
With hysteresis, the overhead on the hot path is 1 null check + 2 int comparisons — all
on fields already in L1 cache from the preceding serialize(). For normal-sized records
the buffer never reaches 4MB, so `resetIfTooLarge()` short-circuits at the size check.
The JIT will inline this. No branch is taken and no allocation occurs.
**4. Prior art**
The hysteresis approach here is inspired by the same principle behind Spark's
`UnsafeExternalSorter` — retain capacity when the workload demands it, release when it
doesn't. A full memory-pressure-based release (integrating with Paimon's managed memory
pool) would be a larger scope change; the simple hysteresis guard covers the production
failure mode while keeping the change minimal and safe.
Also, the Parquet config pass-through has been moved to #7956.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common][format] Fix OOM when writing/compacting table with large records [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #7621:
URL: https://github.com/apache/paimon/pull/7621#issuecomment-4233589072
cc @LsomeYeah to take a review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common] Fix OOM when writing/compacting table with large records [paimon]
Posted by "yugan95 (via GitHub)" <gi...@apache.org>.
yugan95 closed pull request #7621: [common] Fix OOM when writing/compacting table with large records
URL: https://github.com/apache/paimon/pull/7621
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix OOM when writing/compacting table with large records [paimon]
Posted by "yugan95 (via GitHub)" <gi...@apache.org>.
yugan95 commented on PR #7621:
URL: https://github.com/apache/paimon/pull/7621#issuecomment-4221345491
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common] Fix OOM when writing/compacting table with large records [paimon]
Posted by "yugan95 (via GitHub)" <gi...@apache.org>.
yugan95 commented on PR #7621:
URL: https://github.com/apache/paimon/pull/7621#issuecomment-4645273715
@JingsongLi Good suggestion — I've split this into three independent PRs, one per root cause:
1. **HeapBytesVector `reserveBytes()` integer overflow** — promote to `long` arithmetic, extract `calculateNewBytesCapacity(long)` helper, cap at `MAX_ARRAY_SIZE`. #8158
2. **RowHelper / InternalRowSerializer buffer release** — add `resetIfTooLarge()` with hysteresis in `finally` block to handle EOFException exit path. #8159
3. **BinaryRowSerializer reuse buffer shrink** — add hysteresis shrink in `deserialize(reuse)` to reclaim oversized buffers when workload transitions to small records. #8160
The Parquet config pass-through is already merged in #7956.
Each PR is self-contained with its own tests and no cross-dependencies. Will link them here once submitted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common] Fix OOM when writing/compacting table with large records [paimon]
Posted by "yugan95 (via GitHub)" <gi...@apache.org>.
yugan95 commented on PR #7621:
URL: https://github.com/apache/paimon/pull/7621#issuecomment-4678685072
All fixes have been split into independent PRs and merged:
1. #7956 — Parquet config pass-through
2. #8158 — HeapBytesVector reserveBytes() integer overflow
3. #8159 — RowHelper internal buffer never shrinking
4. #8160 — BinaryRowSerializer reuse buffer never shrinking
Closing this PR. Thanks for the reviews!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [common] Fix OOM when writing/compacting table with large records [paimon]
Posted by "yugan95 (via GitHub)" <gi...@apache.org>.
yugan95 commented on PR #7621:
URL: https://github.com/apache/paimon/pull/7621#issuecomment-4678707083
All fixes have been split into independent PRs and merged:
1. #7956 — Parquet config pass-through
2. #8158 — HeapBytesVector reserveBytes() integer overflow
3. #8159 — RowHelper internal buffer never shrinking
4. #8160 — BinaryRowSerializer reuse buffer never shrinking
Closing this PR. Thanks for the reviews!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org