You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2026/09/26 17:45:08 UTC
[PR] [core] Close writer on failure in DataEvolutionNormalCompactTask [paimon]
LuciferYang opened a new pull request, #10224:
URL: https://github.com/apache/paimon/pull/10224
### Purpose
`DataEvolutionNormalCompactTask` closed its writer and store write only on the success path. When the compact loop threw (the missing-rows or extra-rows checks after the reader returned fewer or more rows than the plan expected, or an IO error from `writer.write` / `prepareCommit`), the exception propagated before the close ran, so the writer's open files and write buffers leaked; repeated failures could exhaust file handles.
The closes now live in a `finally` block, each guarded by its own warn-and-continue so a close failure cannot mask the original exception, matching `DataEvolutionMaterializeDeletionCompactTask`. Output-range planning is hoisted above resource creation as well, so a planning failure no longer leaks an already-created writer.
### Tests
Added `DataEvolutionNormalCompactTaskTest.testStoreWriteClosedWhenCompactionFails`: it forces the compact loop to fail after the writer is created and asserts both the writer and the store write are still closed. The test fails against the current code and passes with this change.
### API and Format
No.
### Documentation
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Close writer on failure in DataEvolutionNormalCompactTask [paimon]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #10224:
URL: https://github.com/apache/paimon/pull/10224#issuecomment-5945007066
Closing. The leak only fires on the "Missing rows in normal compaction input" check, which a real compaction never reaches. The test forces it by inflating rowCount on a spy and then only asserts close() was called. There is no table-level symptom (orphan file, failed-then-retried commit) that separates master from the patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Close writer on failure in DataEvolutionNormalCompactTask [paimon]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang closed pull request #10224: [core] Close writer on failure in DataEvolutionNormalCompactTask
URL: https://github.com/apache/paimon/pull/10224
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org