You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "wForget (via GitHub)" <gi...@apache.org> on 2026/10/10 02:33:01 UTC
[PR] [#2810] fix(client): Release destination buffers when decompression fails [uniffle]
wForget opened a new pull request, #2811:
URL: https://github.com/apache/uniffle/pull/2811
### What changes were proposed in this pull request?
Release the destination buffer in `DecompressionWorker` when `codec.decompress()` throws, then rethrow the original failure to the existing exception handler. Successful decompression continues to transfer buffer ownership to the consumer.
### Why are the changes needed?
The worker allocates a destination buffer before decompression. Previously, a decompression failure left the direct buffer without explicit cleanup, relying on JVM reclamation. Repeated failures could therefore cause direct-memory pressure and contribute to YARN killing executor containers for exceeding physical memory limits.
This change addresses cleanup on the failure path; it does not establish the root cause of the decompression errors.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2810] fix(client): Release destination buffers when decompression fails [uniffle]
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston merged PR #2811:
URL: https://github.com/apache/uniffle/pull/2811
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2810] fix(client): Release destination buffers when decompression fails [uniffle]
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on code in PR #2811:
URL: https://github.com/apache/uniffle/pull/2811#discussion_r4236960572
##########
client/src/main/java/org/apache/uniffle/client/impl/DecompressionWorker.java:
##########
@@ -143,7 +144,13 @@ public void add(int batchIndex, ShuffleDataResult shuffleDataResult) {
System.currentTimeMillis() - startBufferAllocation);
long startDecompression = System.currentTimeMillis();
- codec.decompress(buffer, uncompressedLen, dst, 0);
+ try {
+ codec.decompress(buffer, uncompressedLen, dst, 0);
+ } catch (Throwable t) {
+ // Failed destinations are never handed to the consumer for cleanup.
+ RssUtils.releaseByteBuffer(dst);
Review Comment:
nice catch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2810] fix(client): Release destination buffers when decompression fails [uniffle]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2811:
URL: https://github.com/apache/uniffle/pull/2811#issuecomment-6093315959
## Test Results
3 846 files ±0 3 846 suites ±0 7h 41m 12s ⏱️ +56s
1 275 tests ±0 1 264 ✅ ±0 11 💤 ±0 0 ❌ ±0
19 256 runs ±0 19 208 ✅ ±0 48 💤 ±0 0 ❌ ±0
Results for commit e18aeff2. ± Comparison against base commit 45572105.
[test-results]:data:application/gzip;base64,H4sIAEixyWoC/03Myw7CIBCF4VdpWLtghuFSX8YghYTYi6GwMr67tFbs8nwz+V8sxNGv7NoJQ+rSsbXEfN5DSTbHZa6CWmmsVO95+wDU8jdva3FuN0V/e8TnZtAk2DhW4Q18Sks6JJV5z/Yo1TFbtkdumn2z1OBU3fc56pZpirkO5sFYHwLywLVAAuGMFB56TwaIYDDOyXsAwd4fyPfgxhUBAAA=
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org