[I] [Bug] Writer batches can hang forever after synchronous RPC encoding failure [fluss]

["gyang94 (via GitHub)" <gi...@apache.org>]

gyang94 opened a new issue, #4018:
URL: https://github.com/apache/fluss/issues/4018

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
   
   
   ### Fluss version
   
   main (development)
   
   ### Please describe the bug 🐞
   
   A synchronous failure while encoding/sending a write RPC can leave drained batches permanently incomplete.
   
   Observed stack trace:
   
   ```
   java.lang.OutOfMemoryError: Direct buffer memory
     at MessageCodec.encodeRequest
     at ServerConnection$InflightRequest.toByteBuf
     at ServerConnection.doSend
     at NettyClient.sendRequest
     at Sender.sendPutKvRequestAndHandleResponse
   ```
   
   Root cause:
   
   1. `Sender.sendWriteData` drains batches and adds them to `inFlightBatches` before sending.
   2. `ServerConnection.doSend` registers the RPC, but its encoding cleanup catches only `Exception`. `OutOfMemoryError` escapes synchronously.
   3. Since `gateway.putKv(request)` throws before returning a future, Sender never registers its `whenComplete` callback.
   4. The outer Sender loop catches `Throwable`, logs it, and continues without completing, re-enqueuing, or deallocating those batches.
   
   The affected batches retain writer-buffer pages with no final state and incomplete request futures. With idempotence enabled, later requests receive `OUT_OF_ORDER_SEQUENCE_EXCEPTION` because the missing sequence was never sent or re-enqueued.
   
   In the observed case, almost all pages in a 64 MiB writer buffer remained occupied. The Flink sink thread then blocked forever in `LazyMemorySegmentPool.waitForSegment`, while the Sender still reported itself as running.
   
   Expected behavior: every drained batch must either have a completable RPC future, be re-enqueued, complete successfully, or fail and release its memory. Fatal errors should fail the writer/task rather than only being logged.
   
   ### Solution
   
   - Make `ServerConnection.doSend` remove the RPC registration for every encoding failure, including fatal `Throwable`.
   - Ensure synchronous failures from `putKv` and `produceLog` cannot orphan Sender batches.
   - Stop the Sender and fail/abort incomplete batches on fatal errors.
   - Add tests that inject `OutOfMemoryError` during RPC encoding and assert that RPC/Sender in-flight entries are removed, futures fail, and writer-buffer pages are returned.
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Writer batches can hang forever after synchronous RPC encoding failure [fluss]

["leekeiabstraction (via GitHub)" <gi...@apache.org>]

leekeiabstraction closed issue #4018: [Bug] Writer batches can hang forever after synchronous RPC encoding failure
URL: https://github.com/apache/fluss/issues/4018


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org