[I] [Bug] Proxy telemetry dispatch failure leaves relay futures pending until timeout [rocketmq]

["Aias00 (via GitHub)" <gi...@apache.org>]

Aias00 opened a new issue, #10766:
URL: https://github.com/apache/rocketmq/issues/10766

   ### Before Creating the Bug Report
   
   - [x] I have confirmed this problem is not already covered by an existing issue or open pull request.
   
   ### Runtime platform environment
   
   RocketMQ Proxy with gRPC telemetry clients.
   
   ### RocketMQ version
   
   develop branch
   
   ### JDK Version
   
   N/A
   
   ### Describe the Bug
   
   `GrpcClientChannel` registers response futures before writing telemetry commands for remote diagnostic/admin operations. If the telemetry observer is already unavailable, or if `observer.onNext()` fails before the command is delivered, `writeTelemetryCommand()` only logs and returns.
   
   The registered nonce remains in `GrpcChannelManager.resultNonceFutureMap` until the periodic timeout scanner eventually completes it with `SYSTEM_BUSY`.
   
   Affected flows include:
   
   - `processGetConsumerRunningInfo()` when thread-stack collection is requested
   - `processConsumeMessageDirectly()`
   
   ### Steps to Reproduce
   
   1. Register a gRPC client channel.
   2. Trigger a diagnostic command such as consumer running info with `jstackEnable=true`.
   3. Make the telemetry observer unavailable or make `observer.onNext()` throw.
   4. Observe that the response future remains pending until the relay timeout scan runs.
   
   ### What Did You Expect to See?
   
   When telemetry dispatch fails locally, the registered nonce should be removed and the waiting response future should complete immediately with an error result.
   
   ### What Did You See Instead?
   
   The response future stays pending until the periodic timeout cleanup fires.
   
   ### Additional Context
   
   This is related to the Proxy Admin/runtime diagnostics work: admin calls should fail fast when the gRPC telemetry stream is not writable, instead of waiting for the full relay timeout.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Proxy telemetry dispatch failure leaves relay futures pending until timeout [rocketmq]

["RockteMQ-AI (via GitHub)" <gi...@apache.org>]

RockteMQ-AI commented on issue #10766:
URL: https://github.com/apache/rocketmq/issues/10766#issuecomment-5162893483

   **Issue Evaluation**
   
   Category: `type/bug` | Status: **Confirmed**
   
   The reported issue has been verified against the current codebase.
   
   **Root Cause:** `GrpcClientChannel.writeTelemetryCommand()` catches `StatusRuntimeException` / `IllegalStateException` from `observer.onNext()` and only logs + clears the observer. However, callers (e.g. `processGetConsumerRunningInfo()`, `processConsumeMessageDirectly()`) register a nonce in `GrpcChannelManager.resultNonceFutureMap` before dispatching. When the write fails, the registered future is never completed and remains pending until the periodic timeout scanner fires with `SYSTEM_BUSY`.
   
   **Impact:** Proxy admin/diagnostic calls (consumer running info, direct consume) hang until relay timeout when the gRPC telemetry stream is broken.
   
   **Severity:** medium
   
   **Suggested fix:** In `writeTelemetryCommand()`, when the write fails, the method should signal the failure back to the caller so the registered nonce/future can be cleaned up immediately, or the cleanup should happen within the catch block.
   
   An automated fix proposal will be generated. Reply `/approve` to proceed with PR generation.
   
   ---
   *Automated evaluation by github-manager-bot*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org