[I] [Bug] Stale-client eviction is local-only: the cluster subscription is never deleted (eventmesh)

["qqeasonchen (via GitHub)" <gi...@apache.org>]

qqeasonchen opened a new issue, #5419:
URL: https://github.com/apache/eventmesh/issues/5419

   ### Describe the bug
   
   `UniIngressService.cleanupStaleClients(thresholdMs)` (the §13.6.5 reaper, run every 60 s by `UniRuntime`) evicts a client from the **local** subscription manager only:
   
   ```java
   for (String cid : pushService.getStaleClientIds(thresholdMs)) {
       removed += subscriptionManager.unsubscribeByClient(cid);
       pushService.removeClient(cid);
       ...
   }
   ```
   
   The cluster-wide record `/em/subs/<topic>/<clientId>` is never deleted. Every peer keeps mirroring that entry, so a client that died without unsubscribing (SIGKILL, network partition, suspended laptop) becomes a **permanent ghost subscriber**:
   
   - peers keep forwarding its events to an instance whose push buffer was already dropped → the events are lost after the retry budget;
   - the topic stays in `activeTopics()` cluster-wide, so the cluster keeps polling/assigning partitions for a subscriber that no longer exists;
   - nothing expires it, because the reaper only looks at the local manager.
   
   This is exactly the case the reaper exists for, so being local-only defeats its purpose.
   
   ### Reproduction
   
   1. Start instance A; subscribe from a **separate process** (`clientId=c-crash`), then `kill -9` that process (no `/events/unsubscribe`).
   2. Wait for the reaper (or call `ingress().cleanupStaleClients(1L)` directly).
   3. Read `/em/subs/<topic>/c-crash` from Nacos: still present after the local subscription is gone.
   
   E2E: `ClusterBalanceAndResilienceE2EIntegrationTest#crashedClientSubscriptionIsEvictedFromClusterMeta` (child JVM client + SIGKILL + direct reaper drive).
   
   ### Root cause
   
   The reaper bypasses `UniIngressService.unsubscribeByClient(...)` and calls the inner `SubscriptionManager` directly, skipping the `cluster.unsubscribe(topic, clientId)` propagation (and, before this fix, also duplicating the push-buffer cleanup).
   
   ### Fix
   
   `eventmesh-runtime/.../ingress/UniIngressService.java` — route eviction through `unsubscribeByClient(cid)` so local removal, cluster Meta deletion and push-buffer cleanup happen together, with a Javadoc explaining why bypassing it leaks ghosts.
   
   ### Verification
   
   - E2E `crashedClientSubscriptionIsEvictedFromClusterMeta`: the crashed client's `/em/subs` entry disappears (previously it never did).
   - Existing `UniIngressServiceTest` cases (incl. the two `unsubscribeByClient` paths) still pass.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: issues-help@eventmesh.apache.org

Re: [I] [Bug] Stale-client eviction is local-only: the cluster subscription is never deleted (eventmesh)

["qqeasonchen (via GitHub)" <gi...@apache.org>]

qqeasonchen closed issue #5419: [Bug] Stale-client eviction is local-only: the cluster subscription is never deleted
URL: https://github.com/apache/eventmesh/issues/5419


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: issues-help@eventmesh.apache.org