[jira] [Work started] (CAMEL-25838) camel-iggy - after one failed poll the consumer polls with a new client that never joined the consumer group, and consumes nothing again until the route restarts

["shashank (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-25838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on CAMEL-25838 started by shashank.
----------------------------------------
> camel-iggy - after one failed poll the consumer polls with a new client that never joined the consumer group, and consumes nothing again until the route restarts
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-25838
>                 URL: https://issues.apache.org/jira/browse/CAMEL-25838
>             Project: Camel
>          Issue Type: Bug
>            Reporter: shashank
>            Assignee: shashank
>            Priority: Major
>
> The iggy consumer joins its consumer group once: {{IggyConsumer.doStart}} borrows a client from its pool and calls {{IggyEndpoint.initializeConsumerGroup}} (which ends with {{joinConsumerGroup}}), then gives the client back (main 20cdb309f7d0). Since CAMEL-25289 (4.23.0) {{IggyFetchRecords.pollMessages}} discards the client of a failed poll ({{invalidateClient}}), so that a broken connection is not reused; the pool then creates a new client with {{IggyClientFactory.create}}, which only logs in.
> The Iggy server keeps the consumer group membership per connection. Its client contract ({{MessageClient.poll_messages}} in apache/iggy) says that polling a consumer group the client is not, or no longer, a member of fails with {{ConsumerGroupMemberNotFound}} (error 5006) "so the caller can rejoin". Camel does not rejoin: after one failed poll (a dropped connection, a timeout, a server restart) every later poll fails with that error, the consumer logs it once per second, and nothing is consumed until the route is restarted. The messages stay in the stream, but the route has stopped for good while it looks started.
> *Fix*: the consumer remembers which pooled clients joined the group (the client used at start joined in {{initializeConsumerGroup}}), makes a client join before its first poll ({{IggyConsumer.joinConsumerGroupIfNeeded}}, the join part of {{initializeConsumerGroup}} is now {{IggyEndpoint.joinConsumerGroup}}), and forgets a client when it is discarded. A join that fails counts as a failed poll: the client is discarded and the next poll tries a new one.
> *Test*: new {{IggyConsumerRejoinGroupTest}} with mocked Iggy clients that behave like the server here: the first client fails its poll ("Connection reset"); a client that did not join gets {{IggyResourceNotFoundException}} with {{CONSUMER_GROUP_MEMBER_NOT_FOUND}}, a member gets one message. On main it fails ({{expected: <[after the failed poll]> but was: <[]> within 20 seconds}}); with the fix it passes. camel-iggy unit tests with the fix: 18 run, 0 failures (the integration tests need a container and were not run).
> *How it was found*: variant analysis of CAMEL-25815 (pattern: a consumer relies on server-side state that belongs to a connection or session, and the reconnect path does not create it again), then the JUnit test. The server rule is taken from the Iggy client contract quoted above; it was not run against an Iggy server.
> *Affected*: main and camel-4.23.x (4.23.0), since CAMEL-25289. camel-4.22.x and older keep the client of a failed poll (the pool then runs out after 8 failures, fixed by CAMEL-25289) and are not affected by this issue (read, not run).
> _Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)