[jira] [Resolved] (CAMEL-10333) (Consumer goes into loop while processing messages) DEBUG Cannot auto-commit offsets now since the coordinator is unknown

["Claus Ibsen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-10333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Claus Ibsen resolved CAMEL-10333.
---------------------------------
    Resolution: Later

Try with latest release as camel-kafka has been improved/update

> (Consumer goes into loop while processing messages) DEBUG Cannot auto-commit offsets now since the coordinator is unknown
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-10333
>                 URL: https://issues.apache.org/jira/browse/CAMEL-10333
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-kafka
>    Affects Versions: 2.17.1
>         Environment: Spring Framework is used for creating consumers, which processes the messages.
>            Reporter: Shashank Madhav
>
> I have a topic which have 300 messages in the queue. When the camel is used for processing the messages with event-based consumer, I am getting following error after all the messages are processed.
> /////error message start
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            DEBUG Cannot auto-commit offsets now since the coordinator is unknown, will retry after backoff
> [ad #0 - KafkaTopic[Audit_logs]] AbstractCoordinator            DEBUG Issuing group metadata request to broker 0
> [ad #0 - KafkaTopic[Audit_logs]] AbstractCoordinator            DEBUG Group metadata response ClientResponse(receivedTimeMs=1473953020273, disconnected=false, request=ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@1edc39d, request=RequestSend(header={api_key=10,api_version=0,correlation_id=9,client_id=consumer-1}, body={group_id=test3}), createdTimeMs=1473953020218, sendTimeMs=1473953020218), responseBody={error_code=0,coordinator={node_id=0,host=<host-name>,port=<port-no>}})
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            ERROR Error ILLEGAL_GENERATION occurred while committing offsets for group test3
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            WARN  Auto offset commit failed: Commit cannot be completed due to group rebalance
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            ERROR Error ILLEGAL_GENERATION occurred while committing offsets for group test3
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            WARN  Auto offset commit failed: 
> ////error message end
>  Following the error, it then starts from the beginning and processes the entire queue again causing issues. The route used for camel is as follows 
> //camel-route start
>         <c:endpoint id="kafkaTest" uri="kafka:${broker.server}?topic=${topics1}&amp;groupId=test3&amp;autoOffsetReset=earliest&amp;consumersCount=1&amp;sessionTimeoutMs=30000&amp;consumerRequestTimeoutMs=40000&amp;keyDeserializer=${broker.keyDeserializer}&amp;valueDeserializer=${broker.valueDeserializer}"/>
> 		
> 		<c:route id="route1">
>             <c:from uri="ref:kafkaTest"/>
>             <c:delay>
>             <c:constant>500</c:constant>
>             </c:delay>
>              <c:to uri="log:input"/> 
>         </c:route>
> //camel-route ends
> Delay is purposely added in the route to create message processing scenario. If the delay is reduced or the sessionTimeoutMs is increased then it works fine, but this is not a feasible solution. Without camel it works fine and messages are processed properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)