[jira] [Commented] (APEXMALHAR-2518) Kafka input operator stops reading tuples when there is a UNKNOWN_MEMBER_ID error during committed offset processing

["Chaitanya (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16079967#comment-16079967 ] 

Chaitanya commented on APEXMALHAR-2518:
---------------------------------------

[~pramod.immaneni] Could you please share the complete stack trace ? 
Also, Please go through the below links, looks like a similar issue:
* https://issues.apache.org/jira/browse/KAFKA-2985
* https://stackoverflow.com/questions/38394662/error-unknown-member-id-occurred-while-committing-offsets-for-group-xxx

> Kafka input operator stops reading tuples when there is a UNKNOWN_MEMBER_ID error during committed offset processing
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: APEXMALHAR-2518
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2518
>             Project: Apache Apex Malhar
>          Issue Type: Bug
>            Reporter: Pramod Immaneni
>            Assignee: Pramod Immaneni
>
> Kafka 0.9 operator stores offsets that are completely processed and no longer needed (committed offsets) back in kafka. It does so by making a kafka API call. If the response from kafka server to this call comes back with an UNKNOWN_MEMBER_ID error, it results in the kafka consumer state changing to needing partition re-assignment and no further messages are returned by the consumer. There are a couple of other errors that result in the same state including when rebalance is in progress.
> What exactly caused this error is not known but the following is the likely reason due to the conditions surrounding the application. When the operator has temporarily stalled due to back-pressure exerted by the slow downstream, it will eventually stall the operator kafka consumer thread that is reading messages from kafka. This will result in the thread not making any kafka consumer API calls and it will result in no heartbeats being sent to kafka server. This can cause the server to evict the consumer after a timeout period. This could have been the cause for the UNKNOWN_MEMBER_ID error.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)