[jira] [Resolved] (KAFKA-2725) high level consumer rebalances with auto-commit disabled should throw an exception

["Ismael Juma (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ismael Juma resolved KAFKA-2725.
--------------------------------
    Resolution: Auto Closed

The Scala consumers are no longer supported. If this still applies to the Java consumer, please file a new issue.

> high level consumer rebalances with auto-commit disabled should throw an exception
> ----------------------------------------------------------------------------------
>
>                 Key: KAFKA-2725
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2725
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions: 0.8.2.1
>         Environment: Experienced on Java running in linux
>            Reporter: Cliff Rhyne
>
> Auto-commit is a very resilient mode.  Drops in zookeeper sessions due to JVM garbage collection, network, rebalance or other interference are handled gracefully within the kafka client.
> Systems still can drop due to unexpected gc or network behavior.  My proposal is to handle this drop better when auto-commit is turned off:
> - If a rebalance or similar occur (which cause the offset to get reverted in the client), check and see if the client was assigned back to the same partition or a different one.  If it's the same partition, find the place last consumed (it doesn't do this today for us).  This is to make a graceful recovery.
> - If the partition assignment changes (which can mean duplicate data is getting processed), throw an exception back to the application code.  This lets the application code handle this exception-case with respect to the work it's doing (with might be transactional).  Failing "silently" (yes it's still getting logged) is very dangerous in our situation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)