[jira] [Created] (KAFKA-6822) Kafka Consumer 0.10.2.1 can not normally read data from Kafka 1.0.0

["Phil Mikhailov (JIRA)" <ji...@apache.org>]

Phil Mikhailov created KAFKA-6822:
-------------------------------------

             Summary: Kafka Consumer 0.10.2.1 can not normally read data from Kafka 1.0.0
                 Key: KAFKA-6822
                 URL: https://issues.apache.org/jira/browse/KAFKA-6822
             Project: Kafka
          Issue Type: Bug
          Components: consumer
            Reporter: Phil Mikhailov


Kafka consumer 0.10.2.1 calculates offsets like this:
 Fetcher:524
{code:java}
long nextOffset = partRecords.get(partRecords.size() - 1).offset() + 1;
{code}
Get the latest offset from records (which were got from {{poll}}) plus 1.
 So the next offset is estimated.

In Kafka 1.0.0 otherwise broker explicitly provides the next offset to fetch:
{code:java}
long nextOffset = partitionRecords.nextFetchOffset;
{code}
It returns the actual next offset but not estimated.

This is the reason why {{ProcessorStateManager.restoreActiveState:245}} keeps spinning in consumer loop 'cause this condition will never happen:
{code:java}
       } else if (restoreConsumer.position(storePartition) == endOffset) {
           break;
       }
{code}
 

 

 

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)