[jira] [Created] (KAFKA-3553) Issue with getting data of next offset after restarting the consumer and producer is always running

["pooja sadashiv deokar (JIRA)" <ji...@apache.org>]

pooja sadashiv deokar created KAFKA-3553:
--------------------------------------------

             Summary: Issue with getting data of next offset after restarting the consumer and producer is always running
                 Key: KAFKA-3553
                 URL: https://issues.apache.org/jira/browse/KAFKA-3553
             Project: Kafka
          Issue Type: Bug
          Components: consumer, offset manager
    Affects Versions: 0.9.0.0
         Environment: operating system : ubuntu 14.04
using kafka-python
            Reporter: pooja sadashiv deokar
            Assignee: Neha Narkhede


I am pushing data to kafka topic after every 1 second in python. And written consumer to fetch data from topic with consumer timeout as 500 ms and enable_auto_commit as false.
Following are the scripts:

1) prod.py

from kafka import KafkaClient,SimpleConsumer
from kafka.producer import SimpleProducer
import random, time
kafka =  KafkaClient("localhost:9092")

producer = SimpleProducer(kafka)
i=0
while(True):
  l1 = ['Rahul','Narendra','NaMo','ManMohan','Sonia']
  msg = str(i) + ',' + str(random.choice(l1))
  print('putting data : '+msg) 
  producer.send_messages("test", msg)
  i = i + 1
  time.sleep(1)

2) con.py

from kafka import KafkaConsumer
consumer = KafkaConsumer('test',bootstrap_servers=['localhost:9092'],consumer_timeout_ms = 500, enable_auto_commit = False)
for message in consumer:
    print ("%s:%d:%d: key=%s value=%s" % (message.topic, message.partition,
                                          message.offset, message.key,
                                          message.value))
consumer.commit()


My producer is continuously running. And I am  running consumer again when it stops. For first time consumer works well and gets data for me on time. But after first time its taking minimum 1 minute to maximum  5 minutes (or above 5 min also) to get next data. It should give me data after 1 second or maximum after 5 seconds. But its giving problem :( . 
Also 1 thing which I observe is If I wait for starting consumer about 1 min then data comes as expected.

Please correct me if I am going wrong.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)