[jira] [Created] (KAFKA-3793) Kafka Python Consumer library messages gets truncated

["Rahul (JIRA)" <ji...@apache.org>]

Rahul created KAFKA-3793:
----------------------------

             Summary: Kafka Python Consumer library messages gets truncated
                 Key: KAFKA-3793
                 URL: https://issues.apache.org/jira/browse/KAFKA-3793
             Project: Kafka
          Issue Type: Bug
            Reporter: Rahul


Snippet code is below:
from kafka import KafkaConsumer
from kafka.client import KafkaClient
from kafka.consumer import SimpleConsumer

consumer = KafkaConsumer('eventdetails_ingestion' , group_id='1',bootstrap_servers=‘xxxx:9092', max_partition_fetch_bytes=10555555)
for msg in consumer:
   try:
       jValue = json.loads(str(msg.value))
  except ValueError:
       fileErr.write(str(msg.value)+"\n")

Steps:
We send/produce large sets of messages to Kafka of around 20 to 30 KB size each messages in JSON format and producing around 200 messages / sec for 1 hour duration. We have 3 Kafka Brokers running and I am trying to consume the messages from these 3 Kafka Brokers from the same topic using the above code. The problem is that sometimes some of the messages gets truncated, I am not sure why it happen ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)