[jira] [Resolved] (KAFKA-3793) Kafka Python Consumer library messages gets truncated

["Manikumar (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Manikumar resolved KAFKA-3793.
------------------------------
    Resolution: Cannot Reproduce

{color:#000000}Closing inactive issue. Please reopen if you think the issue still exists{color}
 

> Kafka Python Consumer library messages gets truncated
> -----------------------------------------------------
>
>                 Key: KAFKA-3793
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3793
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Rahul
>            Priority: Major
>
> Snippet code is below:
> from kafka import KafkaConsumer
> from kafka.client import KafkaClient
> from kafka.consumer import SimpleConsumer
> consumer = KafkaConsumer('eventdetails_ingestion' , group_id='1',bootstrap_servers=‘xxxx:9092', max_partition_fetch_bytes=10555555)
> for msg in consumer:
>    try:
>        jValue = json.loads(str(msg.value))
>   except ValueError:
>        fileErr.write(str(msg.value)+"\n")
> Steps:
> We send/produce large sets of messages to Kafka of around 20 to 30 KB size each messages in JSON format and producing around 200 messages / sec for 1 hour duration. We have 3 Kafka Brokers running and I am trying to consume the messages from these 3 Kafka Brokers from the same topic using the above code. The problem is that sometimes some of the messages gets truncated, I am not sure why it happen ?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)