You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Parag Shah <ps...@proofpoint.com> on 2014/11/17 23:51:21 UTC

Large Messages

Hi all,

    I am fairly new to Kafka and need some general advice on handling large messages. My message sizes range from 20k – 10M, with most of my messages upto 250K, with some messages ranging in sizes up to 20M.

    I am wondering if there is some kind of streaming support in Kafka. I have read the recommendation about manually chunking the message and reassembling it at the processing end(server). I was wondering if there is a better solution for this concern.

Regards
Parag

Re: Large Messages

Posted by Jagat Singh <ja...@gmail.com>.

You can see the compression options.

To change the size the property to look for is related to
message.max.bytes.

On Tue, Nov 18, 2014 at 9:51 AM, Parag Shah <ps...@proofpoint.com> wrote:

> Hi all,
>
>     I am fairly new to Kafka and need some general advice on handling
> large messages. My message sizes range from 20k – 10M, with most of my
> messages upto 250K, with some messages ranging in sizes up to 20M.
>
>     I am wondering if there is some kind of streaming support in Kafka. I
> have read the recommendation about manually chunking the message and
> reassembling it at the processing end(server). I was wondering if there is
> a better solution for this concern.
>
> Regards
> Parag
>