You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Magnus Edenhill <ma...@edenhill.se> on 2014/11/03 17:15:01 UTC
Re: Can both uncompressed data and compressed data reside in the same
topic (partition)
Hi Zuoning,
since compression is a per-message(set) attribute a topic can have both
compressed and uncompressed messages, as Guozhang says,
and yes, this is supported by both the broker and client (librdkafka in
this case).
Regards,
Magnus
2014-10-31 17:14 GMT+01:00 Zuoning Yin <zu...@gmail.com>:
> Hi Guozhang,
> Just want to double check: does this have some requirement on the Kafka
> version (we are using 0.8.0) or the kafka client (we are using librdkafka)?
>
> Thanks,
> --Zuoning
>
> On Fri, Oct 31, 2014 at 12:09 PM, Zuoning Yin <zu...@gmail.com>
> wrote:
>
> > Thanks so much for the reply, Guozhang!
> >
> > On Fri, Oct 31, 2014 at 12:06 PM, Guozhang Wang <wa...@gmail.com>
> > wrote:
> >
> >> Hi Zuoning,
> >>
> >> Yes, the same topic's partitions can hold both compressed and
> uncompressed
> >> data, and the consumer is able to read them in mixed mode.
> >>
> >> Guozhang
> >>
> >> On Fri, Oct 31, 2014 at 8:40 AM, Zuoning Yin <zu...@gmail.com>
> >> wrote:
> >>
> >> > Hey Guys,
> >> > If at the beginning, I configure the producer to not use
> >> compression
> >> > and produce a number of message to a topic. Then later, I reconfigure
> >> the
> >> > producer to use compression and produce another batch of messages to
> the
> >> > same topic. Can a consumer correctly read all the messages in the
> topic?
> >> > Will the consumer stop reading correctly when it reach the compressed
> >> part?
> >> >
> >> > Thanks so much,
> >> > --Zuoning
> >> >
> >>
> >>
> >>
> >> --
> >> -- Guozhang
> >>
> >
> >
>
Re: Can both uncompressed data and compressed data reside in the same
topic (partition)
Posted by Zuoning Yin <zu...@gmail.com>.
Thanks for the clarification, Magnus!
On Mon, Nov 3, 2014 at 11:15 AM, Magnus Edenhill <ma...@edenhill.se> wrote:
> Hi Zuoning,
> since compression is a per-message(set) attribute a topic can have both
> compressed and uncompressed messages, as Guozhang says,
> and yes, this is supported by both the broker and client (librdkafka in
> this case).
>
> Regards,
> Magnus
>
>
> 2014-10-31 17:14 GMT+01:00 Zuoning Yin <zu...@gmail.com>:
>
> > Hi Guozhang,
> > Just want to double check: does this have some requirement on the
> Kafka
> > version (we are using 0.8.0) or the kafka client (we are using
> librdkafka)?
> >
> > Thanks,
> > --Zuoning
> >
> > On Fri, Oct 31, 2014 at 12:09 PM, Zuoning Yin <zu...@gmail.com>
> > wrote:
> >
> > > Thanks so much for the reply, Guozhang!
> > >
> > > On Fri, Oct 31, 2014 at 12:06 PM, Guozhang Wang <wa...@gmail.com>
> > > wrote:
> > >
> > >> Hi Zuoning,
> > >>
> > >> Yes, the same topic's partitions can hold both compressed and
> > uncompressed
> > >> data, and the consumer is able to read them in mixed mode.
> > >>
> > >> Guozhang
> > >>
> > >> On Fri, Oct 31, 2014 at 8:40 AM, Zuoning Yin <zu...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hey Guys,
> > >> > If at the beginning, I configure the producer to not use
> > >> compression
> > >> > and produce a number of message to a topic. Then later, I
> reconfigure
> > >> the
> > >> > producer to use compression and produce another batch of messages to
> > the
> > >> > same topic. Can a consumer correctly read all the messages in the
> > topic?
> > >> > Will the consumer stop reading correctly when it reach the
> compressed
> > >> part?
> > >> >
> > >> > Thanks so much,
> > >> > --Zuoning
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> -- Guozhang
> > >>
> > >
> > >
> >
>