You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by Guozhang Wang <wa...@gmail.com> on 2014/10/28 00:27:39 UTC

[DISCUSSION] Nested compression in Kafka?

Hello folks,

I came across this "testComplexCompressDecompress" in
kafka.message.MessageCompressionTest while I'm working some consumer
decompression optimization. This test checks if nested compression is
supported.

I remember vaguely that some time ago we decide not to support nested
compression at Kafka, and in the new producer's MemoryRecords I also make
this assumption in this iterator implementation. Is that still the case? If
yes shall we remove this test case?

-- Guozhang

Re: [DISCUSSION] Nested compression in Kafka?

Posted by Guozhang Wang <wa...@gmail.com>.

Thanks Jun and Neha. I will go ahead and file the JIRA to remove this test
case then.

Re: [DISCUSSION] Nested compression in Kafka?

Posted by Neha Narkhede <ne...@gmail.com>.

+1 on not having more than one level of compression.

On Thu, Oct 30, 2014 at 1:30 PM, Jun Rao <ju...@gmail.com> wrote:

> Yes, I agree that we don't need to support more than 1 level of nesting for
> the compressed messages.
>
> Thanks,
>
> Jun
>
> On Mon, Oct 27, 2014 at 4:27 PM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > Hello folks,
> >
> > I came across this "testComplexCompressDecompress" in
> > kafka.message.MessageCompressionTest while I'm working some consumer
> > decompression optimization. This test checks if nested compression is
> > supported.
> >
> > I remember vaguely that some time ago we decide not to support nested
> > compression at Kafka, and in the new producer's MemoryRecords I also make
> > this assumption in this iterator implementation. Is that still the case?
> If
> > yes shall we remove this test case?
> >
> > -- Guozhang
> >
>

Re: [DISCUSSION] Nested compression in Kafka?

Posted by Jun Rao <ju...@gmail.com>.

Yes, I agree that we don't need to support more than 1 level of nesting for
the compressed messages.

Thanks,

Jun

On Mon, Oct 27, 2014 at 4:27 PM, Guozhang Wang <wa...@gmail.com> wrote:

> Hello folks,
>
> I came across this "testComplexCompressDecompress" in
> kafka.message.MessageCompressionTest while I'm working some consumer
> decompression optimization. This test checks if nested compression is
> supported.
>
> I remember vaguely that some time ago we decide not to support nested
> compression at Kafka, and in the new producer's MemoryRecords I also make
> this assumption in this iterator implementation. Is that still the case? If
> yes shall we remove this test case?
>
> -- Guozhang
>