You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Marc Labbe <mr...@gmail.com> on 2015/03/13 14:45:15 UTC
Log cleaner patch (KAFKA-1641) on 0.8.2.1
Hello,
we're often seeing log cleaner exceptions reported in KAFKA-1641 and I'd
like to know if it's safe to apply the patch from that issue resolution to
0.8.2.1?
Reference: https://issues.apache.org/jira/browse/KAFKA-1641
Also there are 2 patches in there, I suppose I should be using only the
latest of the two.
thanks!
marc
Re: Log cleaner patch (KAFKA-1641) on 0.8.2.1
Posted by Mayuresh Gharat <gh...@gmail.com>.
I suppose that the patch for KAFKA-1641 had a fix for this issue.
Also it might be worth looking at Kafka-1755.
Thanks,
Mayuresh
On Fri, Mar 13, 2015 at 8:13 AM, Marc Labbe <mr...@gmail.com> wrote:
> No exactly, the topics are compacted but messages are not compressed.
>
> I get the exact same error though. Any other options I should consider?
> We're on 0.8.2.0 and we also had this on 0.8.1.1 before.
>
> marc
>
> On Fri, Mar 13, 2015 at 10:47 AM, Jun Rao <ju...@confluent.io> wrote:
>
> > Did you get into that issue for the same reason as in the jira, i.e.,
> > somehow compressed messages were sent to the compact topics?
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Mar 13, 2015 at 6:45 AM, Marc Labbe <mr...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > we're often seeing log cleaner exceptions reported in KAFKA-1641 and
> I'd
> > > like to know if it's safe to apply the patch from that issue resolution
> > to
> > > 0.8.2.1?
> > >
> > > Reference: https://issues.apache.org/jira/browse/KAFKA-1641
> > >
> > > Also there are 2 patches in there, I suppose I should be using only the
> > > latest of the two.
> > >
> > > thanks!
> > > marc
> > >
> >
>
--
-Regards,
Mayuresh R. Gharat
(862) 250-7125
Re: Log cleaner patch (KAFKA-1641) on 0.8.2.1
Posted by Marc Labbe <mr...@gmail.com>.
So far no. It happens periodically and seems to resolve when we restart the
broker (only to eventually reappear). There are a couple of things I need
to get out of the way in our setup first but I'll see what I can do to
reproduce it.
We had a couple of bad shutdowns performed and brokers restarted quickly
after that. Then, because of bad shutdown it takes a while to recover
unflushed segments.
My original question was to know if I should apply KAFKA-1641 patch to
0.8.2.1 release tag and if it was safe to do it. Any recommendation (other
than I will obviously upgrade to 0.8.2.1).
On Fri, Mar 13, 2015 at 6:23 PM, Joel Koshy <jj...@gmail.com> wrote:
> +1 - if you have a way to reproduce that would be ideal. We don't know
> the root cause of this yet. Our guess is a corner case around
> shutdowns, but not sure.
>
> On Fri, Mar 13, 2015 at 03:13:45PM -0700, Jun Rao wrote:
> > Is there a way that you can reproduce this easily?
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Mar 13, 2015 at 8:13 AM, Marc Labbe <mr...@gmail.com> wrote:
> >
> > > No exactly, the topics are compacted but messages are not compressed.
> > >
> > > I get the exact same error though. Any other options I should consider?
> > > We're on 0.8.2.0 and we also had this on 0.8.1.1 before.
> > >
> > > marc
> > >
> > > On Fri, Mar 13, 2015 at 10:47 AM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Did you get into that issue for the same reason as in the jira, i.e.,
> > > > somehow compressed messages were sent to the compact topics?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Fri, Mar 13, 2015 at 6:45 AM, Marc Labbe <mr...@gmail.com>
> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > we're often seeing log cleaner exceptions reported in KAFKA-1641
> and
> > > I'd
> > > > > like to know if it's safe to apply the patch from that issue
> resolution
> > > > to
> > > > > 0.8.2.1?
> > > > >
> > > > > Reference: https://issues.apache.org/jira/browse/KAFKA-1641
> > > > >
> > > > > Also there are 2 patches in there, I suppose I should be using
> only the
> > > > > latest of the two.
> > > > >
> > > > > thanks!
> > > > > marc
> > > > >
> > > >
> > >
>
>
Re: Log cleaner patch (KAFKA-1641) on 0.8.2.1
Posted by Joel Koshy <jj...@gmail.com>.
+1 - if you have a way to reproduce that would be ideal. We don't know
the root cause of this yet. Our guess is a corner case around
shutdowns, but not sure.
On Fri, Mar 13, 2015 at 03:13:45PM -0700, Jun Rao wrote:
> Is there a way that you can reproduce this easily?
>
> Thanks,
>
> Jun
>
> On Fri, Mar 13, 2015 at 8:13 AM, Marc Labbe <mr...@gmail.com> wrote:
>
> > No exactly, the topics are compacted but messages are not compressed.
> >
> > I get the exact same error though. Any other options I should consider?
> > We're on 0.8.2.0 and we also had this on 0.8.1.1 before.
> >
> > marc
> >
> > On Fri, Mar 13, 2015 at 10:47 AM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Did you get into that issue for the same reason as in the jira, i.e.,
> > > somehow compressed messages were sent to the compact topics?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Fri, Mar 13, 2015 at 6:45 AM, Marc Labbe <mr...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > we're often seeing log cleaner exceptions reported in KAFKA-1641 and
> > I'd
> > > > like to know if it's safe to apply the patch from that issue resolution
> > > to
> > > > 0.8.2.1?
> > > >
> > > > Reference: https://issues.apache.org/jira/browse/KAFKA-1641
> > > >
> > > > Also there are 2 patches in there, I suppose I should be using only the
> > > > latest of the two.
> > > >
> > > > thanks!
> > > > marc
> > > >
> > >
> >
Re: Log cleaner patch (KAFKA-1641) on 0.8.2.1
Posted by Jun Rao <ju...@confluent.io>.
Is there a way that you can reproduce this easily?
Thanks,
Jun
On Fri, Mar 13, 2015 at 8:13 AM, Marc Labbe <mr...@gmail.com> wrote:
> No exactly, the topics are compacted but messages are not compressed.
>
> I get the exact same error though. Any other options I should consider?
> We're on 0.8.2.0 and we also had this on 0.8.1.1 before.
>
> marc
>
> On Fri, Mar 13, 2015 at 10:47 AM, Jun Rao <ju...@confluent.io> wrote:
>
> > Did you get into that issue for the same reason as in the jira, i.e.,
> > somehow compressed messages were sent to the compact topics?
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Mar 13, 2015 at 6:45 AM, Marc Labbe <mr...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > we're often seeing log cleaner exceptions reported in KAFKA-1641 and
> I'd
> > > like to know if it's safe to apply the patch from that issue resolution
> > to
> > > 0.8.2.1?
> > >
> > > Reference: https://issues.apache.org/jira/browse/KAFKA-1641
> > >
> > > Also there are 2 patches in there, I suppose I should be using only the
> > > latest of the two.
> > >
> > > thanks!
> > > marc
> > >
> >
>
Re: Log cleaner patch (KAFKA-1641) on 0.8.2.1
Posted by Marc Labbe <mr...@gmail.com>.
No exactly, the topics are compacted but messages are not compressed.
I get the exact same error though. Any other options I should consider?
We're on 0.8.2.0 and we also had this on 0.8.1.1 before.
marc
On Fri, Mar 13, 2015 at 10:47 AM, Jun Rao <ju...@confluent.io> wrote:
> Did you get into that issue for the same reason as in the jira, i.e.,
> somehow compressed messages were sent to the compact topics?
>
> Thanks,
>
> Jun
>
> On Fri, Mar 13, 2015 at 6:45 AM, Marc Labbe <mr...@gmail.com> wrote:
>
> > Hello,
> >
> > we're often seeing log cleaner exceptions reported in KAFKA-1641 and I'd
> > like to know if it's safe to apply the patch from that issue resolution
> to
> > 0.8.2.1?
> >
> > Reference: https://issues.apache.org/jira/browse/KAFKA-1641
> >
> > Also there are 2 patches in there, I suppose I should be using only the
> > latest of the two.
> >
> > thanks!
> > marc
> >
>
Re: Log cleaner patch (KAFKA-1641) on 0.8.2.1
Posted by Jun Rao <ju...@confluent.io>.
Did you get into that issue for the same reason as in the jira, i.e.,
somehow compressed messages were sent to the compact topics?
Thanks,
Jun
On Fri, Mar 13, 2015 at 6:45 AM, Marc Labbe <mr...@gmail.com> wrote:
> Hello,
>
> we're often seeing log cleaner exceptions reported in KAFKA-1641 and I'd
> like to know if it's safe to apply the patch from that issue resolution to
> 0.8.2.1?
>
> Reference: https://issues.apache.org/jira/browse/KAFKA-1641
>
> Also there are 2 patches in there, I suppose I should be using only the
> latest of the two.
>
> thanks!
> marc
>