You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Yu Yang <yu...@gmail.com> on 2014/12/02 07:23:45 UTC
Best practice for upgrading Kafka cluster from 0.8.1 to 0.8.1.1
Hi,
We have a kafka cluster that runs Kafka 0.8.1 that we are considering
upgrade to 0.8.1.1. The Kafka documentation
<http://kafka.apache.org/documentation.html#upgrade> mentions upgrading
from 0.8 to 0.8.1, but not from 0.8.1 to 0.8.1.1. Will doing one broker at
a time by brining the broker down, updating the code, and restarting it be
sufficient? Any best practice suggestions?
Thanks!
Regards,
Yu
Re: Best practice for upgrading Kafka cluster from 0.8.1 to 0.8.1.1
Posted by Yu Yang <yu...@gmail.com>.
Thanks, Guozhang!
On Thu, Dec 4, 2014 at 9:08 AM, Guozhang Wang <wa...@gmail.com> wrote:
> You can still do the in-place upgrade, and the logs on the broker will be
> then mixed with uncompressed and compressed messages. This is fine also
> since the consumers are able to de-compress dynamically based on the
> message type when consuming the data.
>
> Guozhang
>
> On Wed, Dec 3, 2014 at 11:33 AM, Yu Yang <yu...@gmail.com> wrote:
>
> > Guozhang,
> >
> > We haven't enable message compression yet. In this case, what shall we do
> > when we upgrade to 0.8.2? Must we launch a new cluster, redirect the
> > traffic to the new cluster, and turn off the old one?
> >
> > Thanks!
> >
> > -Yu
> >
> >
> > On Tue, Dec 2, 2014 at 4:33 PM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> > > Yu,
> > >
> > > Are you enabling message compression in 0.8.1 now? If you have already
> > then
> > > upgrading to 0.8.2 will not change its behavior.
> > >
> > > Guozhang
> > >
> > > On Tue, Dec 2, 2014 at 4:21 PM, Yu Yang <yu...@gmail.com> wrote:
> > >
> > > > Hi Neha,
> > > >
> > > > Thanks for the reply! We know that Kafka 0.8.2 will be released
> soon.
> > If
> > > > we want to upgrade to Kafka 0.8.2 and enable message compression,
> will
> > we
> > > > still be able do this in the same way, or we need to handle it
> > > differently?
> > > >
> > > > Thanks!
> > > >
> > > > Regards,
> > > > -Yu
> > > >
> > > > On Tue, Dec 2, 2014 at 3:11 PM, Neha Narkhede <
> neha.narkhede@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Will doing one broker at
> > > > > a time by brining the broker down, updating the code, and
> restarting
> > it
> > > > be
> > > > > sufficient?
> > > > >
> > > > > Yes this should work for the upgrade.
> > > > >
> > > > > On Mon, Dec 1, 2014 at 10:23 PM, Yu Yang <yu...@gmail.com>
> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > We have a kafka cluster that runs Kafka 0.8.1 that we are
> > considering
> > > > > > upgrade to 0.8.1.1. The Kafka documentation
> > > > > > <http://kafka.apache.org/documentation.html#upgrade> mentions
> > > > upgrading
> > > > > > from 0.8 to 0.8.1, but not from 0.8.1 to 0.8.1.1. Will doing one
> > > > broker
> > > > > at
> > > > > > a time by brining the broker down, updating the code, and
> > restarting
> > > it
> > > > > be
> > > > > > sufficient? Any best practice suggestions?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Regards,
> > > > > > Yu
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>
>
>
> --
> -- Guozhang
>
Re: Best practice for upgrading Kafka cluster from 0.8.1 to 0.8.1.1
Posted by Guozhang Wang <wa...@gmail.com>.
You can still do the in-place upgrade, and the logs on the broker will be
then mixed with uncompressed and compressed messages. This is fine also
since the consumers are able to de-compress dynamically based on the
message type when consuming the data.
Guozhang
On Wed, Dec 3, 2014 at 11:33 AM, Yu Yang <yu...@gmail.com> wrote:
> Guozhang,
>
> We haven't enable message compression yet. In this case, what shall we do
> when we upgrade to 0.8.2? Must we launch a new cluster, redirect the
> traffic to the new cluster, and turn off the old one?
>
> Thanks!
>
> -Yu
>
>
> On Tue, Dec 2, 2014 at 4:33 PM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > Yu,
> >
> > Are you enabling message compression in 0.8.1 now? If you have already
> then
> > upgrading to 0.8.2 will not change its behavior.
> >
> > Guozhang
> >
> > On Tue, Dec 2, 2014 at 4:21 PM, Yu Yang <yu...@gmail.com> wrote:
> >
> > > Hi Neha,
> > >
> > > Thanks for the reply! We know that Kafka 0.8.2 will be released soon.
> If
> > > we want to upgrade to Kafka 0.8.2 and enable message compression, will
> we
> > > still be able do this in the same way, or we need to handle it
> > differently?
> > >
> > > Thanks!
> > >
> > > Regards,
> > > -Yu
> > >
> > > On Tue, Dec 2, 2014 at 3:11 PM, Neha Narkhede <neha.narkhede@gmail.com
> >
> > > wrote:
> > >
> > > > Will doing one broker at
> > > > a time by brining the broker down, updating the code, and restarting
> it
> > > be
> > > > sufficient?
> > > >
> > > > Yes this should work for the upgrade.
> > > >
> > > > On Mon, Dec 1, 2014 at 10:23 PM, Yu Yang <yu...@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > We have a kafka cluster that runs Kafka 0.8.1 that we are
> considering
> > > > > upgrade to 0.8.1.1. The Kafka documentation
> > > > > <http://kafka.apache.org/documentation.html#upgrade> mentions
> > > upgrading
> > > > > from 0.8 to 0.8.1, but not from 0.8.1 to 0.8.1.1. Will doing one
> > > broker
> > > > at
> > > > > a time by brining the broker down, updating the code, and
> restarting
> > it
> > > > be
> > > > > sufficient? Any best practice suggestions?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Regards,
> > > > > Yu
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
--
-- Guozhang
Re: Best practice for upgrading Kafka cluster from 0.8.1 to 0.8.1.1
Posted by Yu Yang <yu...@gmail.com>.
Guozhang,
We haven't enable message compression yet. In this case, what shall we do
when we upgrade to 0.8.2? Must we launch a new cluster, redirect the
traffic to the new cluster, and turn off the old one?
Thanks!
-Yu
On Tue, Dec 2, 2014 at 4:33 PM, Guozhang Wang <wa...@gmail.com> wrote:
> Yu,
>
> Are you enabling message compression in 0.8.1 now? If you have already then
> upgrading to 0.8.2 will not change its behavior.
>
> Guozhang
>
> On Tue, Dec 2, 2014 at 4:21 PM, Yu Yang <yu...@gmail.com> wrote:
>
> > Hi Neha,
> >
> > Thanks for the reply! We know that Kafka 0.8.2 will be released soon. If
> > we want to upgrade to Kafka 0.8.2 and enable message compression, will we
> > still be able do this in the same way, or we need to handle it
> differently?
> >
> > Thanks!
> >
> > Regards,
> > -Yu
> >
> > On Tue, Dec 2, 2014 at 3:11 PM, Neha Narkhede <ne...@gmail.com>
> > wrote:
> >
> > > Will doing one broker at
> > > a time by brining the broker down, updating the code, and restarting it
> > be
> > > sufficient?
> > >
> > > Yes this should work for the upgrade.
> > >
> > > On Mon, Dec 1, 2014 at 10:23 PM, Yu Yang <yu...@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > We have a kafka cluster that runs Kafka 0.8.1 that we are considering
> > > > upgrade to 0.8.1.1. The Kafka documentation
> > > > <http://kafka.apache.org/documentation.html#upgrade> mentions
> > upgrading
> > > > from 0.8 to 0.8.1, but not from 0.8.1 to 0.8.1.1. Will doing one
> > broker
> > > at
> > > > a time by brining the broker down, updating the code, and restarting
> it
> > > be
> > > > sufficient? Any best practice suggestions?
> > > >
> > > > Thanks!
> > > >
> > > > Regards,
> > > > Yu
> > > >
> > >
> >
>
>
>
> --
> -- Guozhang
>
Re: Best practice for upgrading Kafka cluster from 0.8.1 to 0.8.1.1
Posted by Guozhang Wang <wa...@gmail.com>.
Yu,
Are you enabling message compression in 0.8.1 now? If you have already then
upgrading to 0.8.2 will not change its behavior.
Guozhang
On Tue, Dec 2, 2014 at 4:21 PM, Yu Yang <yu...@gmail.com> wrote:
> Hi Neha,
>
> Thanks for the reply! We know that Kafka 0.8.2 will be released soon. If
> we want to upgrade to Kafka 0.8.2 and enable message compression, will we
> still be able do this in the same way, or we need to handle it differently?
>
> Thanks!
>
> Regards,
> -Yu
>
> On Tue, Dec 2, 2014 at 3:11 PM, Neha Narkhede <ne...@gmail.com>
> wrote:
>
> > Will doing one broker at
> > a time by brining the broker down, updating the code, and restarting it
> be
> > sufficient?
> >
> > Yes this should work for the upgrade.
> >
> > On Mon, Dec 1, 2014 at 10:23 PM, Yu Yang <yu...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > We have a kafka cluster that runs Kafka 0.8.1 that we are considering
> > > upgrade to 0.8.1.1. The Kafka documentation
> > > <http://kafka.apache.org/documentation.html#upgrade> mentions
> upgrading
> > > from 0.8 to 0.8.1, but not from 0.8.1 to 0.8.1.1. Will doing one
> broker
> > at
> > > a time by brining the broker down, updating the code, and restarting it
> > be
> > > sufficient? Any best practice suggestions?
> > >
> > > Thanks!
> > >
> > > Regards,
> > > Yu
> > >
> >
>
--
-- Guozhang
Re: Best practice for upgrading Kafka cluster from 0.8.1 to 0.8.1.1
Posted by Yu Yang <yu...@gmail.com>.
Hi Neha,
Thanks for the reply! We know that Kafka 0.8.2 will be released soon. If
we want to upgrade to Kafka 0.8.2 and enable message compression, will we
still be able do this in the same way, or we need to handle it differently?
Thanks!
Regards,
-Yu
On Tue, Dec 2, 2014 at 3:11 PM, Neha Narkhede <ne...@gmail.com>
wrote:
> Will doing one broker at
> a time by brining the broker down, updating the code, and restarting it be
> sufficient?
>
> Yes this should work for the upgrade.
>
> On Mon, Dec 1, 2014 at 10:23 PM, Yu Yang <yu...@gmail.com> wrote:
>
> > Hi,
> >
> > We have a kafka cluster that runs Kafka 0.8.1 that we are considering
> > upgrade to 0.8.1.1. The Kafka documentation
> > <http://kafka.apache.org/documentation.html#upgrade> mentions upgrading
> > from 0.8 to 0.8.1, but not from 0.8.1 to 0.8.1.1. Will doing one broker
> at
> > a time by brining the broker down, updating the code, and restarting it
> be
> > sufficient? Any best practice suggestions?
> >
> > Thanks!
> >
> > Regards,
> > Yu
> >
>
Re: Best practice for upgrading Kafka cluster from 0.8.1 to 0.8.1.1
Posted by Neha Narkhede <ne...@gmail.com>.
Will doing one broker at
a time by brining the broker down, updating the code, and restarting it be
sufficient?
Yes this should work for the upgrade.
On Mon, Dec 1, 2014 at 10:23 PM, Yu Yang <yu...@gmail.com> wrote:
> Hi,
>
> We have a kafka cluster that runs Kafka 0.8.1 that we are considering
> upgrade to 0.8.1.1. The Kafka documentation
> <http://kafka.apache.org/documentation.html#upgrade> mentions upgrading
> from 0.8 to 0.8.1, but not from 0.8.1 to 0.8.1.1. Will doing one broker at
> a time by brining the broker down, updating the code, and restarting it be
> sufficient? Any best practice suggestions?
>
> Thanks!
>
> Regards,
> Yu
>