You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Wouter de Bie <wo...@spotify.com> on 2013/01/15 18:07:55 UTC

The future of 0.7

Hi all,

We're currently using Kafka 0.7 and here and there we're running in a few bugs here and there. Since 0.8 is around the corner, it might not make too much sense in sending the patches for 0.8 upstream, unless 0.7 will live a (long) life next to 0.8. What's your take on the future of the 0.7 branch?

Thanks!

--
Wouter de Bie
Team Lead Analytics and Data Infrastructure, Spotify
wouter@spotify.com
+46 72 018 0777

Re: The future of 0.7

Posted by Neha Narkhede <ne...@gmail.com>.
Hi Jason,

This is included with Kafka 0.8 - kafka.tools.KafkaMigrationTool. It runs a
0.7 consumer and 0.8 producer to copy the data between your 0.7 Kafka
cluster and 0.8 Kafka cluster.

Thanks,
Neha


On Tue, Jan 22, 2013 at 12:21 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> Hi Neha,
>
> Can you describe the migration tool you mention below, for copying data
> from 0.7 to 0.8?  Is this something provided with 0.8?  Or do apps need to
> write custom migration tools?
>
> Thanks,
>
> Jason
>
> On Tue, Jan 15, 2013 at 11:06 AM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > Broadly, the strategy stays the same, except that the 0.8 broker will not
> > be able to serve requests coming from a pre-0.8 client.
> > So, the upgrade path will be and we will document this in detail soon -
> >
> > 1. Start a 0.8 Kafka cluster overlaid on the 0.6/0.7 hardware or on
> > separate hardware if you can spare
> > 2. Start a migration tool to copy data from 0.7 to 0.8. Currently we
> don't
> > have migration tool support for 0.6 but it is very easy to add, so please
> > file a JIRA for this.
> > 3. Move consumers to the 0.8 cluster (since it has all the data that your
> > 0.6/0.7 cluster has)
> > 4. Once all consumers have moved, start moving producers over to the 0.8
> > cluster
> > 5. Stop and decommission the 0.6/0.7 Kafka cluster.
> >
> > Thanks,
> > Neha
> >
> >
> > On Tue, Jan 15, 2013 at 11:01 AM, Evan Chan <ev...@ooyala.com> wrote:
> >
> > > Will it be possible to upgrade from 0.6 directly to 0.8, and what would
> > the
> > > migration strategy be like?   I understand for 0.6 to 0.7, the strategy
> > was
> > > to first upgrade your consumers, then the broker, then the producers.
> > >
> > > -Evan
> > >
> > >
> > > On Tue, Jan 15, 2013 at 9:44 AM, Neha Narkhede <
> neha.narkhede@gmail.com
> > > >wrote:
> > >
> > > > To answer your question, even if the team is working on 0.8, we
> > > understand
> > > > that it will take some time for 0.8 to be completely stable and for
> > users
> > > > to build some confidence in moving to 0.8 since it is backwards
> > > > incompatible. Keeping that in mind, we will try our best to patch the
> > 0.7
> > > > branch for bugs and do point releases from there for a few months
> down
> > > the
> > > > line.
> > > > Feel free to file bugs and we can discuss how to handle a release for
> > > those
> > > > bug fixes.
> > > >
> > > > Thanks,
> > > > Neha
> > > >
> > > >
> > > > On Tue, Jan 15, 2013 at 9:39 AM, Jun Rao <ju...@gmail.com> wrote:
> > > >
> > > > > Our current plan to to get 0.8 out asap and only patch blocker
> issues
> > > in
> > > > > 0.7. If the issues that you found apply to 0.8, please feel free to
> > > open
> > > > a
> > > > > jira and/or attach a patch.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Jan 15, 2013 at 9:07 AM, Wouter de Bie <wouter@spotify.com
> >
> > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > We're currently using Kafka 0.7 and here and there we're running
> > in a
> > > > few
> > > > > > bugs here and there. Since 0.8 is around the corner, it might not
> > > make
> > > > > too
> > > > > > much sense in sending the patches for 0.8 upstream, unless 0.7
> will
> > > > live
> > > > > a
> > > > > > (long) life next to 0.8. What's your take on the future of the
> 0.7
> > > > > branch?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > --
> > > > > > Wouter de Bie
> > > > > > Team Lead Analytics and Data Infrastructure, Spotify
> > > > > > wouter@spotify.com
> > > > > > +46 72 018 0777
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > --
> > > *Evan Chan*
> > > Senior Software Engineer |
> > > ev@ooyala.com | (650) 996-4600
> > > www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > > @ooyala<http://www.twitter.com/ooyala>
> > >
> >
>

Re: The future of 0.7

Posted by Jason Rosenberg <jb...@squareup.com>.
Hi Neha,

Can you describe the migration tool you mention below, for copying data
from 0.7 to 0.8?  Is this something provided with 0.8?  Or do apps need to
write custom migration tools?

Thanks,

Jason

On Tue, Jan 15, 2013 at 11:06 AM, Neha Narkhede <ne...@gmail.com>wrote:

> Broadly, the strategy stays the same, except that the 0.8 broker will not
> be able to serve requests coming from a pre-0.8 client.
> So, the upgrade path will be and we will document this in detail soon -
>
> 1. Start a 0.8 Kafka cluster overlaid on the 0.6/0.7 hardware or on
> separate hardware if you can spare
> 2. Start a migration tool to copy data from 0.7 to 0.8. Currently we don't
> have migration tool support for 0.6 but it is very easy to add, so please
> file a JIRA for this.
> 3. Move consumers to the 0.8 cluster (since it has all the data that your
> 0.6/0.7 cluster has)
> 4. Once all consumers have moved, start moving producers over to the 0.8
> cluster
> 5. Stop and decommission the 0.6/0.7 Kafka cluster.
>
> Thanks,
> Neha
>
>
> On Tue, Jan 15, 2013 at 11:01 AM, Evan Chan <ev...@ooyala.com> wrote:
>
> > Will it be possible to upgrade from 0.6 directly to 0.8, and what would
> the
> > migration strategy be like?   I understand for 0.6 to 0.7, the strategy
> was
> > to first upgrade your consumers, then the broker, then the producers.
> >
> > -Evan
> >
> >
> > On Tue, Jan 15, 2013 at 9:44 AM, Neha Narkhede <neha.narkhede@gmail.com
> > >wrote:
> >
> > > To answer your question, even if the team is working on 0.8, we
> > understand
> > > that it will take some time for 0.8 to be completely stable and for
> users
> > > to build some confidence in moving to 0.8 since it is backwards
> > > incompatible. Keeping that in mind, we will try our best to patch the
> 0.7
> > > branch for bugs and do point releases from there for a few months down
> > the
> > > line.
> > > Feel free to file bugs and we can discuss how to handle a release for
> > those
> > > bug fixes.
> > >
> > > Thanks,
> > > Neha
> > >
> > >
> > > On Tue, Jan 15, 2013 at 9:39 AM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > > > Our current plan to to get 0.8 out asap and only patch blocker issues
> > in
> > > > 0.7. If the issues that you found apply to 0.8, please feel free to
> > open
> > > a
> > > > jira and/or attach a patch.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Jan 15, 2013 at 9:07 AM, Wouter de Bie <wo...@spotify.com>
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > We're currently using Kafka 0.7 and here and there we're running
> in a
> > > few
> > > > > bugs here and there. Since 0.8 is around the corner, it might not
> > make
> > > > too
> > > > > much sense in sending the patches for 0.8 upstream, unless 0.7 will
> > > live
> > > > a
> > > > > (long) life next to 0.8. What's your take on the future of the 0.7
> > > > branch?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > --
> > > > > Wouter de Bie
> > > > > Team Lead Analytics and Data Infrastructure, Spotify
> > > > > wouter@spotify.com
> > > > > +46 72 018 0777
> > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > --
> > *Evan Chan*
> > Senior Software Engineer |
> > ev@ooyala.com | (650) 996-4600
> > www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > @ooyala<http://www.twitter.com/ooyala>
> >
>

Re: The future of 0.7

Posted by Neha Narkhede <ne...@gmail.com>.
Broadly, the strategy stays the same, except that the 0.8 broker will not
be able to serve requests coming from a pre-0.8 client.
So, the upgrade path will be and we will document this in detail soon -

1. Start a 0.8 Kafka cluster overlaid on the 0.6/0.7 hardware or on
separate hardware if you can spare
2. Start a migration tool to copy data from 0.7 to 0.8. Currently we don't
have migration tool support for 0.6 but it is very easy to add, so please
file a JIRA for this.
3. Move consumers to the 0.8 cluster (since it has all the data that your
0.6/0.7 cluster has)
4. Once all consumers have moved, start moving producers over to the 0.8
cluster
5. Stop and decommission the 0.6/0.7 Kafka cluster.

Thanks,
Neha


On Tue, Jan 15, 2013 at 11:01 AM, Evan Chan <ev...@ooyala.com> wrote:

> Will it be possible to upgrade from 0.6 directly to 0.8, and what would the
> migration strategy be like?   I understand for 0.6 to 0.7, the strategy was
> to first upgrade your consumers, then the broker, then the producers.
>
> -Evan
>
>
> On Tue, Jan 15, 2013 at 9:44 AM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > To answer your question, even if the team is working on 0.8, we
> understand
> > that it will take some time for 0.8 to be completely stable and for users
> > to build some confidence in moving to 0.8 since it is backwards
> > incompatible. Keeping that in mind, we will try our best to patch the 0.7
> > branch for bugs and do point releases from there for a few months down
> the
> > line.
> > Feel free to file bugs and we can discuss how to handle a release for
> those
> > bug fixes.
> >
> > Thanks,
> > Neha
> >
> >
> > On Tue, Jan 15, 2013 at 9:39 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > Our current plan to to get 0.8 out asap and only patch blocker issues
> in
> > > 0.7. If the issues that you found apply to 0.8, please feel free to
> open
> > a
> > > jira and/or attach a patch.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jan 15, 2013 at 9:07 AM, Wouter de Bie <wo...@spotify.com>
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > We're currently using Kafka 0.7 and here and there we're running in a
> > few
> > > > bugs here and there. Since 0.8 is around the corner, it might not
> make
> > > too
> > > > much sense in sending the patches for 0.8 upstream, unless 0.7 will
> > live
> > > a
> > > > (long) life next to 0.8. What's your take on the future of the 0.7
> > > branch?
> > > >
> > > > Thanks!
> > > >
> > > > --
> > > > Wouter de Bie
> > > > Team Lead Analytics and Data Infrastructure, Spotify
> > > > wouter@spotify.com
> > > > +46 72 018 0777
> > > >
> > > >
> > >
> >
>
>
>
> --
> --
> *Evan Chan*
> Senior Software Engineer |
> ev@ooyala.com | (650) 996-4600
> www.ooyala.com | blog <http://www.ooyala.com/blog> |
> @ooyala<http://www.twitter.com/ooyala>
>

Re: The future of 0.7

Posted by Evan Chan <ev...@ooyala.com>.
Will it be possible to upgrade from 0.6 directly to 0.8, and what would the
migration strategy be like?   I understand for 0.6 to 0.7, the strategy was
to first upgrade your consumers, then the broker, then the producers.

-Evan


On Tue, Jan 15, 2013 at 9:44 AM, Neha Narkhede <ne...@gmail.com>wrote:

> To answer your question, even if the team is working on 0.8, we understand
> that it will take some time for 0.8 to be completely stable and for users
> to build some confidence in moving to 0.8 since it is backwards
> incompatible. Keeping that in mind, we will try our best to patch the 0.7
> branch for bugs and do point releases from there for a few months down the
> line.
> Feel free to file bugs and we can discuss how to handle a release for those
> bug fixes.
>
> Thanks,
> Neha
>
>
> On Tue, Jan 15, 2013 at 9:39 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > Our current plan to to get 0.8 out asap and only patch blocker issues in
> > 0.7. If the issues that you found apply to 0.8, please feel free to open
> a
> > jira and/or attach a patch.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jan 15, 2013 at 9:07 AM, Wouter de Bie <wo...@spotify.com>
> wrote:
> >
> > > Hi all,
> > >
> > > We're currently using Kafka 0.7 and here and there we're running in a
> few
> > > bugs here and there. Since 0.8 is around the corner, it might not make
> > too
> > > much sense in sending the patches for 0.8 upstream, unless 0.7 will
> live
> > a
> > > (long) life next to 0.8. What's your take on the future of the 0.7
> > branch?
> > >
> > > Thanks!
> > >
> > > --
> > > Wouter de Bie
> > > Team Lead Analytics and Data Infrastructure, Spotify
> > > wouter@spotify.com
> > > +46 72 018 0777
> > >
> > >
> >
>



-- 
--
*Evan Chan*
Senior Software Engineer |
ev@ooyala.com | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>

Re: The future of 0.7

Posted by Neha Narkhede <ne...@gmail.com>.
To answer your question, even if the team is working on 0.8, we understand
that it will take some time for 0.8 to be completely stable and for users
to build some confidence in moving to 0.8 since it is backwards
incompatible. Keeping that in mind, we will try our best to patch the 0.7
branch for bugs and do point releases from there for a few months down the
line.
Feel free to file bugs and we can discuss how to handle a release for those
bug fixes.

Thanks,
Neha


On Tue, Jan 15, 2013 at 9:39 AM, Jun Rao <ju...@gmail.com> wrote:

> Our current plan to to get 0.8 out asap and only patch blocker issues in
> 0.7. If the issues that you found apply to 0.8, please feel free to open a
> jira and/or attach a patch.
>
> Thanks,
>
> Jun
>
> On Tue, Jan 15, 2013 at 9:07 AM, Wouter de Bie <wo...@spotify.com> wrote:
>
> > Hi all,
> >
> > We're currently using Kafka 0.7 and here and there we're running in a few
> > bugs here and there. Since 0.8 is around the corner, it might not make
> too
> > much sense in sending the patches for 0.8 upstream, unless 0.7 will live
> a
> > (long) life next to 0.8. What's your take on the future of the 0.7
> branch?
> >
> > Thanks!
> >
> > --
> > Wouter de Bie
> > Team Lead Analytics and Data Infrastructure, Spotify
> > wouter@spotify.com
> > +46 72 018 0777
> >
> >
>

Re: The future of 0.7

Posted by Jun Rao <ju...@gmail.com>.
Our current plan to to get 0.8 out asap and only patch blocker issues in
0.7. If the issues that you found apply to 0.8, please feel free to open a
jira and/or attach a patch.

Thanks,

Jun

On Tue, Jan 15, 2013 at 9:07 AM, Wouter de Bie <wo...@spotify.com> wrote:

> Hi all,
>
> We're currently using Kafka 0.7 and here and there we're running in a few
> bugs here and there. Since 0.8 is around the corner, it might not make too
> much sense in sending the patches for 0.8 upstream, unless 0.7 will live a
> (long) life next to 0.8. What's your take on the future of the 0.7 branch?
>
> Thanks!
>
> --
> Wouter de Bie
> Team Lead Analytics and Data Infrastructure, Spotify
> wouter@spotify.com
> +46 72 018 0777
>
>