You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Gwen Shapira <gw...@confluent.io> on 2015/08/14 07:24:09 UTC
Re: Copycat (KIP-26. PR-99) - plan on moving forward
Reminder:
If no one objects, I'm planning on merging PR-99 tomorrow.
On Wed, Aug 12, 2015 at 11:17 AM, Gwen Shapira <gw...@confluent.io> wrote:
> Hi Kafka fans,
>
> I'd like to move ahead with getting the initial Copycat patches into Kafka.
>
> If you'll look at the PR, we still have a number of open questions on the
> API (batch connectors, connector upgrades, data formats). I believe it will
> be easier to iterate on those independently of getting the basic patch in,
> but this means that the API we are committing may change in the next few
> month (before the 0.8.3 release).
>
> Here is my plan:
> 1. Add Stability attributes (The type that Hadoop and Guavas have) to
> Kafka.
> 2. Ewen agreed to mark the data API and connector APIs as "unstable", so
> no one will get caught by surprise if they implement a connector and the
> APIs move
> 3. I will start few concurrent discussions on the mailing list on what we
> see as open issues (the data API discussion already started by Ewen).
> Please chime in.
> 4. While the discussions are still on-going we will commit the initial
> CopyCat patch to allow people to try it out and give feedback.
>
> If someone objects to this plan, please respond ASAP. We'd like to get the
> plan done by Friday.
>
> Thanks,
> Gwen
>
>
Re: Copycat (KIP-26. PR-99) - plan on moving forward
Posted by Gwen Shapira <gw...@confluent.io>.
Yes, I think the rename (other than being fun) will prevent some confusion
with existing coordinator.
On Aug 14, 2015 1:09 AM, "Ewen Cheslack-Postava" <ew...@confluent.io> wrote:
> For obvious reasons, I definitely won't object, but I will probably quickly
> have a follow-up patch to add a very basic system test for Copycat. I was
> trying to get it in with this patch since I think it is a good idea to
> encourage people to write system tests with major additions like this, but
> I think we need to release an update to ducktape first.
>
> @Gwen, if you want me to rename Coordinator -> Herder, let me know before
> you merge :) I'm definitely not opposed to a little naming silliness, I
> just hadn't gotten to updating the patch yet.
>
> -Ewen
>
> On Thu, Aug 13, 2015 at 11:26 PM, Neha Narkhede <ne...@confluent.io> wrote:
>
> > +1 on resolving some of these open issues separately. It is a pain to
> > maintain, review, discuss and commit these type of large patches.
> Typically
> > the set of improvements to really stabilize new features like this go
> well
> > beyond one large initial patch anyway.
> >
> > On Thu, Aug 13, 2015 at 10:24 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >
> > > Reminder:
> > > If no one objects, I'm planning on merging PR-99 tomorrow.
> > >
> > > On Wed, Aug 12, 2015 at 11:17 AM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > >
> > > > Hi Kafka fans,
> > > >
> > > > I'd like to move ahead with getting the initial Copycat patches into
> > > Kafka.
> > > >
> > > > If you'll look at the PR, we still have a number of open questions on
> > the
> > > > API (batch connectors, connector upgrades, data formats). I believe
> it
> > > will
> > > > be easier to iterate on those independently of getting the basic
> patch
> > > in,
> > > > but this means that the API we are committing may change in the next
> > few
> > > > month (before the 0.8.3 release).
> > > >
> > > > Here is my plan:
> > > > 1. Add Stability attributes (The type that Hadoop and Guavas have) to
> > > > Kafka.
> > > > 2. Ewen agreed to mark the data API and connector APIs as "unstable",
> > so
> > > > no one will get caught by surprise if they implement a connector and
> > the
> > > > APIs move
> > > > 3. I will start few concurrent discussions on the mailing list on
> what
> > we
> > > > see as open issues (the data API discussion already started by Ewen).
> > > > Please chime in.
> > > > 4. While the discussions are still on-going we will commit the
> initial
> > > > CopyCat patch to allow people to try it out and give feedback.
> > > >
> > > > If someone objects to this plan, please respond ASAP. We'd like to
> get
> > > the
> > > > plan done by Friday.
> > > >
> > > > Thanks,
> > > > Gwen
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > Neha
> >
>
>
>
> --
> Thanks,
> Ewen
>
Re: Copycat (KIP-26. PR-99) - plan on moving forward
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
For obvious reasons, I definitely won't object, but I will probably quickly
have a follow-up patch to add a very basic system test for Copycat. I was
trying to get it in with this patch since I think it is a good idea to
encourage people to write system tests with major additions like this, but
I think we need to release an update to ducktape first.
@Gwen, if you want me to rename Coordinator -> Herder, let me know before
you merge :) I'm definitely not opposed to a little naming silliness, I
just hadn't gotten to updating the patch yet.
-Ewen
On Thu, Aug 13, 2015 at 11:26 PM, Neha Narkhede <ne...@confluent.io> wrote:
> +1 on resolving some of these open issues separately. It is a pain to
> maintain, review, discuss and commit these type of large patches. Typically
> the set of improvements to really stabilize new features like this go well
> beyond one large initial patch anyway.
>
> On Thu, Aug 13, 2015 at 10:24 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Reminder:
> > If no one objects, I'm planning on merging PR-99 tomorrow.
> >
> > On Wed, Aug 12, 2015 at 11:17 AM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >
> > > Hi Kafka fans,
> > >
> > > I'd like to move ahead with getting the initial Copycat patches into
> > Kafka.
> > >
> > > If you'll look at the PR, we still have a number of open questions on
> the
> > > API (batch connectors, connector upgrades, data formats). I believe it
> > will
> > > be easier to iterate on those independently of getting the basic patch
> > in,
> > > but this means that the API we are committing may change in the next
> few
> > > month (before the 0.8.3 release).
> > >
> > > Here is my plan:
> > > 1. Add Stability attributes (The type that Hadoop and Guavas have) to
> > > Kafka.
> > > 2. Ewen agreed to mark the data API and connector APIs as "unstable",
> so
> > > no one will get caught by surprise if they implement a connector and
> the
> > > APIs move
> > > 3. I will start few concurrent discussions on the mailing list on what
> we
> > > see as open issues (the data API discussion already started by Ewen).
> > > Please chime in.
> > > 4. While the discussions are still on-going we will commit the initial
> > > CopyCat patch to allow people to try it out and give feedback.
> > >
> > > If someone objects to this plan, please respond ASAP. We'd like to get
> > the
> > > plan done by Friday.
> > >
> > > Thanks,
> > > Gwen
> > >
> > >
> >
>
>
>
> --
> Thanks,
> Neha
>
--
Thanks,
Ewen
Re: Copycat (KIP-26. PR-99) - plan on moving forward
Posted by Neha Narkhede <ne...@confluent.io>.
+1 on resolving some of these open issues separately. It is a pain to
maintain, review, discuss and commit these type of large patches. Typically
the set of improvements to really stabilize new features like this go well
beyond one large initial patch anyway.
On Thu, Aug 13, 2015 at 10:24 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Reminder:
> If no one objects, I'm planning on merging PR-99 tomorrow.
>
> On Wed, Aug 12, 2015 at 11:17 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Hi Kafka fans,
> >
> > I'd like to move ahead with getting the initial Copycat patches into
> Kafka.
> >
> > If you'll look at the PR, we still have a number of open questions on the
> > API (batch connectors, connector upgrades, data formats). I believe it
> will
> > be easier to iterate on those independently of getting the basic patch
> in,
> > but this means that the API we are committing may change in the next few
> > month (before the 0.8.3 release).
> >
> > Here is my plan:
> > 1. Add Stability attributes (The type that Hadoop and Guavas have) to
> > Kafka.
> > 2. Ewen agreed to mark the data API and connector APIs as "unstable", so
> > no one will get caught by surprise if they implement a connector and the
> > APIs move
> > 3. I will start few concurrent discussions on the mailing list on what we
> > see as open issues (the data API discussion already started by Ewen).
> > Please chime in.
> > 4. While the discussions are still on-going we will commit the initial
> > CopyCat patch to allow people to try it out and give feedback.
> >
> > If someone objects to this plan, please respond ASAP. We'd like to get
> the
> > plan done by Friday.
> >
> > Thanks,
> > Gwen
> >
> >
>
--
Thanks,
Neha