0.8.3 release plan

[Joe Stein <jo...@stealth.ly>]

There hasn't been any public discussion about the 0.8.3 release plan.

There seems to be a lot of work in flight, work with patches and review
that could/should get committed but now just pending KIPS, work without KIP
but that is in trunk already (e.g. the new Consumer) that would be the the
release but missing the KIP for the release...

What does this mean for the 0.8.3 release? What are we trying to get out
and when?

Also looking at
https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan there
seems to be things we are getting earlier (which is great of course) so are
we going to try to up the version and go with 0.9.0?

0.8.2.0 ended up getting very bloated and that delayed it much longer than
we had originally communicated to the community and want to make sure we
take that feedback from the community and try to improve upon it.

Thanks!

~ Joe Stein
- - - - - - - - - - - - - - - - -

  http://www.stealth.ly
- - - - - - - - - - - - - - - - -

Re: 0.8.3 release plan

[Dana Powers <da...@rd.io>]

Re last beta, I think it was extremely useful.  We identified big issues
wrt API versioning and third-party client compatibility.  Even if the
server code doesnt get deployed widely, I think the beta period is still a
good signal to third party client devs to rev up tests and make any updates
necessary to support the new version in advance of a full release.

Dana
On Mar 12, 2015 2:37 PM, "Jun Rao" <ju...@confluent.io> wrote:

> Since we have decided to only support security on the new clients, the
> security feature will only be available after the new consumer is done. So,
> for the next release, we probably should just target finishing the new
> consumer as the main feature. If security can be done in the same release,
> that's just a bonus.
>
> Thanks,
>
> Jun
>
> On Thu, Mar 12, 2015 at 6:37 AM, Harsha <ka...@harsha.io> wrote:
>
> > I would like to include ssl/sasl
> > 1) Kafka-1684 (Patch posted for a review)
> > 2) Kafka-1686 (Patch depends on kafka-1684)
> > 3) Kafka-1688 (work is in progress)
> > Thanks,
> > Harsha
> > On Thu, Mar 12, 2015, at 04:35 AM, Guozhang Wang wrote:
> > > Gwen,
> > >
> > > Just for clarification, you were suggesting we should or should not
> > > include
> > > MM improvement in 0.8.3? I personally would prefer it (KAFKA-1650 and
> > > KAFKA-1997) to go into 0.8.3.
> > >
> > > I see Joe has made a pass over the tickets and mark them 0.8.3. We can
> > > probably do another pass and consider adding:
> > >
> > > 1) Purgatory improvement (KAFKA-1989).
> > > 2) Compression improvement (KAFKA-527).
> > > 3) Some unit test failures (KAFKA-1501, I think we are pretty close in
> > > getting it fixed).
> > > 4) any other tickets?
> > >
> > > Guozhang
> > >
> > > On Wed, Mar 11, 2015 at 9:40 PM, Jay Kreps <ja...@gmail.com>
> wrote:
> > >
> > > > With regard to mm, I was kind of assuming just based on the amount of
> > work
> > > > that that would go in for sure, but yeah I agree it is important.
> > > >
> > > > -Jay
> > > >
> > > > On Wed, Mar 11, 2015 at 9:39 PM, Jay Kreps <ja...@gmail.com>
> > wrote:
> > > >
> > > > > What I was trying to say was let's do a real release whenever
> either
> > > > > consumer or authn is done whichever happens first (or both if they
> > can
> > > > > happen close together)--not sure which is more likely to slip.
> > > > >
> > > > > WRT the beta thing I think the question for people is whether the
> > beta
> > > > > period was helpful or not in getting a more stable release? We
> could
> > > > either
> > > > > do a beta release again or we could just do a normal release and
> > call the
> > > > > consumer feature "experimental" or whatever...basically something
> to
> > get
> > > > it
> > > > > in peoples hands before it is supposed to work perfectly and never
> > change
> > > > > again.
> > > > >
> > > > > -Jay
> > > > >
> > > > >
> > > > > On Wed, Mar 11, 2015 at 9:27 PM, Gwen Shapira <
> gshapira@cloudera.com
> > >
> > > > > wrote:
> > > > >
> > > > >> So basically you are suggesting - lets do a beta release whenever
> we
> > > > >> feel the new consumer is done?
> > > > >>
> > > > >> This can definitely work.
> > > > >>
> > > > >> I'd prefer holding for MM improvements too. IMO, its not just more
> > > > >> improvements like flush() and compression optimization.
> > > > >> Current MirrorMaker can lose data, which makes it pretty useless
> for
> > > > >> its job. We hear lots of requests for robust MM from our
> customers,
> > so
> > > > >> I can imagine its pretty important to the Kafka community (unless
> I
> > > > >> have a completely skewed sample).
> > > > >>
> > > > >> Gwen
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <ja...@gmail.com>
> > wrote:
> > > > >> > Yeah the real question is always what will we block on?
> > > > >> >
> > > > >> > I don't think we should try to hold back smaller changes. In
> this
> > > > >> bucket I
> > > > >> > would include most things you described: mm improvements,
> replica
> > > > >> > assignment tool improvements, flush, purgatory improvements,
> > > > compression
> > > > >> > optimization, etc. Likely these will all get done in time as
> well
> > as
> > > > >> many
> > > > >> > things that kind of pop up from users but probably aren't worth
> > doing
> > > > a
> > > > >> > release for on their own. If one of them slips that fine. I also
> > don't
> > > > >> > think we should try to hold back work that is done if it isn't
> on
> > a
> > > > >> list.
> > > > >> >
> > > > >> > I would consider either SSL+SASL or the consumer worthy of a
> > release
> > > > on
> > > > >> its
> > > > >> > own. If they finish close to the same time that is great. We can
> > maybe
> > > > >> just
> > > > >> > assess as these evolve where the other one is at and make a call
> > > > >> whether it
> > > > >> > will be one or both?
> > > > >> >
> > > > >> > -Jay
> > > > >> >
> > > > >> > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <
> > gshapira@cloudera.com>
> > > > >> wrote:
> > > > >> >
> > > > >> >> If we are going in terms of features, I can see the following
> > > > features
> > > > >> >> getting in in the next month or two:
> > > > >> >>
> > > > >> >> * New consumer
> > > > >> >> * Improved Mirror Maker (I've seen tons of interest)
> > > > >> >> * Centralized admin requests (aka KIP-4)
> > > > >> >> * Nicer replica-reassignment tool
> > > > >> >> * SSL (and perhaps also SASL)?
> > > > >> >>
> > > > >> >> I think this collection will make a nice release. Perhaps we
> can
> > cap
> > > > >> >> it there and focus (as a community) on getting these in, we can
> > have
> > > > a
> > > > >> >> release without too much scope creep in the
> > not-very-distant-future?
> > > > >> >> Even just 3 out of these 5 will still make a nice incremental
> > > > >> >> improvement.
> > > > >> >>
> > > > >> >> Gwen
> > > > >> >>
> > > > >> >>
> > > > >> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <
> jay.kreps@gmail.com>
> > > > >> wrote:
> > > > >> >> > Yeah I'd be in favor of a quicker, smaller release but I
> think
> > as
> > > > >> long as
> > > > >> >> > we have these big things in flight we should probably keep
> the
> > > > >> release
> > > > >> >> > criteria feature-based rather than time-based, though (e.g.
> > "when X
> > > > >> >> works"
> > > > >> >> > not "every other month).
> > > > >> >> >
> > > > >> >> > Ideally the next release would have at least a "beta" version
> > of
> > > > the
> > > > >> new
> > > > >> >> > consumer. I think having a new hunk of code like that
> > available but
> > > > >> >> marked
> > > > >> >> > as "beta" is maybe a good way to go, as it gets it into
> peoples
> > > > >> hands for
> > > > >> >> > testing. This way we can declare the API not fully locked
> down
> > > > until
> > > > >> the
> > > > >> >> > final release too, since mostly users only look at stuff
> after
> > we
> > > > >> release
> > > > >> >> > it. Maybe we can try to construct a schedule around this?
> > > > >> >> >
> > > > >> >> > -Jay
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <
> > joe.stein@stealth.ly>
> > > > >> wrote:
> > > > >> >> >
> > > > >> >> >> There hasn't been any public discussion about the 0.8.3
> > release
> > > > >> plan.
> > > > >> >> >>
> > > > >> >> >> There seems to be a lot of work in flight, work with patches
> > and
> > > > >> review
> > > > >> >> >> that could/should get committed but now just pending KIPS,
> > work
> > > > >> without
> > > > >> >> KIP
> > > > >> >> >> but that is in trunk already (e.g. the new Consumer) that
> > would be
> > > > >> the
> > > > >> >> the
> > > > >> >> >> release but missing the KIP for the release...
> > > > >> >> >>
> > > > >> >> >> What does this mean for the 0.8.3 release? What are we
> trying
> > to
> > > > >> get out
> > > > >> >> >> and when?
> > > > >> >> >>
> > > > >> >> >> Also looking at
> > > > >> >> >>
> > > > >>
> > https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
> > > > >> >> >> there
> > > > >> >> >> seems to be things we are getting earlier (which is great of
> > > > >> course) so
> > > > >> >> are
> > > > >> >> >> we going to try to up the version and go with 0.9.0?
> > > > >> >> >>
> > > > >> >> >> 0.8.2.0 ended up getting very bloated and that delayed it
> much
> > > > >> longer
> > > > >> >> than
> > > > >> >> >> we had originally communicated to the community and want to
> > make
> > > > >> sure we
> > > > >> >> >> take that feedback from the community and try to improve
> upon
> > it.
> > > > >> >> >>
> > > > >> >> >> Thanks!
> > > > >> >> >>
> > > > >> >> >> ~ Joe Stein
> > > > >> >> >> - - - - - - - - - - - - - - - - -
> > > > >> >> >>
> > > > >> >> >>   http://www.stealth.ly
> > > > >> >> >> - - - - - - - - - - - - - - - - -
> > > > >> >> >>
> > > > >> >>
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> >
>

Re: 0.8.3 release plan

[Jun Rao <ju...@confluent.io>]

Since we have decided to only support security on the new clients, the
security feature will only be available after the new consumer is done. So,
for the next release, we probably should just target finishing the new
consumer as the main feature. If security can be done in the same release,
that's just a bonus.

Thanks,

Jun

On Thu, Mar 12, 2015 at 6:37 AM, Harsha <ka...@harsha.io> wrote:

> I would like to include ssl/sasl
> 1) Kafka-1684 (Patch posted for a review)
> 2) Kafka-1686 (Patch depends on kafka-1684)
> 3) Kafka-1688 (work is in progress)
> Thanks,
> Harsha
> On Thu, Mar 12, 2015, at 04:35 AM, Guozhang Wang wrote:
> > Gwen,
> >
> > Just for clarification, you were suggesting we should or should not
> > include
> > MM improvement in 0.8.3? I personally would prefer it (KAFKA-1650 and
> > KAFKA-1997) to go into 0.8.3.
> >
> > I see Joe has made a pass over the tickets and mark them 0.8.3. We can
> > probably do another pass and consider adding:
> >
> > 1) Purgatory improvement (KAFKA-1989).
> > 2) Compression improvement (KAFKA-527).
> > 3) Some unit test failures (KAFKA-1501, I think we are pretty close in
> > getting it fixed).
> > 4) any other tickets?
> >
> > Guozhang
> >
> > On Wed, Mar 11, 2015 at 9:40 PM, Jay Kreps <ja...@gmail.com> wrote:
> >
> > > With regard to mm, I was kind of assuming just based on the amount of
> work
> > > that that would go in for sure, but yeah I agree it is important.
> > >
> > > -Jay
> > >
> > > On Wed, Mar 11, 2015 at 9:39 PM, Jay Kreps <ja...@gmail.com>
> wrote:
> > >
> > > > What I was trying to say was let's do a real release whenever either
> > > > consumer or authn is done whichever happens first (or both if they
> can
> > > > happen close together)--not sure which is more likely to slip.
> > > >
> > > > WRT the beta thing I think the question for people is whether the
> beta
> > > > period was helpful or not in getting a more stable release? We could
> > > either
> > > > do a beta release again or we could just do a normal release and
> call the
> > > > consumer feature "experimental" or whatever...basically something to
> get
> > > it
> > > > in peoples hands before it is supposed to work perfectly and never
> change
> > > > again.
> > > >
> > > > -Jay
> > > >
> > > >
> > > > On Wed, Mar 11, 2015 at 9:27 PM, Gwen Shapira <gshapira@cloudera.com
> >
> > > > wrote:
> > > >
> > > >> So basically you are suggesting - lets do a beta release whenever we
> > > >> feel the new consumer is done?
> > > >>
> > > >> This can definitely work.
> > > >>
> > > >> I'd prefer holding for MM improvements too. IMO, its not just more
> > > >> improvements like flush() and compression optimization.
> > > >> Current MirrorMaker can lose data, which makes it pretty useless for
> > > >> its job. We hear lots of requests for robust MM from our customers,
> so
> > > >> I can imagine its pretty important to the Kafka community (unless I
> > > >> have a completely skewed sample).
> > > >>
> > > >> Gwen
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <ja...@gmail.com>
> wrote:
> > > >> > Yeah the real question is always what will we block on?
> > > >> >
> > > >> > I don't think we should try to hold back smaller changes. In this
> > > >> bucket I
> > > >> > would include most things you described: mm improvements, replica
> > > >> > assignment tool improvements, flush, purgatory improvements,
> > > compression
> > > >> > optimization, etc. Likely these will all get done in time as well
> as
> > > >> many
> > > >> > things that kind of pop up from users but probably aren't worth
> doing
> > > a
> > > >> > release for on their own. If one of them slips that fine. I also
> don't
> > > >> > think we should try to hold back work that is done if it isn't on
> a
> > > >> list.
> > > >> >
> > > >> > I would consider either SSL+SASL or the consumer worthy of a
> release
> > > on
> > > >> its
> > > >> > own. If they finish close to the same time that is great. We can
> maybe
> > > >> just
> > > >> > assess as these evolve where the other one is at and make a call
> > > >> whether it
> > > >> > will be one or both?
> > > >> >
> > > >> > -Jay
> > > >> >
> > > >> > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <
> gshapira@cloudera.com>
> > > >> wrote:
> > > >> >
> > > >> >> If we are going in terms of features, I can see the following
> > > features
> > > >> >> getting in in the next month or two:
> > > >> >>
> > > >> >> * New consumer
> > > >> >> * Improved Mirror Maker (I've seen tons of interest)
> > > >> >> * Centralized admin requests (aka KIP-4)
> > > >> >> * Nicer replica-reassignment tool
> > > >> >> * SSL (and perhaps also SASL)?
> > > >> >>
> > > >> >> I think this collection will make a nice release. Perhaps we can
> cap
> > > >> >> it there and focus (as a community) on getting these in, we can
> have
> > > a
> > > >> >> release without too much scope creep in the
> not-very-distant-future?
> > > >> >> Even just 3 out of these 5 will still make a nice incremental
> > > >> >> improvement.
> > > >> >>
> > > >> >> Gwen
> > > >> >>
> > > >> >>
> > > >> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <ja...@gmail.com>
> > > >> wrote:
> > > >> >> > Yeah I'd be in favor of a quicker, smaller release but I think
> as
> > > >> long as
> > > >> >> > we have these big things in flight we should probably keep the
> > > >> release
> > > >> >> > criteria feature-based rather than time-based, though (e.g.
> "when X
> > > >> >> works"
> > > >> >> > not "every other month).
> > > >> >> >
> > > >> >> > Ideally the next release would have at least a "beta" version
> of
> > > the
> > > >> new
> > > >> >> > consumer. I think having a new hunk of code like that
> available but
> > > >> >> marked
> > > >> >> > as "beta" is maybe a good way to go, as it gets it into peoples
> > > >> hands for
> > > >> >> > testing. This way we can declare the API not fully locked down
> > > until
> > > >> the
> > > >> >> > final release too, since mostly users only look at stuff after
> we
> > > >> release
> > > >> >> > it. Maybe we can try to construct a schedule around this?
> > > >> >> >
> > > >> >> > -Jay
> > > >> >> >
> > > >> >> >
> > > >> >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <
> joe.stein@stealth.ly>
> > > >> wrote:
> > > >> >> >
> > > >> >> >> There hasn't been any public discussion about the 0.8.3
> release
> > > >> plan.
> > > >> >> >>
> > > >> >> >> There seems to be a lot of work in flight, work with patches
> and
> > > >> review
> > > >> >> >> that could/should get committed but now just pending KIPS,
> work
> > > >> without
> > > >> >> KIP
> > > >> >> >> but that is in trunk already (e.g. the new Consumer) that
> would be
> > > >> the
> > > >> >> the
> > > >> >> >> release but missing the KIP for the release...
> > > >> >> >>
> > > >> >> >> What does this mean for the 0.8.3 release? What are we trying
> to
> > > >> get out
> > > >> >> >> and when?
> > > >> >> >>
> > > >> >> >> Also looking at
> > > >> >> >>
> > > >>
> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
> > > >> >> >> there
> > > >> >> >> seems to be things we are getting earlier (which is great of
> > > >> course) so
> > > >> >> are
> > > >> >> >> we going to try to up the version and go with 0.9.0?
> > > >> >> >>
> > > >> >> >> 0.8.2.0 ended up getting very bloated and that delayed it much
> > > >> longer
> > > >> >> than
> > > >> >> >> we had originally communicated to the community and want to
> make
> > > >> sure we
> > > >> >> >> take that feedback from the community and try to improve upon
> it.
> > > >> >> >>
> > > >> >> >> Thanks!
> > > >> >> >>
> > > >> >> >> ~ Joe Stein
> > > >> >> >> - - - - - - - - - - - - - - - - -
> > > >> >> >>
> > > >> >> >>   http://www.stealth.ly
> > > >> >> >> - - - - - - - - - - - - - - - - -
> > > >> >> >>
> > > >> >>
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
>

Re: 0.8.3 release plan

[Harsha <ka...@harsha.io>]

I would like to include ssl/sasl
1) Kafka-1684 (Patch posted for a review)
2) Kafka-1686 (Patch depends on kafka-1684)
3) Kafka-1688 (work is in progress)
Thanks,
Harsha
On Thu, Mar 12, 2015, at 04:35 AM, Guozhang Wang wrote:
> Gwen,
> 
> Just for clarification, you were suggesting we should or should not
> include
> MM improvement in 0.8.3? I personally would prefer it (KAFKA-1650 and
> KAFKA-1997) to go into 0.8.3.
> 
> I see Joe has made a pass over the tickets and mark them 0.8.3. We can
> probably do another pass and consider adding:
> 
> 1) Purgatory improvement (KAFKA-1989).
> 2) Compression improvement (KAFKA-527).
> 3) Some unit test failures (KAFKA-1501, I think we are pretty close in
> getting it fixed).
> 4) any other tickets?
> 
> Guozhang
> 
> On Wed, Mar 11, 2015 at 9:40 PM, Jay Kreps <ja...@gmail.com> wrote:
> 
> > With regard to mm, I was kind of assuming just based on the amount of work
> > that that would go in for sure, but yeah I agree it is important.
> >
> > -Jay
> >
> > On Wed, Mar 11, 2015 at 9:39 PM, Jay Kreps <ja...@gmail.com> wrote:
> >
> > > What I was trying to say was let's do a real release whenever either
> > > consumer or authn is done whichever happens first (or both if they can
> > > happen close together)--not sure which is more likely to slip.
> > >
> > > WRT the beta thing I think the question for people is whether the beta
> > > period was helpful or not in getting a more stable release? We could
> > either
> > > do a beta release again or we could just do a normal release and call the
> > > consumer feature "experimental" or whatever...basically something to get
> > it
> > > in peoples hands before it is supposed to work perfectly and never change
> > > again.
> > >
> > > -Jay
> > >
> > >
> > > On Wed, Mar 11, 2015 at 9:27 PM, Gwen Shapira <gs...@cloudera.com>
> > > wrote:
> > >
> > >> So basically you are suggesting - lets do a beta release whenever we
> > >> feel the new consumer is done?
> > >>
> > >> This can definitely work.
> > >>
> > >> I'd prefer holding for MM improvements too. IMO, its not just more
> > >> improvements like flush() and compression optimization.
> > >> Current MirrorMaker can lose data, which makes it pretty useless for
> > >> its job. We hear lots of requests for robust MM from our customers, so
> > >> I can imagine its pretty important to the Kafka community (unless I
> > >> have a completely skewed sample).
> > >>
> > >> Gwen
> > >>
> > >>
> > >>
> > >> On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <ja...@gmail.com> wrote:
> > >> > Yeah the real question is always what will we block on?
> > >> >
> > >> > I don't think we should try to hold back smaller changes. In this
> > >> bucket I
> > >> > would include most things you described: mm improvements, replica
> > >> > assignment tool improvements, flush, purgatory improvements,
> > compression
> > >> > optimization, etc. Likely these will all get done in time as well as
> > >> many
> > >> > things that kind of pop up from users but probably aren't worth doing
> > a
> > >> > release for on their own. If one of them slips that fine. I also don't
> > >> > think we should try to hold back work that is done if it isn't on a
> > >> list.
> > >> >
> > >> > I would consider either SSL+SASL or the consumer worthy of a release
> > on
> > >> its
> > >> > own. If they finish close to the same time that is great. We can maybe
> > >> just
> > >> > assess as these evolve where the other one is at and make a call
> > >> whether it
> > >> > will be one or both?
> > >> >
> > >> > -Jay
> > >> >
> > >> > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <gs...@cloudera.com>
> > >> wrote:
> > >> >
> > >> >> If we are going in terms of features, I can see the following
> > features
> > >> >> getting in in the next month or two:
> > >> >>
> > >> >> * New consumer
> > >> >> * Improved Mirror Maker (I've seen tons of interest)
> > >> >> * Centralized admin requests (aka KIP-4)
> > >> >> * Nicer replica-reassignment tool
> > >> >> * SSL (and perhaps also SASL)?
> > >> >>
> > >> >> I think this collection will make a nice release. Perhaps we can cap
> > >> >> it there and focus (as a community) on getting these in, we can have
> > a
> > >> >> release without too much scope creep in the not-very-distant-future?
> > >> >> Even just 3 out of these 5 will still make a nice incremental
> > >> >> improvement.
> > >> >>
> > >> >> Gwen
> > >> >>
> > >> >>
> > >> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <ja...@gmail.com>
> > >> wrote:
> > >> >> > Yeah I'd be in favor of a quicker, smaller release but I think as
> > >> long as
> > >> >> > we have these big things in flight we should probably keep the
> > >> release
> > >> >> > criteria feature-based rather than time-based, though (e.g. "when X
> > >> >> works"
> > >> >> > not "every other month).
> > >> >> >
> > >> >> > Ideally the next release would have at least a "beta" version of
> > the
> > >> new
> > >> >> > consumer. I think having a new hunk of code like that available but
> > >> >> marked
> > >> >> > as "beta" is maybe a good way to go, as it gets it into peoples
> > >> hands for
> > >> >> > testing. This way we can declare the API not fully locked down
> > until
> > >> the
> > >> >> > final release too, since mostly users only look at stuff after we
> > >> release
> > >> >> > it. Maybe we can try to construct a schedule around this?
> > >> >> >
> > >> >> > -Jay
> > >> >> >
> > >> >> >
> > >> >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <jo...@stealth.ly>
> > >> wrote:
> > >> >> >
> > >> >> >> There hasn't been any public discussion about the 0.8.3 release
> > >> plan.
> > >> >> >>
> > >> >> >> There seems to be a lot of work in flight, work with patches and
> > >> review
> > >> >> >> that could/should get committed but now just pending KIPS, work
> > >> without
> > >> >> KIP
> > >> >> >> but that is in trunk already (e.g. the new Consumer) that would be
> > >> the
> > >> >> the
> > >> >> >> release but missing the KIP for the release...
> > >> >> >>
> > >> >> >> What does this mean for the 0.8.3 release? What are we trying to
> > >> get out
> > >> >> >> and when?
> > >> >> >>
> > >> >> >> Also looking at
> > >> >> >>
> > >> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
> > >> >> >> there
> > >> >> >> seems to be things we are getting earlier (which is great of
> > >> course) so
> > >> >> are
> > >> >> >> we going to try to up the version and go with 0.9.0?
> > >> >> >>
> > >> >> >> 0.8.2.0 ended up getting very bloated and that delayed it much
> > >> longer
> > >> >> than
> > >> >> >> we had originally communicated to the community and want to make
> > >> sure we
> > >> >> >> take that feedback from the community and try to improve upon it.
> > >> >> >>
> > >> >> >> Thanks!
> > >> >> >>
> > >> >> >> ~ Joe Stein
> > >> >> >> - - - - - - - - - - - - - - - - -
> > >> >> >>
> > >> >> >>   http://www.stealth.ly
> > >> >> >> - - - - - - - - - - - - - - - - -
> > >> >> >>
> > >> >>
> > >>
> > >
> > >
> >
> 
> 
> 
> -- 
> -- Guozhang

Re: 0.8.3 release plan

[Guozhang Wang <wa...@gmail.com>]

Gwen,

Just for clarification, you were suggesting we should or should not include
MM improvement in 0.8.3? I personally would prefer it (KAFKA-1650 and
KAFKA-1997) to go into 0.8.3.

I see Joe has made a pass over the tickets and mark them 0.8.3. We can
probably do another pass and consider adding:

1) Purgatory improvement (KAFKA-1989).
2) Compression improvement (KAFKA-527).
3) Some unit test failures (KAFKA-1501, I think we are pretty close in
getting it fixed).
4) any other tickets?

Guozhang

On Wed, Mar 11, 2015 at 9:40 PM, Jay Kreps <ja...@gmail.com> wrote:

> With regard to mm, I was kind of assuming just based on the amount of work
> that that would go in for sure, but yeah I agree it is important.
>
> -Jay
>
> On Wed, Mar 11, 2015 at 9:39 PM, Jay Kreps <ja...@gmail.com> wrote:
>
> > What I was trying to say was let's do a real release whenever either
> > consumer or authn is done whichever happens first (or both if they can
> > happen close together)--not sure which is more likely to slip.
> >
> > WRT the beta thing I think the question for people is whether the beta
> > period was helpful or not in getting a more stable release? We could
> either
> > do a beta release again or we could just do a normal release and call the
> > consumer feature "experimental" or whatever...basically something to get
> it
> > in peoples hands before it is supposed to work perfectly and never change
> > again.
> >
> > -Jay
> >
> >
> > On Wed, Mar 11, 2015 at 9:27 PM, Gwen Shapira <gs...@cloudera.com>
> > wrote:
> >
> >> So basically you are suggesting - lets do a beta release whenever we
> >> feel the new consumer is done?
> >>
> >> This can definitely work.
> >>
> >> I'd prefer holding for MM improvements too. IMO, its not just more
> >> improvements like flush() and compression optimization.
> >> Current MirrorMaker can lose data, which makes it pretty useless for
> >> its job. We hear lots of requests for robust MM from our customers, so
> >> I can imagine its pretty important to the Kafka community (unless I
> >> have a completely skewed sample).
> >>
> >> Gwen
> >>
> >>
> >>
> >> On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <ja...@gmail.com> wrote:
> >> > Yeah the real question is always what will we block on?
> >> >
> >> > I don't think we should try to hold back smaller changes. In this
> >> bucket I
> >> > would include most things you described: mm improvements, replica
> >> > assignment tool improvements, flush, purgatory improvements,
> compression
> >> > optimization, etc. Likely these will all get done in time as well as
> >> many
> >> > things that kind of pop up from users but probably aren't worth doing
> a
> >> > release for on their own. If one of them slips that fine. I also don't
> >> > think we should try to hold back work that is done if it isn't on a
> >> list.
> >> >
> >> > I would consider either SSL+SASL or the consumer worthy of a release
> on
> >> its
> >> > own. If they finish close to the same time that is great. We can maybe
> >> just
> >> > assess as these evolve where the other one is at and make a call
> >> whether it
> >> > will be one or both?
> >> >
> >> > -Jay
> >> >
> >> > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <gs...@cloudera.com>
> >> wrote:
> >> >
> >> >> If we are going in terms of features, I can see the following
> features
> >> >> getting in in the next month or two:
> >> >>
> >> >> * New consumer
> >> >> * Improved Mirror Maker (I've seen tons of interest)
> >> >> * Centralized admin requests (aka KIP-4)
> >> >> * Nicer replica-reassignment tool
> >> >> * SSL (and perhaps also SASL)?
> >> >>
> >> >> I think this collection will make a nice release. Perhaps we can cap
> >> >> it there and focus (as a community) on getting these in, we can have
> a
> >> >> release without too much scope creep in the not-very-distant-future?
> >> >> Even just 3 out of these 5 will still make a nice incremental
> >> >> improvement.
> >> >>
> >> >> Gwen
> >> >>
> >> >>
> >> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <ja...@gmail.com>
> >> wrote:
> >> >> > Yeah I'd be in favor of a quicker, smaller release but I think as
> >> long as
> >> >> > we have these big things in flight we should probably keep the
> >> release
> >> >> > criteria feature-based rather than time-based, though (e.g. "when X
> >> >> works"
> >> >> > not "every other month).
> >> >> >
> >> >> > Ideally the next release would have at least a "beta" version of
> the
> >> new
> >> >> > consumer. I think having a new hunk of code like that available but
> >> >> marked
> >> >> > as "beta" is maybe a good way to go, as it gets it into peoples
> >> hands for
> >> >> > testing. This way we can declare the API not fully locked down
> until
> >> the
> >> >> > final release too, since mostly users only look at stuff after we
> >> release
> >> >> > it. Maybe we can try to construct a schedule around this?
> >> >> >
> >> >> > -Jay
> >> >> >
> >> >> >
> >> >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <jo...@stealth.ly>
> >> wrote:
> >> >> >
> >> >> >> There hasn't been any public discussion about the 0.8.3 release
> >> plan.
> >> >> >>
> >> >> >> There seems to be a lot of work in flight, work with patches and
> >> review
> >> >> >> that could/should get committed but now just pending KIPS, work
> >> without
> >> >> KIP
> >> >> >> but that is in trunk already (e.g. the new Consumer) that would be
> >> the
> >> >> the
> >> >> >> release but missing the KIP for the release...
> >> >> >>
> >> >> >> What does this mean for the 0.8.3 release? What are we trying to
> >> get out
> >> >> >> and when?
> >> >> >>
> >> >> >> Also looking at
> >> >> >>
> >> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
> >> >> >> there
> >> >> >> seems to be things we are getting earlier (which is great of
> >> course) so
> >> >> are
> >> >> >> we going to try to up the version and go with 0.9.0?
> >> >> >>
> >> >> >> 0.8.2.0 ended up getting very bloated and that delayed it much
> >> longer
> >> >> than
> >> >> >> we had originally communicated to the community and want to make
> >> sure we
> >> >> >> take that feedback from the community and try to improve upon it.
> >> >> >>
> >> >> >> Thanks!
> >> >> >>
> >> >> >> ~ Joe Stein
> >> >> >> - - - - - - - - - - - - - - - - -
> >> >> >>
> >> >> >>   http://www.stealth.ly
> >> >> >> - - - - - - - - - - - - - - - - -
> >> >> >>
> >> >>
> >>
> >
> >
>



-- 
-- Guozhang

Re: 0.8.3 release plan

[Jay Kreps <ja...@gmail.com>]

With regard to mm, I was kind of assuming just based on the amount of work
that that would go in for sure, but yeah I agree it is important.

-Jay

On Wed, Mar 11, 2015 at 9:39 PM, Jay Kreps <ja...@gmail.com> wrote:

> What I was trying to say was let's do a real release whenever either
> consumer or authn is done whichever happens first (or both if they can
> happen close together)--not sure which is more likely to slip.
>
> WRT the beta thing I think the question for people is whether the beta
> period was helpful or not in getting a more stable release? We could either
> do a beta release again or we could just do a normal release and call the
> consumer feature "experimental" or whatever...basically something to get it
> in peoples hands before it is supposed to work perfectly and never change
> again.
>
> -Jay
>
>
> On Wed, Mar 11, 2015 at 9:27 PM, Gwen Shapira <gs...@cloudera.com>
> wrote:
>
>> So basically you are suggesting - lets do a beta release whenever we
>> feel the new consumer is done?
>>
>> This can definitely work.
>>
>> I'd prefer holding for MM improvements too. IMO, its not just more
>> improvements like flush() and compression optimization.
>> Current MirrorMaker can lose data, which makes it pretty useless for
>> its job. We hear lots of requests for robust MM from our customers, so
>> I can imagine its pretty important to the Kafka community (unless I
>> have a completely skewed sample).
>>
>> Gwen
>>
>>
>>
>> On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <ja...@gmail.com> wrote:
>> > Yeah the real question is always what will we block on?
>> >
>> > I don't think we should try to hold back smaller changes. In this
>> bucket I
>> > would include most things you described: mm improvements, replica
>> > assignment tool improvements, flush, purgatory improvements, compression
>> > optimization, etc. Likely these will all get done in time as well as
>> many
>> > things that kind of pop up from users but probably aren't worth doing a
>> > release for on their own. If one of them slips that fine. I also don't
>> > think we should try to hold back work that is done if it isn't on a
>> list.
>> >
>> > I would consider either SSL+SASL or the consumer worthy of a release on
>> its
>> > own. If they finish close to the same time that is great. We can maybe
>> just
>> > assess as these evolve where the other one is at and make a call
>> whether it
>> > will be one or both?
>> >
>> > -Jay
>> >
>> > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <gs...@cloudera.com>
>> wrote:
>> >
>> >> If we are going in terms of features, I can see the following features
>> >> getting in in the next month or two:
>> >>
>> >> * New consumer
>> >> * Improved Mirror Maker (I've seen tons of interest)
>> >> * Centralized admin requests (aka KIP-4)
>> >> * Nicer replica-reassignment tool
>> >> * SSL (and perhaps also SASL)?
>> >>
>> >> I think this collection will make a nice release. Perhaps we can cap
>> >> it there and focus (as a community) on getting these in, we can have a
>> >> release without too much scope creep in the not-very-distant-future?
>> >> Even just 3 out of these 5 will still make a nice incremental
>> >> improvement.
>> >>
>> >> Gwen
>> >>
>> >>
>> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <ja...@gmail.com>
>> wrote:
>> >> > Yeah I'd be in favor of a quicker, smaller release but I think as
>> long as
>> >> > we have these big things in flight we should probably keep the
>> release
>> >> > criteria feature-based rather than time-based, though (e.g. "when X
>> >> works"
>> >> > not "every other month).
>> >> >
>> >> > Ideally the next release would have at least a "beta" version of the
>> new
>> >> > consumer. I think having a new hunk of code like that available but
>> >> marked
>> >> > as "beta" is maybe a good way to go, as it gets it into peoples
>> hands for
>> >> > testing. This way we can declare the API not fully locked down until
>> the
>> >> > final release too, since mostly users only look at stuff after we
>> release
>> >> > it. Maybe we can try to construct a schedule around this?
>> >> >
>> >> > -Jay
>> >> >
>> >> >
>> >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <jo...@stealth.ly>
>> wrote:
>> >> >
>> >> >> There hasn't been any public discussion about the 0.8.3 release
>> plan.
>> >> >>
>> >> >> There seems to be a lot of work in flight, work with patches and
>> review
>> >> >> that could/should get committed but now just pending KIPS, work
>> without
>> >> KIP
>> >> >> but that is in trunk already (e.g. the new Consumer) that would be
>> the
>> >> the
>> >> >> release but missing the KIP for the release...
>> >> >>
>> >> >> What does this mean for the 0.8.3 release? What are we trying to
>> get out
>> >> >> and when?
>> >> >>
>> >> >> Also looking at
>> >> >>
>> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
>> >> >> there
>> >> >> seems to be things we are getting earlier (which is great of
>> course) so
>> >> are
>> >> >> we going to try to up the version and go with 0.9.0?
>> >> >>
>> >> >> 0.8.2.0 ended up getting very bloated and that delayed it much
>> longer
>> >> than
>> >> >> we had originally communicated to the community and want to make
>> sure we
>> >> >> take that feedback from the community and try to improve upon it.
>> >> >>
>> >> >> Thanks!
>> >> >>
>> >> >> ~ Joe Stein
>> >> >> - - - - - - - - - - - - - - - - -
>> >> >>
>> >> >>   http://www.stealth.ly
>> >> >> - - - - - - - - - - - - - - - - -
>> >> >>
>> >>
>>
>
>

Re: 0.8.3 release plan

[Jay Kreps <ja...@gmail.com>]

What I was trying to say was let's do a real release whenever either
consumer or authn is done whichever happens first (or both if they can
happen close together)--not sure which is more likely to slip.

WRT the beta thing I think the question for people is whether the beta
period was helpful or not in getting a more stable release? We could either
do a beta release again or we could just do a normal release and call the
consumer feature "experimental" or whatever...basically something to get it
in peoples hands before it is supposed to work perfectly and never change
again.

-Jay


On Wed, Mar 11, 2015 at 9:27 PM, Gwen Shapira <gs...@cloudera.com> wrote:

> So basically you are suggesting - lets do a beta release whenever we
> feel the new consumer is done?
>
> This can definitely work.
>
> I'd prefer holding for MM improvements too. IMO, its not just more
> improvements like flush() and compression optimization.
> Current MirrorMaker can lose data, which makes it pretty useless for
> its job. We hear lots of requests for robust MM from our customers, so
> I can imagine its pretty important to the Kafka community (unless I
> have a completely skewed sample).
>
> Gwen
>
>
>
> On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <ja...@gmail.com> wrote:
> > Yeah the real question is always what will we block on?
> >
> > I don't think we should try to hold back smaller changes. In this bucket
> I
> > would include most things you described: mm improvements, replica
> > assignment tool improvements, flush, purgatory improvements, compression
> > optimization, etc. Likely these will all get done in time as well as many
> > things that kind of pop up from users but probably aren't worth doing a
> > release for on their own. If one of them slips that fine. I also don't
> > think we should try to hold back work that is done if it isn't on a list.
> >
> > I would consider either SSL+SASL or the consumer worthy of a release on
> its
> > own. If they finish close to the same time that is great. We can maybe
> just
> > assess as these evolve where the other one is at and make a call whether
> it
> > will be one or both?
> >
> > -Jay
> >
> > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <gs...@cloudera.com>
> wrote:
> >
> >> If we are going in terms of features, I can see the following features
> >> getting in in the next month or two:
> >>
> >> * New consumer
> >> * Improved Mirror Maker (I've seen tons of interest)
> >> * Centralized admin requests (aka KIP-4)
> >> * Nicer replica-reassignment tool
> >> * SSL (and perhaps also SASL)?
> >>
> >> I think this collection will make a nice release. Perhaps we can cap
> >> it there and focus (as a community) on getting these in, we can have a
> >> release without too much scope creep in the not-very-distant-future?
> >> Even just 3 out of these 5 will still make a nice incremental
> >> improvement.
> >>
> >> Gwen
> >>
> >>
> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <ja...@gmail.com> wrote:
> >> > Yeah I'd be in favor of a quicker, smaller release but I think as
> long as
> >> > we have these big things in flight we should probably keep the release
> >> > criteria feature-based rather than time-based, though (e.g. "when X
> >> works"
> >> > not "every other month).
> >> >
> >> > Ideally the next release would have at least a "beta" version of the
> new
> >> > consumer. I think having a new hunk of code like that available but
> >> marked
> >> > as "beta" is maybe a good way to go, as it gets it into peoples hands
> for
> >> > testing. This way we can declare the API not fully locked down until
> the
> >> > final release too, since mostly users only look at stuff after we
> release
> >> > it. Maybe we can try to construct a schedule around this?
> >> >
> >> > -Jay
> >> >
> >> >
> >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <jo...@stealth.ly>
> wrote:
> >> >
> >> >> There hasn't been any public discussion about the 0.8.3 release plan.
> >> >>
> >> >> There seems to be a lot of work in flight, work with patches and
> review
> >> >> that could/should get committed but now just pending KIPS, work
> without
> >> KIP
> >> >> but that is in trunk already (e.g. the new Consumer) that would be
> the
> >> the
> >> >> release but missing the KIP for the release...
> >> >>
> >> >> What does this mean for the 0.8.3 release? What are we trying to get
> out
> >> >> and when?
> >> >>
> >> >> Also looking at
> >> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
> >> >> there
> >> >> seems to be things we are getting earlier (which is great of course)
> so
> >> are
> >> >> we going to try to up the version and go with 0.9.0?
> >> >>
> >> >> 0.8.2.0 ended up getting very bloated and that delayed it much longer
> >> than
> >> >> we had originally communicated to the community and want to make
> sure we
> >> >> take that feedback from the community and try to improve upon it.
> >> >>
> >> >> Thanks!
> >> >>
> >> >> ~ Joe Stein
> >> >> - - - - - - - - - - - - - - - - -
> >> >>
> >> >>   http://www.stealth.ly
> >> >> - - - - - - - - - - - - - - - - -
> >> >>
> >>
>

Re: 0.8.3 release plan

[Gwen Shapira <gs...@cloudera.com>]

So basically you are suggesting - lets do a beta release whenever we
feel the new consumer is done?

This can definitely work.

I'd prefer holding for MM improvements too. IMO, its not just more
improvements like flush() and compression optimization.
Current MirrorMaker can lose data, which makes it pretty useless for
its job. We hear lots of requests for robust MM from our customers, so
I can imagine its pretty important to the Kafka community (unless I
have a completely skewed sample).

Gwen



On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <ja...@gmail.com> wrote:
> Yeah the real question is always what will we block on?
>
> I don't think we should try to hold back smaller changes. In this bucket I
> would include most things you described: mm improvements, replica
> assignment tool improvements, flush, purgatory improvements, compression
> optimization, etc. Likely these will all get done in time as well as many
> things that kind of pop up from users but probably aren't worth doing a
> release for on their own. If one of them slips that fine. I also don't
> think we should try to hold back work that is done if it isn't on a list.
>
> I would consider either SSL+SASL or the consumer worthy of a release on its
> own. If they finish close to the same time that is great. We can maybe just
> assess as these evolve where the other one is at and make a call whether it
> will be one or both?
>
> -Jay
>
> On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <gs...@cloudera.com> wrote:
>
>> If we are going in terms of features, I can see the following features
>> getting in in the next month or two:
>>
>> * New consumer
>> * Improved Mirror Maker (I've seen tons of interest)
>> * Centralized admin requests (aka KIP-4)
>> * Nicer replica-reassignment tool
>> * SSL (and perhaps also SASL)?
>>
>> I think this collection will make a nice release. Perhaps we can cap
>> it there and focus (as a community) on getting these in, we can have a
>> release without too much scope creep in the not-very-distant-future?
>> Even just 3 out of these 5 will still make a nice incremental
>> improvement.
>>
>> Gwen
>>
>>
>> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <ja...@gmail.com> wrote:
>> > Yeah I'd be in favor of a quicker, smaller release but I think as long as
>> > we have these big things in flight we should probably keep the release
>> > criteria feature-based rather than time-based, though (e.g. "when X
>> works"
>> > not "every other month).
>> >
>> > Ideally the next release would have at least a "beta" version of the new
>> > consumer. I think having a new hunk of code like that available but
>> marked
>> > as "beta" is maybe a good way to go, as it gets it into peoples hands for
>> > testing. This way we can declare the API not fully locked down until the
>> > final release too, since mostly users only look at stuff after we release
>> > it. Maybe we can try to construct a schedule around this?
>> >
>> > -Jay
>> >
>> >
>> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <jo...@stealth.ly> wrote:
>> >
>> >> There hasn't been any public discussion about the 0.8.3 release plan.
>> >>
>> >> There seems to be a lot of work in flight, work with patches and review
>> >> that could/should get committed but now just pending KIPS, work without
>> KIP
>> >> but that is in trunk already (e.g. the new Consumer) that would be the
>> the
>> >> release but missing the KIP for the release...
>> >>
>> >> What does this mean for the 0.8.3 release? What are we trying to get out
>> >> and when?
>> >>
>> >> Also looking at
>> >> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
>> >> there
>> >> seems to be things we are getting earlier (which is great of course) so
>> are
>> >> we going to try to up the version and go with 0.9.0?
>> >>
>> >> 0.8.2.0 ended up getting very bloated and that delayed it much longer
>> than
>> >> we had originally communicated to the community and want to make sure we
>> >> take that feedback from the community and try to improve upon it.
>> >>
>> >> Thanks!
>> >>
>> >> ~ Joe Stein
>> >> - - - - - - - - - - - - - - - - -
>> >>
>> >>   http://www.stealth.ly
>> >> - - - - - - - - - - - - - - - - -
>> >>
>>

Re: 0.8.3 release plan

[Jay Kreps <ja...@gmail.com>]

Yeah the real question is always what will we block on?

I don't think we should try to hold back smaller changes. In this bucket I
would include most things you described: mm improvements, replica
assignment tool improvements, flush, purgatory improvements, compression
optimization, etc. Likely these will all get done in time as well as many
things that kind of pop up from users but probably aren't worth doing a
release for on their own. If one of them slips that fine. I also don't
think we should try to hold back work that is done if it isn't on a list.

I would consider either SSL+SASL or the consumer worthy of a release on its
own. If they finish close to the same time that is great. We can maybe just
assess as these evolve where the other one is at and make a call whether it
will be one or both?

-Jay

On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <gs...@cloudera.com> wrote:

> If we are going in terms of features, I can see the following features
> getting in in the next month or two:
>
> * New consumer
> * Improved Mirror Maker (I've seen tons of interest)
> * Centralized admin requests (aka KIP-4)
> * Nicer replica-reassignment tool
> * SSL (and perhaps also SASL)?
>
> I think this collection will make a nice release. Perhaps we can cap
> it there and focus (as a community) on getting these in, we can have a
> release without too much scope creep in the not-very-distant-future?
> Even just 3 out of these 5 will still make a nice incremental
> improvement.
>
> Gwen
>
>
> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <ja...@gmail.com> wrote:
> > Yeah I'd be in favor of a quicker, smaller release but I think as long as
> > we have these big things in flight we should probably keep the release
> > criteria feature-based rather than time-based, though (e.g. "when X
> works"
> > not "every other month).
> >
> > Ideally the next release would have at least a "beta" version of the new
> > consumer. I think having a new hunk of code like that available but
> marked
> > as "beta" is maybe a good way to go, as it gets it into peoples hands for
> > testing. This way we can declare the API not fully locked down until the
> > final release too, since mostly users only look at stuff after we release
> > it. Maybe we can try to construct a schedule around this?
> >
> > -Jay
> >
> >
> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <jo...@stealth.ly> wrote:
> >
> >> There hasn't been any public discussion about the 0.8.3 release plan.
> >>
> >> There seems to be a lot of work in flight, work with patches and review
> >> that could/should get committed but now just pending KIPS, work without
> KIP
> >> but that is in trunk already (e.g. the new Consumer) that would be the
> the
> >> release but missing the KIP for the release...
> >>
> >> What does this mean for the 0.8.3 release? What are we trying to get out
> >> and when?
> >>
> >> Also looking at
> >> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
> >> there
> >> seems to be things we are getting earlier (which is great of course) so
> are
> >> we going to try to up the version and go with 0.9.0?
> >>
> >> 0.8.2.0 ended up getting very bloated and that delayed it much longer
> than
> >> we had originally communicated to the community and want to make sure we
> >> take that feedback from the community and try to improve upon it.
> >>
> >> Thanks!
> >>
> >> ~ Joe Stein
> >> - - - - - - - - - - - - - - - - -
> >>
> >>   http://www.stealth.ly
> >> - - - - - - - - - - - - - - - - -
> >>
>

Re: 0.8.3 release plan

[Gwen Shapira <gs...@cloudera.com>]

If we are going in terms of features, I can see the following features
getting in in the next month or two:

* New consumer
* Improved Mirror Maker (I've seen tons of interest)
* Centralized admin requests (aka KIP-4)
* Nicer replica-reassignment tool
* SSL (and perhaps also SASL)?

I think this collection will make a nice release. Perhaps we can cap
it there and focus (as a community) on getting these in, we can have a
release without too much scope creep in the not-very-distant-future?
Even just 3 out of these 5 will still make a nice incremental
improvement.

Gwen


On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <ja...@gmail.com> wrote:
> Yeah I'd be in favor of a quicker, smaller release but I think as long as
> we have these big things in flight we should probably keep the release
> criteria feature-based rather than time-based, though (e.g. "when X works"
> not "every other month).
>
> Ideally the next release would have at least a "beta" version of the new
> consumer. I think having a new hunk of code like that available but marked
> as "beta" is maybe a good way to go, as it gets it into peoples hands for
> testing. This way we can declare the API not fully locked down until the
> final release too, since mostly users only look at stuff after we release
> it. Maybe we can try to construct a schedule around this?
>
> -Jay
>
>
> On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <jo...@stealth.ly> wrote:
>
>> There hasn't been any public discussion about the 0.8.3 release plan.
>>
>> There seems to be a lot of work in flight, work with patches and review
>> that could/should get committed but now just pending KIPS, work without KIP
>> but that is in trunk already (e.g. the new Consumer) that would be the the
>> release but missing the KIP for the release...
>>
>> What does this mean for the 0.8.3 release? What are we trying to get out
>> and when?
>>
>> Also looking at
>> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
>> there
>> seems to be things we are getting earlier (which is great of course) so are
>> we going to try to up the version and go with 0.9.0?
>>
>> 0.8.2.0 ended up getting very bloated and that delayed it much longer than
>> we had originally communicated to the community and want to make sure we
>> take that feedback from the community and try to improve upon it.
>>
>> Thanks!
>>
>> ~ Joe Stein
>> - - - - - - - - - - - - - - - - -
>>
>>   http://www.stealth.ly
>> - - - - - - - - - - - - - - - - -
>>

Re: 0.8.3 release plan

[Jay Kreps <ja...@gmail.com>]

Yeah I'd be in favor of a quicker, smaller release but I think as long as
we have these big things in flight we should probably keep the release
criteria feature-based rather than time-based, though (e.g. "when X works"
not "every other month).

Ideally the next release would have at least a "beta" version of the new
consumer. I think having a new hunk of code like that available but marked
as "beta" is maybe a good way to go, as it gets it into peoples hands for
testing. This way we can declare the API not fully locked down until the
final release too, since mostly users only look at stuff after we release
it. Maybe we can try to construct a schedule around this?

-Jay


On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <jo...@stealth.ly> wrote:

> There hasn't been any public discussion about the 0.8.3 release plan.
>
> There seems to be a lot of work in flight, work with patches and review
> that could/should get committed but now just pending KIPS, work without KIP
> but that is in trunk already (e.g. the new Consumer) that would be the the
> release but missing the KIP for the release...
>
> What does this mean for the 0.8.3 release? What are we trying to get out
> and when?
>
> Also looking at
> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
> there
> seems to be things we are getting earlier (which is great of course) so are
> we going to try to up the version and go with 0.9.0?
>
> 0.8.2.0 ended up getting very bloated and that delayed it much longer than
> we had originally communicated to the community and want to make sure we
> take that feedback from the community and try to improve upon it.
>
> Thanks!
>
> ~ Joe Stein
> - - - - - - - - - - - - - - - - -
>
>   http://www.stealth.ly
> - - - - - - - - - - - - - - - - -
>