Zookeeper with SSL release date

[ma...@free.fr]

Hi,

I can see that Zookeeper supports SSL in version 3.5.1-alpha. When SSL support will be finally released so that it can be used in production ?

Thanks

Re: Zookeeper with SSL release date

[Jason Rosenberg <jb...@squareup.com>]

By the way, would someone formally describe what is meant by the "re-config
feature"?

On Wed, Mar 16, 2016 at 3:14 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> So, it would seem sensible to me to have a release where all features are
> stable, except where noted.  E.g. mark certain features as only 'alpha
> quality', e.g. the 're-config feature'.  (I assume it's possible to happily
> use 3.5.X without exercising the unstable re-config bits?).
>
> There's precedent for doing this sort of thing in other projects, e.g. in
> Kafka, they've had several release where a new "Consumer API" is shipped
> that is available for beta-testing, but you can still just use the older
> stable consumer api, etc.
>
> Jason
>
> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti <
> powellm79@yahoo.com.invalid> wrote:
>
>> Hi Doug,
>> Is 3.5 being an alpha release preventing you from using it?. Or have you
>> run into issues with it?. In general perhaps ZK 3.5 being labeled as alpha
>> might not be fair, since it is far more stable then what most people
>> associate an alpha release to be.
>> Perhaps if you do not use re-config feature may be it will just work for
>> you?.
>> There are many examples of 3.5.X being used in productions from my
>> limited knowledge.
>> ThanksPowell.
>>
>>     On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>> fpj@apache.org> wrote:
>>
>>
>>  None of us expected the reconfig changes to take this long to stabilize.
>> Until we get there, I don't think we can do anything else with 3.5. The
>> best bet we have is to work harder to bring 3.5 into a stable rather than
>> trying to work around it.
>>
>> There are lots of people interested in seeing 3.5 stable, and if we get
>> everyone to contribute more patches and code reviews, we should be able to
>> do it sooner. After all, it is a community based effort, so the community
>> shouldn't rely on just 2-3 people doing the work.
>>
>> -Flavio
>>
>> > On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>> >
>> > Doug, I forgot to respond to your question about 3.4.  Since 3.4 is the
>> > stable maintenance line, we are very conservative about back-porting to
>> > it.  Our policy is to limit back-ports to critical bug fixes and not
>> > introduce any new features in the 3.4 line.  This is a matter of
>> managing
>> > risk.
>> >
>> > Jason, your question about release cadence is a fair one.  If it's any
>> > consolation, we are now taking the approach of trying to limit the scope
>> > of anything new introduced in 3.5 too.  That would allow us to focus on
>> > stabilization: resolving blocker bugs and freezing public APIs.  I think
>> > this will help us accelerate the releases into beta and eventual GA.
>> >
>> > I am new to ZooKeeper release management, so I'd like to hear thoughts
>> > from more experienced committers and PMC members about your proposal to
>> > try to cut a stable release for a limited subset of what is in
>> branch-3.5
>> > now.  My instinct is that it would be challenging to cherry-pick out
>> > pieces of branch-3.5 piecemeal at this point.  This would become another
>> > release line for an already resource-constrained volunteer staff to
>> > manage.  I'd prefer to dedicate those limited resources to overall 3.5
>> > stabilization.  Also, a 3.5 release in which certain features "vanished"
>> > because of not meeting some stability criteria would be undesirable.
>> >
>> > --Chris Nauroth
>> >
>> >
>> >
>> >
>> > On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>> >
>> >> Chris,
>> >>
>> >> Can you say whether some parts of 3.5.X are more stable than others
>> (e.g.
>> >> if we don't care about certain new features, is it relatively stable)?
>> >> Would it be possible to cut out a version that only has the bits we
>> think
>> >> are stable (and release that)?
>> >>
>> >> From that timeline, and the historic release cadence, it would seem to
>> be
>> >> a
>> >> years away before we get to the stable release?
>> >>
>> >> Thanks,
>> >>
>> >> Jason
>> >>
>> >> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>> cnauroth@hortonworks.com>
>> >> wrote:
>> >>
>> >>> Hello Doug,
>> >>>
>> >>> Thanks for your interest in the SSL feature!
>> >>>
>> >>> At this point, I think we're still pretty far away from declaring a
>> >>> stable
>> >>> release in the 3.5 line.  I don't think we're close enough that anyone
>> >>> can
>> >>> offer a reliable ETA.  This is an earlier thread that describes the
>> >>> high-level strategy for release planning in the 3.5 line:
>> >>>
>> >>> https://s.apache.org/ADK1
>> >>>
>> >>> The next step is a 3.5.2-alpha release.  We're working on resolving a
>> >>> few
>> >>> more blockers before we produce a release candidate.  Hopefully that
>> >>> will
>> >>> get done in the next few weeks.
>> >>>
>> >>> --Chris Nauroth
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>> >>>
>> >>>> I know it's only been a few months, but I was wondering if there was
>> a
>> >>>> ballpark release date for a stable version of 3.5.1. Or is there any
>> >>>> chance
>> >>>> the SSL feature would be added to 3.4.8? Just another person looking
>> to
>> >>>> have
>> >>>> that feature in a stable version. Thanks for all you do! :)
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> View this message in context:
>> >>>>
>> >>>
>> >>>
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>> >>> e
>> >>>> -tp7581744p7582136.html
>> >>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>> >>>>
>> >>>
>> >>>
>> >
>>
>>
>>
>>
>
>

Re: Zookeeper with SSL release date

[powell molleti <po...@yahoo.com.INVALID>]

Apologies for assuming that this release is gated to stabilize re-config.
Please share the Jira search string to list the issues that only exist in 3.5 and not in 3.4(released versions), which are must fix for 3.5 to upgrade it to full a release.
I am seeing lot of issues on both releases hence my question.

Thanks
Powell.


On Wednesday, March 16, 2016 1:59 PM, Alexander Shraer <sh...@gmail.com> wrote:



Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
are related to reconfig. Given this, and the fact that it is run in
production since 2012 in multiple companies, I don't think its more
unstable than any other part of ZooKeeper.

There are multiple reconfig-related bugs that turned out really difficult
to debug without access to the actual system or at least to the Hudson
machines where some tests are failing. In the past, Michi and I physically
went to Hortonworks to debug one such issue, but this is of course not a
good method, and we weren't able to arrange such a visit again.

Regarding making it optional - the reconfig logic has several different
parts, some would be really difficult to disable using a configuration
parameter. But the actual dynamic expansion of the cluster is triggered by
the reconfig command, so it should not affect users who don't invoke it.


On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:

> I suppose we could give it a try. How do other folks feel about it?
>
> -Flavio
> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>
> > So, you could enable the dynamic reconfiguration feature behind a config
> > option, and document that it should only be enabled experimentally, use
> at
> > your own risk.  Keep it off by default.  Allow only static config by
> > default, until it's stable?
> >
> > Jason
> >
> > On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
> wrote:
> >
> > > Hi Jason,
> > >
> > > The consumer in Kafka is pretty independent from the core (brokers),
> > > that's how that project manages to make such a separation. We don't
> have
> > > the same with ZooKeeper as the feature we are talking about is part of
> > the
> > > server and the only way I see of doing what you say is to turn off
> > > features. More specifically, we'd need to disable the reconfig API and
> do
> > > not allow any change to the configuration, even though the code is
> there.
> > >
> > > Reconfig here refers to the ability of changing the configuration of an
> > > ensemble (e.g., changing the set of servers).
> > >
> > > -Flavio
> > >
> > > > On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
> > > >
> > > > So, it would seem sensible to me to have a release where all features
> > are
> > > > stable, except where noted.  E.g. mark certain features as only
> 'alpha
> > > > quality', e.g. the 're-config feature'.  (I assume it's possible to
> > > happily
> > > > use 3.5.X without exercising the unstable re-config bits?).
> > > >
> > > > There's precedent for doing this sort of thing in other projects,
> e.g.
> > in
> > > > Kafka, they've had several release where a new "Consumer API" is
> > shipped
> > > > that is available for beta-testing, but you can still just use the
> > older
> > > > stable consumer api, etc.
> > > >
> > > > Jason
> > > >
> > > > On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > > <powellm79@yahoo.com.invalid
> > > >> wrote:
> > > >
> > > >> Hi Doug,
> > > >> Is 3.5 being an alpha release preventing you from using it?. Or have
> > you
> > > >> run into issues with it?. In general perhaps ZK 3.5 being labeled as
> > > alpha
> > > >> might not be fair, since it is far more stable then what most people
> > > >> associate an alpha release to be.
> > > >> Perhaps if you do not use re-config feature may be it will just work
> > for
> > > >> you?.
> > > >> There are many examples of 3.5.X being used in productions from my
> > > limited
> > > >> knowledge.
> > > >> ThanksPowell.
> > > >>
> > > >>    On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> > > fpj@apache.org>
> > > >> wrote:
> > > >>
> > > >>
> > > >> None of us expected the reconfig changes to take this long to
> > stabilize.
> > > >> Until we get there, I don't think we can do anything else with 3.5.
> > The
> > > >> best bet we have is to work harder to bring 3.5 into a stable rather
> > > than
> > > >> trying to work around it.
> > > >>
> > > >> There are lots of people interested in seeing 3.5 stable, and if we
> > get
> > > >> everyone to contribute more patches and code reviews, we should be
> > able
> > > to
> > > >> do it sooner. After all, it is a community based effort, so the
> > > community
> > > >> shouldn't rely on just 2-3 people doing the work.
> > > >>
> > > >> -Flavio
> > > >>
> > > >>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
> > > >> wrote:
> > > >>>
> > > >>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
> > the
> > > >>> stable maintenance line, we are very conservative about
> back-porting
> > to
> > > >>> it.  Our policy is to limit back-ports to critical bug fixes and
> not
> > > >>> introduce any new features in the 3.4 line.  This is a matter of
> > > managing
> > > >>> risk.
> > > >>>
> > > >>> Jason, your question about release cadence is a fair one.  If it's
> > any
> > > >>> consolation, we are now taking the approach of trying to limit the
> > > scope
> > > >>> of anything new introduced in 3.5 too.  That would allow us to
> focus
> > on
> > > >>> stabilization: resolving blocker bugs and freezing public APIs.  I
> > > think
> > > >>> this will help us accelerate the releases into beta and eventual
> GA.
> > > >>>
> > > >>> I am new to ZooKeeper release management, so I'd like to hear
> > thoughts
> > > >>> from more experienced committers and PMC members about your
> proposal
> > to
> > > >>> try to cut a stable release for a limited subset of what is in
> > > branch-3.5
> > > >>> now.  My instinct is that it would be challenging to cherry-pick
> out
> > > >>> pieces of branch-3.5 piecemeal at this point.  This would become
> > > another
> > > >>> release line for an already resource-constrained volunteer staff to
> > > >>> manage.  I'd prefer to dedicate those limited resources to overall
> > 3.5
> > > >>> stabilization.  Also, a 3.5 release in which certain features
> > > "vanished"
> > > >>> because of not meeting some stability criteria would be
> undesirable.
> > > >>>
> > > >>> --Chris Nauroth
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> > > >>>
> > > >>>> Chris,
> > > >>>>
> > > >>>> Can you say whether some parts of 3.5.X are more stable than
> others
> > > >> (e.g.
> > > >>>> if we don't care about certain new features, is it relatively
> > stable)?
> > > >>>> Would it be possible to cut out a version that only has the bits
> we
> > > >> think
> > > >>>> are stable (and release that)?
> > > >>>>
> > > >>>> From that timeline, and the historic release cadence, it would
> seem
> > to
> > > >> be
> > > >>>> a
> > > >>>> years away before we get to the stable release?
> > > >>>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Jason
> > > >>>>
> > > >>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> > > >> cnauroth@hortonworks.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Hello Doug,
> > > >>>>>
> > > >>>>> Thanks for your interest in the SSL feature!
> > > >>>>>
> > > >>>>> At this point, I think we're still pretty far away from
> declaring a
> > > >>>>> stable
> > > >>>>> release in the 3.5 line.  I don't think we're close enough that
> > > anyone
> > > >>>>> can
> > > >>>>> offer a reliable ETA.  This is an earlier thread that describes
> the
> > > >>>>> high-level strategy for release planning in the 3.5 line:
> > > >>>>>
> > > >>>>> https://s.apache.org/ADK1
> > > >>>>>
> > > >>>>> The next step is a 3.5.2-alpha release.  We're working on
> > resolving a
> > > >>>>> few
> > > >>>>> more blockers before we produce a release candidate.  Hopefully
> > that
> > > >>>>> will
> > > >>>>> get done in the next few weeks.
> > > >>>>>
> > > >>>>> --Chris Nauroth
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> > > >>>>>
> > > >>>>>> I know it's only been a few months, but I was wondering if there
> > > was a
> > > >>>>>> ballpark release date for a stable version of 3.5.1. Or is there
> > any
> > > >>>>>> chance
> > > >>>>>> the SSL feature would be added to 3.4.8? Just another person
> > looking
> > > >> to
> > > >>>>>> have
> > > >>>>>> that feature in a stable version. Thanks for all you do! :)
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> View this message in context:
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > > >>>>> e
> > > >>>>>> -tp7581744p7582136.html
> > > >>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> >
>

Re: Zookeeper with SSL release date

[Irfan Hamid <ih...@salesforce.com>]

+1 to Powell's point: if you could share out a list of the issues perhaps
some of us can take the more straightforward or mechanical ones to get
closer to 3.5.

Regards,
Irfan.

On Wed, Mar 16, 2016 at 3:08 PM, powell molleti <powellm79@yahoo.com.invalid
> wrote:

> Will this option be supplied via config file/args/API?. Will this option
> be a temporary thing i.e what about its compatibility?.
>
> thanks
> Powell.
>
>
>
> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <fp...@apache.org>
> wrote:
> The main issue to sort out is stability of the API. There is a security
> concern around the fact that clients can freely reconfigure the ensemble.
> If we follow the plan that Pat proposed some time ago:
>
>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> <
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >
>
> Locking the API is the main step to move it to beta. Sorting out bugs is
> definitely necessary, but it isn't the main thing that is keeping 3.5 in
> alpha.
>
> About making it experimental, I was raising the option of having a switch
> that disables the API calls, not the code. The reason why that could work
> is that anyone using 3.5 who uses the "experimental" API must explicit turn
> on the switch and enable the calls. If they do it, they need to be aware
> that the API can change.
>
> I must say that I haven't really looked closely into doing it, and I'm not
> even entirely convinced that this is a good idea, but since Jason raised
> the point, I'm exploring options.
>
> -Flavio
>
>
> > On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
> >
> > Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only
> 3-4
> > are related to reconfig. Given this, and the fact that it is run in
> > production since 2012 in multiple companies, I don't think its more
> > unstable than any other part of ZooKeeper.
> >
> > There are multiple reconfig-related bugs that turned out really difficult
> > to debug without access to the actual system or at least to the Hudson
> > machines where some tests are failing. In the past, Michi and I
> physically
> > went to Hortonworks to debug one such issue, but this is of course not a
> > good method, and we weren't able to arrange such a visit again.
> >
> > Regarding making it optional - the reconfig logic has several different
> > parts, some would be really difficult to disable using a configuration
> > parameter. But the actual dynamic expansion of the cluster is triggered
> by
> > the reconfig command, so it should not affect users who don't invoke it.
> >
> > On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org>
> wrote:
> >
> >> I suppose we could give it a try. How do other folks feel about it?
> >>
> >> -Flavio
> >> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
> >>
> >>> So, you could enable the dynamic reconfiguration feature behind a
> config
> >>> option, and document that it should only be enabled experimentally, use
> >> at
> >>> your own risk.  Keep it off by default.  Allow only static config by
> >>> default, until it's stable?
> >>>
> >>> Jason
> >>>
> >>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
> >> wrote:
> >>>
> >>>> Hi Jason,
> >>>>
> >>>> The consumer in Kafka is pretty independent from the core (brokers),
> >>>> that's how that project manages to make such a separation. We don't
> >> have
> >>>> the same with ZooKeeper as the feature we are talking about is part of
> >>> the
> >>>> server and the only way I see of doing what you say is to turn off
> >>>> features. More specifically, we'd need to disable the reconfig API and
> >> do
> >>>> not allow any change to the configuration, even though the code is
> >> there.
> >>>>
> >>>> Reconfig here refers to the ability of changing the configuration of
> an
> >>>> ensemble (e.g., changing the set of servers).
> >>>>
> >>>> -Flavio
> >>>>
> >>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
> >>>>>
> >>>>> So, it would seem sensible to me to have a release where all features
> >>> are
> >>>>> stable, except where noted.  E.g. mark certain features as only
> >> 'alpha
> >>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
> >>>> happily
> >>>>> use 3.5.X without exercising the unstable re-config bits?).
> >>>>>
> >>>>> There's precedent for doing this sort of thing in other projects,
> >> e.g.
> >>> in
> >>>>> Kafka, they've had several release where a new "Consumer API" is
> >>> shipped
> >>>>> that is available for beta-testing, but you can still just use the
> >>> older
> >>>>> stable consumer api, etc.
> >>>>>
> >>>>> Jason
> >>>>>
> >>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> >>>> <powellm79@yahoo.com.invalid
> >>>>>> wrote:
> >>>>>
> >>>>>> Hi Doug,
> >>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
> >>> you
> >>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
> >>>> alpha
> >>>>>> might not be fair, since it is far more stable then what most people
> >>>>>> associate an alpha release to be.
> >>>>>> Perhaps if you do not use re-config feature may be it will just work
> >>> for
> >>>>>> you?.
> >>>>>> There are many examples of 3.5.X being used in productions from my
> >>>> limited
> >>>>>> knowledge.
> >>>>>> ThanksPowell.
> >>>>>>
> >>>>>>   On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> >>>> fpj@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>
> >>>>>> None of us expected the reconfig changes to take this long to
> >>> stabilize.
> >>>>>> Until we get there, I don't think we can do anything else with 3.5.
> >>> The
> >>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
> >>>> than
> >>>>>> trying to work around it.
> >>>>>>
> >>>>>> There are lots of people interested in seeing 3.5 stable, and if we
> >>> get
> >>>>>> everyone to contribute more patches and code reviews, we should be
> >>> able
> >>>> to
> >>>>>> do it sooner. After all, it is a community based effort, so the
> >>>> community
> >>>>>> shouldn't rely on just 2-3 people doing the work.
> >>>>>>
> >>>>>> -Flavio
> >>>>>>
> >>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
> >>> the
> >>>>>>> stable maintenance line, we are very conservative about
> >> back-porting
> >>> to
> >>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
> >> not
> >>>>>>> introduce any new features in the 3.4 line.  This is a matter of
> >>>> managing
> >>>>>>> risk.
> >>>>>>>
> >>>>>>> Jason, your question about release cadence is a fair one.  If it's
> >>> any
> >>>>>>> consolation, we are now taking the approach of trying to limit the
> >>>> scope
> >>>>>>> of anything new introduced in 3.5 too.  That would allow us to
> >> focus
> >>> on
> >>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
> >>>> think
> >>>>>>> this will help us accelerate the releases into beta and eventual
> >> GA.
> >>>>>>>
> >>>>>>> I am new to ZooKeeper release management, so I'd like to hear
> >>> thoughts
> >>>>>>> from more experienced committers and PMC members about your
> >> proposal
> >>> to
> >>>>>>> try to cut a stable release for a limited subset of what is in
> >>>> branch-3.5
> >>>>>>> now.  My instinct is that it would be challenging to cherry-pick
> >> out
> >>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
> >>>> another
> >>>>>>> release line for an already resource-constrained volunteer staff to
> >>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
> >>> 3.5
> >>>>>>> stabilization.  Also, a 3.5 release in which certain features
> >>>> "vanished"
> >>>>>>> because of not meeting some stability criteria would be
> >> undesirable.
> >>>>>>>
> >>>>>>> --Chris Nauroth
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> >>>>>>>
> >>>>>>>> Chris,
> >>>>>>>>
> >>>>>>>> Can you say whether some parts of 3.5.X are more stable than
> >> others
> >>>>>> (e.g.
> >>>>>>>> if we don't care about certain new features, is it relatively
> >>> stable)?
> >>>>>>>> Would it be possible to cut out a version that only has the bits
> >> we
> >>>>>> think
> >>>>>>>> are stable (and release that)?
> >>>>>>>>
> >>>>>>>> From that timeline, and the historic release cadence, it would
> >> seem
> >>> to
> >>>>>> be
> >>>>>>>> a
> >>>>>>>> years away before we get to the stable release?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>>
> >>>>>>>> Jason
> >>>>>>>>
> >>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> >>>>>> cnauroth@hortonworks.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hello Doug,
> >>>>>>>>>
> >>>>>>>>> Thanks for your interest in the SSL feature!
> >>>>>>>>>
> >>>>>>>>> At this point, I think we're still pretty far away from
> >> declaring a
> >>>>>>>>> stable
> >>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
> >>>> anyone
> >>>>>>>>> can
> >>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
> >> the
> >>>>>>>>> high-level strategy for release planning in the 3.5 line:
> >>>>>>>>>
> >>>>>>>>> https://s.apache.org/ADK1
> >>>>>>>>>
> >>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
> >>> resolving a
> >>>>>>>>> few
> >>>>>>>>> more blockers before we produce a release candidate.  Hopefully
> >>> that
> >>>>>>>>> will
> >>>>>>>>> get done in the next few weeks.
> >>>>>>>>>
> >>>>>>>>> --Chris Nauroth
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> I know it's only been a few months, but I was wondering if there
> >>>> was a
> >>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
> >>> any
> >>>>>>>>>> chance
> >>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
> >>> looking
> >>>>>> to
> >>>>>>>>>> have
> >>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> View this message in context:
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >>>>>>>>> e
> >>>>>>>>>> -tp7581744p7582136.html
> >>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>
>

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

On Mon, Mar 21, 2016 at 9:16 PM, Alexander Shraer <sh...@gmail.com> wrote:
> another thing - shouldn't things like setting quotas also be part of the
> admin API ? how does that
> work now ?
>

Hm, good point. There is no code level API afaik - the quotas are in
/zookeeper znodes iirc. So if you care about controlling access to the
quota settings you need to setup ACLs. I don't think we thought about
that back in the day.

Patrick

> Alex
>
> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <sh...@gmail.com> wrote:
>
>> I don't think that getConfig should be an admin functionality. It is
>> essential for client-side re-balancing
>> that we implemented (all clients shoudl be able to detect configuration
>> changes via getConfig). It could
>> be hidden somewhat by defining higher-level re-balancing
>> policies (ZOOKEEPER-2016)
>> but there hasn't been enough progress on that. Perhaps instead getConfig
>> should be controlled
>> by a separate flag ?
>>
>> Alex
>>
>> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org> wrote:
>>
>>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <sh...@gmail.com>
>>> wrote:
>>> > Hi Patrick, Flavio,
>>> >
>>> > Since there seems to be consensus on this, I can add this flag, unless
>>> > someone else wants to. I assume that getConfig should still work
>>> regardless
>>> > of the flag ? is there a security concern with clients knowing the list
>>> of
>>> > servers?
>>> >
>>>
>>> We've always hidden that detail from users. We don't even expose which
>>> server you're connected to today. I remember Ben (and perhaps Flavio?)
>>> highlighting this as important to maintain although I'm not super
>>> familiar with the specifics on why. It made sense to me though from
>>> the perspective that we don't want clients to make assumptions that
>>> probably shouldn't.
>>>
>>> My thinking is that we should 1) add a config option to enable
>>> reconfig (off by default), 2) move reconfig specific functionality
>>> from ZooKeeper.java (including getconfig) into an "admin" package,
>>> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
>>> when folks do want to enable reconfig and are also worried about auth.
>>> (e.g. turn on kerb)
>>>
>>> Again, I don't see any of this as a quality issue personally. As such
>>> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
>>> were interested in doing such a release. Adjusting the API should be
>>> done before we move to "beta" though. Although that seems like a
>>> pretty mechanical (eclipse/idea) type refactoring?
>>>
>>> Patrick
>>>
>>> > Cheers,
>>> > Alex
>>> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>>> >
>>> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fp...@apache.org>
>>> wrote:
>>> >> > I gotta say that I'm not super excited about this option, but for
>>> some
>>> >> reason I ended up carrying the flag. To recap, I just raised this
>>> option
>>> >> because it seems that there are folks interested in features in 3.5
>>> like
>>> >> SSL and not necessarily in reconfiguration. SSL is important and to
>>> take
>>> >> Kafka as an example, it sucks that we can't have a whole set up using
>>> SSL.
>>> >> For ZK, the real questions are:
>>> >> >
>>> >> > 1- how fast can we make 3.5 stable?
>>> >> > 2- would it be faster if we have a way of disabling reconfiguration?
>>> >> > 3- would enough users care about a stable 3.5 that has
>>> reconfiguration
>>> >> disabled?
>>> >> >
>>> >> > It is taking a long time to get 3.5 to beta. There has been some good
>>> >> activity around 3.5.2 release, which is a great step, but it is unclear
>>> >> when 3.5.3 is going to come and if we will be able to make 3.5 beta
>>> then.
>>> >> Frankly, disabling reconfiguration sounds undesirable because it is an
>>> >> important feature, but I currently don't use it in production, so from
>>> a
>>> >> practical point of view, I can go both ways. Whether we go through the
>>> >> trouble of doing 2 depends on users interested in that option and folks
>>> >> willing to implement it.
>>> >> >
>>> >> > To answer your question, Powell, my pseudo-proposal is kind of a
>>> funny
>>> >> option because once the feature is stable, then we wouldn't need a
>>> switch
>>> >> any longer, so there is not need of a deprecation path, we just start
>>> >> ignoring it from the first beta release. Until it is beta, I'd say that
>>> >> default is disabled.
>>> >>
>>> >> I would argue that we need this even when it does become stable. To me
>>> >> this is not a quality issue so much as it is an auth issue. We want to
>>> >> make it simple for folks to run a vanilla/old ZK cluster and not worry
>>> >> about the security implications of having reconfig enabled.
>>> >>
>>> >> Patrick
>>> >>
>>> >> >
>>> >> > -Flavio
>>> >> >
>>> >> >> On 17 Mar 2016, at 17:44, powell molleti
>>> <po...@yahoo.com.INVALID>
>>> >> wrote:
>>> >> >>
>>> >> >> Hi Flavio,
>>> >> >>
>>> >> >> Generally do config options and command line args come under the
>>> same
>>> >> SLA as API?. I was assuming as such hence my question. Perhaps if the
>>> >> expectation is that this config option is temporary from get go then
>>> may be
>>> >> it is ok. The default for re-config support will be enabled or
>>> disabled?.
>>> >> >>
>>> >> >> I am just thinking from provisioning point of view when people
>>> generate
>>> >> config options etc.
>>> >> >>
>>> >> >> Thanks
>>> >> >> Powell.
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
>>> fpj@apache.org>
>>> >> wrote:
>>> >> >> Hi Powell,
>>> >> >>
>>> >> >> I was thinking config file and system property server side. What's
>>> your
>>> >> concern with compatibility? The API itself wouldn't change, but the
>>> config
>>> >> option wouldn't exist in previous versions and would not exist either
>>> in
>>> >> later stable versions of 3.5.
>>> >> >>
>>> >> >> -Flavio
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>> On 16 Mar 2016, at 22:08, powell molleti
>>> <po...@yahoo.com.INVALID>
>>> >> wrote:
>>> >> >>>
>>> >> >>> Will this option be supplied via config file/args/API?. Will this
>>> >> option be a temporary thing i.e what about its compatibility?.
>>> >> >>>
>>> >> >>> thanks
>>> >> >>> Powell.
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
>>> fpj@apache.org>
>>> >> wrote:
>>> >> >>> The main issue to sort out is stability of the API. There is a
>>> >> security concern around the fact that clients can freely reconfigure
>>> the
>>> >> ensemble. If we follow the plan that Pat proposed some time ago:
>>> >> >>>
>>> >> >>>
>>> >>
>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>>> >> <
>>> >>
>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>>> >> >
>>> >> >>>
>>> >> >>> Locking the API is the main step to move it to beta. Sorting out
>>> bugs
>>> >> is definitely necessary, but it isn't the main thing that is keeping
>>> 3.5 in
>>> >> alpha.
>>> >> >>>
>>> >> >>> About making it experimental, I was raising the option of having a
>>> >> switch that disables the API calls, not the code. The reason why that
>>> could
>>> >> work is that anyone using 3.5 who uses the "experimental" API must
>>> explicit
>>> >> turn on the switch and enable the calls. If they do it, they need to be
>>> >> aware that the API can change.
>>> >> >>>
>>> >> >>> I must say that I haven't really looked closely into doing it, and
>>> I'm
>>> >> not even entirely convinced that this is a good idea, but since Jason
>>> >> raised the point, I'm exploring options.
>>> >> >>>
>>> >> >>> -Flavio
>>> >> >>>
>>> >> >>>
>>> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com>
>>> wrote:
>>> >> >>>>
>>> >> >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
>>> >> only 3-4
>>> >> >>>> are related to reconfig. Given this, and the fact that it is run
>>> in
>>> >> >>>> production since 2012 in multiple companies, I don't think its
>>> more
>>> >> >>>> unstable than any other part of ZooKeeper.
>>> >> >>>>
>>> >> >>>> There are multiple reconfig-related bugs that turned out really
>>> >> difficult
>>> >> >>>> to debug without access to the actual system or at least to the
>>> Hudson
>>> >> >>>> machines where some tests are failing. In the past, Michi and I
>>> >> physically
>>> >> >>>> went to Hortonworks to debug one such issue, but this is of course
>>> >> not a
>>> >> >>>> good method, and we weren't able to arrange such a visit again.
>>> >> >>>>
>>> >> >>>> Regarding making it optional - the reconfig logic has several
>>> >> different
>>> >> >>>> parts, some would be really difficult to disable using a
>>> configuration
>>> >> >>>> parameter. But the actual dynamic expansion of the cluster is
>>> >> triggered by
>>> >> >>>> the reconfig command, so it should not affect users who don't
>>> invoke
>>> >> it.
>>> >> >>>>
>>> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
>>> fpj@apache.org>
>>> >> wrote:
>>> >> >>>>
>>> >> >>>>> I suppose we could give it a try. How do other folks feel about
>>> it?
>>> >> >>>>>
>>> >> >>>>> -Flavio
>>> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com>
>>> wrote:
>>> >> >>>>>
>>> >> >>>>>> So, you could enable the dynamic reconfiguration feature behind
>>> a
>>> >> config
>>> >> >>>>>> option, and document that it should only be enabled
>>> experimentally,
>>> >> use
>>> >> >>>>> at
>>> >> >>>>>> your own risk.  Keep it off by default.  Allow only static
>>> config by
>>> >> >>>>>> default, until it's stable?
>>> >> >>>>>>
>>> >> >>>>>> Jason
>>> >> >>>>>>
>>> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
>>> fpj@apache.org>
>>> >> >>>>> wrote:
>>> >> >>>>>>
>>> >> >>>>>>> Hi Jason,
>>> >> >>>>>>>
>>> >> >>>>>>> The consumer in Kafka is pretty independent from the core
>>> >> (brokers),
>>> >> >>>>>>> that's how that project manages to make such a separation. We
>>> don't
>>> >> >>>>> have
>>> >> >>>>>>> the same with ZooKeeper as the feature we are talking about is
>>> >> part of
>>> >> >>>>>> the
>>> >> >>>>>>> server and the only way I see of doing what you say is to turn
>>> off
>>> >> >>>>>>> features. More specifically, we'd need to disable the reconfig
>>> API
>>> >> and
>>> >> >>>>> do
>>> >> >>>>>>> not allow any change to the configuration, even though the
>>> code is
>>> >> >>>>> there.
>>> >> >>>>>>>
>>> >> >>>>>>> Reconfig here refers to the ability of changing the
>>> configuration
>>> >> of an
>>> >> >>>>>>> ensemble (e.g., changing the set of servers).
>>> >> >>>>>>>
>>> >> >>>>>>> -Flavio
>>> >> >>>>>>>
>>> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com>
>>> >> wrote:
>>> >> >>>>>>>>
>>> >> >>>>>>>> So, it would seem sensible to me to have a release where all
>>> >> features
>>> >> >>>>>> are
>>> >> >>>>>>>> stable, except where noted.  E.g. mark certain features as
>>> only
>>> >> >>>>> 'alpha
>>> >> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's
>>> possible
>>> >> to
>>> >> >>>>>>> happily
>>> >> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>> >> >>>>>>>>
>>> >> >>>>>>>> There's precedent for doing this sort of thing in other
>>> projects,
>>> >> >>>>> e.g.
>>> >> >>>>>> in
>>> >> >>>>>>>> Kafka, they've had several release where a new "Consumer API"
>>> is
>>> >> >>>>>> shipped
>>> >> >>>>>>>> that is available for beta-testing, but you can still just
>>> use the
>>> >> >>>>>> older
>>> >> >>>>>>>> stable consumer api, etc.
>>> >> >>>>>>>>
>>> >> >>>>>>>> Jason
>>> >> >>>>>>>>
>>> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>> >> >>>>>>> <powellm79@yahoo.com.invalid
>>> >> >>>>>>>>> wrote:
>>> >> >>>>>>>>
>>> >> >>>>>>>>> Hi Doug,
>>> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using it?.
>>> Or
>>> >> have
>>> >> >>>>>> you
>>> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being
>>> >> labeled as
>>> >> >>>>>>> alpha
>>> >> >>>>>>>>> might not be fair, since it is far more stable then what most
>>> >> people
>>> >> >>>>>>>>> associate an alpha release to be.
>>> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it will
>>> just
>>> >> work
>>> >> >>>>>> for
>>> >> >>>>>>>>> you?.
>>> >> >>>>>>>>> There are many examples of 3.5.X being used in productions
>>> from
>>> >> my
>>> >> >>>>>>> limited
>>> >> >>>>>>>>> knowledge.
>>> >> >>>>>>>>> ThanksPowell.
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>> >> >>>>>>> fpj@apache.org>
>>> >> >>>>>>>>> wrote:
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> None of us expected the reconfig changes to take this long to
>>> >> >>>>>> stabilize.
>>> >> >>>>>>>>> Until we get there, I don't think we can do anything else
>>> with
>>> >> 3.5.
>>> >> >>>>>> The
>>> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable
>>> >> rather
>>> >> >>>>>>> than
>>> >> >>>>>>>>> trying to work around it.
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> There are lots of people interested in seeing 3.5 stable,
>>> and if
>>> >> we
>>> >> >>>>>> get
>>> >> >>>>>>>>> everyone to contribute more patches and code reviews, we
>>> should
>>> >> be
>>> >> >>>>>> able
>>> >> >>>>>>> to
>>> >> >>>>>>>>> do it sooner. After all, it is a community based effort, so
>>> the
>>> >> >>>>>>> community
>>> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> -Flavio
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
>>> >> cnauroth@hortonworks.com>
>>> >> >>>>>>>>> wrote:
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since
>>> >> 3.4 is
>>> >> >>>>>> the
>>> >> >>>>>>>>>> stable maintenance line, we are very conservative about
>>> >> >>>>> back-porting
>>> >> >>>>>> to
>>> >> >>>>>>>>>> it.  Our policy is to limit back-ports to critical bug
>>> fixes and
>>> >> >>>>> not
>>> >> >>>>>>>>>> introduce any new features in the 3.4 line.  This is a
>>> matter of
>>> >> >>>>>>> managing
>>> >> >>>>>>>>>> risk.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> Jason, your question about release cadence is a fair one.
>>> If
>>> >> it's
>>> >> >>>>>> any
>>> >> >>>>>>>>>> consolation, we are now taking the approach of trying to
>>> limit
>>> >> the
>>> >> >>>>>>> scope
>>> >> >>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us
>>> to
>>> >> >>>>> focus
>>> >> >>>>>> on
>>> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public
>>> >> APIs.  I
>>> >> >>>>>>> think
>>> >> >>>>>>>>>> this will help us accelerate the releases into beta and
>>> eventual
>>> >> >>>>> GA.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to
>>> hear
>>> >> >>>>>> thoughts
>>> >> >>>>>>>>>> from more experienced committers and PMC members about your
>>> >> >>>>> proposal
>>> >> >>>>>> to
>>> >> >>>>>>>>>> try to cut a stable release for a limited subset of what is
>>> in
>>> >> >>>>>>> branch-3.5
>>> >> >>>>>>>>>> now.  My instinct is that it would be challenging to
>>> cherry-pick
>>> >> >>>>> out
>>> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would
>>> become
>>> >> >>>>>>> another
>>> >> >>>>>>>>>> release line for an already resource-constrained volunteer
>>> >> staff to
>>> >> >>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to
>>> >> overall
>>> >> >>>>>> 3.5
>>> >> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain
>>> features
>>> >> >>>>>>> "vanished"
>>> >> >>>>>>>>>> because of not meeting some stability criteria would be
>>> >> >>>>> undesirable.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> --Chris Nauroth
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com>
>>> >> wrote:
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>> Chris,
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable
>>> than
>>> >> >>>>> others
>>> >> >>>>>>>>> (e.g.
>>> >> >>>>>>>>>>> if we don't care about certain new features, is it
>>> relatively
>>> >> >>>>>> stable)?
>>> >> >>>>>>>>>>> Would it be possible to cut out a version that only has the
>>> >> bits
>>> >> >>>>> we
>>> >> >>>>>>>>> think
>>> >> >>>>>>>>>>> are stable (and release that)?
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> From that timeline, and the historic release cadence, it
>>> would
>>> >> >>>>> seem
>>> >> >>>>>> to
>>> >> >>>>>>>>> be
>>> >> >>>>>>>>>>> a
>>> >> >>>>>>>>>>> years away before we get to the stable release?
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> Thanks,
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> Jason
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>> >> >>>>>>>>> cnauroth@hortonworks.com>
>>> >> >>>>>>>>>>> wrote:
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>>> Hello Doug,
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> At this point, I think we're still pretty far away from
>>> >> >>>>> declaring a
>>> >> >>>>>>>>>>>> stable
>>> >> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough
>>> >> that
>>> >> >>>>>>> anyone
>>> >> >>>>>>>>>>>> can
>>> >> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
>>> >> describes
>>> >> >>>>> the
>>> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> https://s.apache.org/ADK1
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>> >> >>>>>> resolving a
>>> >> >>>>>>>>>>>> few
>>> >> >>>>>>>>>>>> more blockers before we produce a release candidate.
>>> >> Hopefully
>>> >> >>>>>> that
>>> >> >>>>>>>>>>>> will
>>> >> >>>>>>>>>>>> get done in the next few weeks.
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> --Chris Nauroth
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> I know it's only been a few months, but I was wondering
>>> if
>>> >> there
>>> >> >>>>>>> was a
>>> >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or
>>> is
>>> >> there
>>> >> >>>>>> any
>>> >> >>>>>>>>>>>>> chance
>>> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another
>>> person
>>> >> >>>>>> looking
>>> >> >>>>>>>>> to
>>> >> >>>>>>>>>>>>> have
>>> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you do!
>>> :)
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> --
>>> >> >>>>>>>>>>>>> View this message in context:
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>
>>> >> >>>>>
>>> >>
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>> >> >>>>>>>>>>>> e
>>> >> >>>>>>>>>>>>> -tp7581744p7582136.html
>>> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
>>> >> Nabble.com.
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>
>>> >> >>>>>
>>> >> >
>>> >>
>>>
>>
>>

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

another thing - shouldn't things like setting quotas also be part of the
admin API ? how does that
work now ?

Alex

On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <sh...@gmail.com> wrote:

> I don't think that getConfig should be an admin functionality. It is
> essential for client-side re-balancing
> that we implemented (all clients shoudl be able to detect configuration
> changes via getConfig). It could
> be hidden somewhat by defining higher-level re-balancing
> policies (ZOOKEEPER-2016)
> but there hasn't been enough progress on that. Perhaps instead getConfig
> should be controlled
> by a separate flag ?
>
> Alex
>
> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org> wrote:
>
>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <sh...@gmail.com>
>> wrote:
>> > Hi Patrick, Flavio,
>> >
>> > Since there seems to be consensus on this, I can add this flag, unless
>> > someone else wants to. I assume that getConfig should still work
>> regardless
>> > of the flag ? is there a security concern with clients knowing the list
>> of
>> > servers?
>> >
>>
>> We've always hidden that detail from users. We don't even expose which
>> server you're connected to today. I remember Ben (and perhaps Flavio?)
>> highlighting this as important to maintain although I'm not super
>> familiar with the specifics on why. It made sense to me though from
>> the perspective that we don't want clients to make assumptions that
>> probably shouldn't.
>>
>> My thinking is that we should 1) add a config option to enable
>> reconfig (off by default), 2) move reconfig specific functionality
>> from ZooKeeper.java (including getconfig) into an "admin" package,
>> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
>> when folks do want to enable reconfig and are also worried about auth.
>> (e.g. turn on kerb)
>>
>> Again, I don't see any of this as a quality issue personally. As such
>> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
>> were interested in doing such a release. Adjusting the API should be
>> done before we move to "beta" though. Although that seems like a
>> pretty mechanical (eclipse/idea) type refactoring?
>>
>> Patrick
>>
>> > Cheers,
>> > Alex
>> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>> >
>> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fp...@apache.org>
>> wrote:
>> >> > I gotta say that I'm not super excited about this option, but for
>> some
>> >> reason I ended up carrying the flag. To recap, I just raised this
>> option
>> >> because it seems that there are folks interested in features in 3.5
>> like
>> >> SSL and not necessarily in reconfiguration. SSL is important and to
>> take
>> >> Kafka as an example, it sucks that we can't have a whole set up using
>> SSL.
>> >> For ZK, the real questions are:
>> >> >
>> >> > 1- how fast can we make 3.5 stable?
>> >> > 2- would it be faster if we have a way of disabling reconfiguration?
>> >> > 3- would enough users care about a stable 3.5 that has
>> reconfiguration
>> >> disabled?
>> >> >
>> >> > It is taking a long time to get 3.5 to beta. There has been some good
>> >> activity around 3.5.2 release, which is a great step, but it is unclear
>> >> when 3.5.3 is going to come and if we will be able to make 3.5 beta
>> then.
>> >> Frankly, disabling reconfiguration sounds undesirable because it is an
>> >> important feature, but I currently don't use it in production, so from
>> a
>> >> practical point of view, I can go both ways. Whether we go through the
>> >> trouble of doing 2 depends on users interested in that option and folks
>> >> willing to implement it.
>> >> >
>> >> > To answer your question, Powell, my pseudo-proposal is kind of a
>> funny
>> >> option because once the feature is stable, then we wouldn't need a
>> switch
>> >> any longer, so there is not need of a deprecation path, we just start
>> >> ignoring it from the first beta release. Until it is beta, I'd say that
>> >> default is disabled.
>> >>
>> >> I would argue that we need this even when it does become stable. To me
>> >> this is not a quality issue so much as it is an auth issue. We want to
>> >> make it simple for folks to run a vanilla/old ZK cluster and not worry
>> >> about the security implications of having reconfig enabled.
>> >>
>> >> Patrick
>> >>
>> >> >
>> >> > -Flavio
>> >> >
>> >> >> On 17 Mar 2016, at 17:44, powell molleti
>> <po...@yahoo.com.INVALID>
>> >> wrote:
>> >> >>
>> >> >> Hi Flavio,
>> >> >>
>> >> >> Generally do config options and command line args come under the
>> same
>> >> SLA as API?. I was assuming as such hence my question. Perhaps if the
>> >> expectation is that this config option is temporary from get go then
>> may be
>> >> it is ok. The default for re-config support will be enabled or
>> disabled?.
>> >> >>
>> >> >> I am just thinking from provisioning point of view when people
>> generate
>> >> config options etc.
>> >> >>
>> >> >> Thanks
>> >> >> Powell.
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
>> fpj@apache.org>
>> >> wrote:
>> >> >> Hi Powell,
>> >> >>
>> >> >> I was thinking config file and system property server side. What's
>> your
>> >> concern with compatibility? The API itself wouldn't change, but the
>> config
>> >> option wouldn't exist in previous versions and would not exist either
>> in
>> >> later stable versions of 3.5.
>> >> >>
>> >> >> -Flavio
>> >> >>
>> >> >>
>> >> >>
>> >> >>> On 16 Mar 2016, at 22:08, powell molleti
>> <po...@yahoo.com.INVALID>
>> >> wrote:
>> >> >>>
>> >> >>> Will this option be supplied via config file/args/API?. Will this
>> >> option be a temporary thing i.e what about its compatibility?.
>> >> >>>
>> >> >>> thanks
>> >> >>> Powell.
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
>> fpj@apache.org>
>> >> wrote:
>> >> >>> The main issue to sort out is stability of the API. There is a
>> >> security concern around the fact that clients can freely reconfigure
>> the
>> >> ensemble. If we follow the plan that Pat proposed some time ago:
>> >> >>>
>> >> >>>
>> >>
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> >> <
>> >>
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> >> >
>> >> >>>
>> >> >>> Locking the API is the main step to move it to beta. Sorting out
>> bugs
>> >> is definitely necessary, but it isn't the main thing that is keeping
>> 3.5 in
>> >> alpha.
>> >> >>>
>> >> >>> About making it experimental, I was raising the option of having a
>> >> switch that disables the API calls, not the code. The reason why that
>> could
>> >> work is that anyone using 3.5 who uses the "experimental" API must
>> explicit
>> >> turn on the switch and enable the calls. If they do it, they need to be
>> >> aware that the API can change.
>> >> >>>
>> >> >>> I must say that I haven't really looked closely into doing it, and
>> I'm
>> >> not even entirely convinced that this is a good idea, but since Jason
>> >> raised the point, I'm exploring options.
>> >> >>>
>> >> >>> -Flavio
>> >> >>>
>> >> >>>
>> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com>
>> wrote:
>> >> >>>>
>> >> >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
>> >> only 3-4
>> >> >>>> are related to reconfig. Given this, and the fact that it is run
>> in
>> >> >>>> production since 2012 in multiple companies, I don't think its
>> more
>> >> >>>> unstable than any other part of ZooKeeper.
>> >> >>>>
>> >> >>>> There are multiple reconfig-related bugs that turned out really
>> >> difficult
>> >> >>>> to debug without access to the actual system or at least to the
>> Hudson
>> >> >>>> machines where some tests are failing. In the past, Michi and I
>> >> physically
>> >> >>>> went to Hortonworks to debug one such issue, but this is of course
>> >> not a
>> >> >>>> good method, and we weren't able to arrange such a visit again.
>> >> >>>>
>> >> >>>> Regarding making it optional - the reconfig logic has several
>> >> different
>> >> >>>> parts, some would be really difficult to disable using a
>> configuration
>> >> >>>> parameter. But the actual dynamic expansion of the cluster is
>> >> triggered by
>> >> >>>> the reconfig command, so it should not affect users who don't
>> invoke
>> >> it.
>> >> >>>>
>> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
>> fpj@apache.org>
>> >> wrote:
>> >> >>>>
>> >> >>>>> I suppose we could give it a try. How do other folks feel about
>> it?
>> >> >>>>>
>> >> >>>>> -Flavio
>> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com>
>> wrote:
>> >> >>>>>
>> >> >>>>>> So, you could enable the dynamic reconfiguration feature behind
>> a
>> >> config
>> >> >>>>>> option, and document that it should only be enabled
>> experimentally,
>> >> use
>> >> >>>>> at
>> >> >>>>>> your own risk.  Keep it off by default.  Allow only static
>> config by
>> >> >>>>>> default, until it's stable?
>> >> >>>>>>
>> >> >>>>>> Jason
>> >> >>>>>>
>> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
>> fpj@apache.org>
>> >> >>>>> wrote:
>> >> >>>>>>
>> >> >>>>>>> Hi Jason,
>> >> >>>>>>>
>> >> >>>>>>> The consumer in Kafka is pretty independent from the core
>> >> (brokers),
>> >> >>>>>>> that's how that project manages to make such a separation. We
>> don't
>> >> >>>>> have
>> >> >>>>>>> the same with ZooKeeper as the feature we are talking about is
>> >> part of
>> >> >>>>>> the
>> >> >>>>>>> server and the only way I see of doing what you say is to turn
>> off
>> >> >>>>>>> features. More specifically, we'd need to disable the reconfig
>> API
>> >> and
>> >> >>>>> do
>> >> >>>>>>> not allow any change to the configuration, even though the
>> code is
>> >> >>>>> there.
>> >> >>>>>>>
>> >> >>>>>>> Reconfig here refers to the ability of changing the
>> configuration
>> >> of an
>> >> >>>>>>> ensemble (e.g., changing the set of servers).
>> >> >>>>>>>
>> >> >>>>>>> -Flavio
>> >> >>>>>>>
>> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com>
>> >> wrote:
>> >> >>>>>>>>
>> >> >>>>>>>> So, it would seem sensible to me to have a release where all
>> >> features
>> >> >>>>>> are
>> >> >>>>>>>> stable, except where noted.  E.g. mark certain features as
>> only
>> >> >>>>> 'alpha
>> >> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's
>> possible
>> >> to
>> >> >>>>>>> happily
>> >> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>> >> >>>>>>>>
>> >> >>>>>>>> There's precedent for doing this sort of thing in other
>> projects,
>> >> >>>>> e.g.
>> >> >>>>>> in
>> >> >>>>>>>> Kafka, they've had several release where a new "Consumer API"
>> is
>> >> >>>>>> shipped
>> >> >>>>>>>> that is available for beta-testing, but you can still just
>> use the
>> >> >>>>>> older
>> >> >>>>>>>> stable consumer api, etc.
>> >> >>>>>>>>
>> >> >>>>>>>> Jason
>> >> >>>>>>>>
>> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>> >> >>>>>>> <powellm79@yahoo.com.invalid
>> >> >>>>>>>>> wrote:
>> >> >>>>>>>>
>> >> >>>>>>>>> Hi Doug,
>> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using it?.
>> Or
>> >> have
>> >> >>>>>> you
>> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being
>> >> labeled as
>> >> >>>>>>> alpha
>> >> >>>>>>>>> might not be fair, since it is far more stable then what most
>> >> people
>> >> >>>>>>>>> associate an alpha release to be.
>> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it will
>> just
>> >> work
>> >> >>>>>> for
>> >> >>>>>>>>> you?.
>> >> >>>>>>>>> There are many examples of 3.5.X being used in productions
>> from
>> >> my
>> >> >>>>>>> limited
>> >> >>>>>>>>> knowledge.
>> >> >>>>>>>>> ThanksPowell.
>> >> >>>>>>>>>
>> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>> >> >>>>>>> fpj@apache.org>
>> >> >>>>>>>>> wrote:
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>> None of us expected the reconfig changes to take this long to
>> >> >>>>>> stabilize.
>> >> >>>>>>>>> Until we get there, I don't think we can do anything else
>> with
>> >> 3.5.
>> >> >>>>>> The
>> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable
>> >> rather
>> >> >>>>>>> than
>> >> >>>>>>>>> trying to work around it.
>> >> >>>>>>>>>
>> >> >>>>>>>>> There are lots of people interested in seeing 3.5 stable,
>> and if
>> >> we
>> >> >>>>>> get
>> >> >>>>>>>>> everyone to contribute more patches and code reviews, we
>> should
>> >> be
>> >> >>>>>> able
>> >> >>>>>>> to
>> >> >>>>>>>>> do it sooner. After all, it is a community based effort, so
>> the
>> >> >>>>>>> community
>> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>> >> >>>>>>>>>
>> >> >>>>>>>>> -Flavio
>> >> >>>>>>>>>
>> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
>> >> cnauroth@hortonworks.com>
>> >> >>>>>>>>> wrote:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since
>> >> 3.4 is
>> >> >>>>>> the
>> >> >>>>>>>>>> stable maintenance line, we are very conservative about
>> >> >>>>> back-porting
>> >> >>>>>> to
>> >> >>>>>>>>>> it.  Our policy is to limit back-ports to critical bug
>> fixes and
>> >> >>>>> not
>> >> >>>>>>>>>> introduce any new features in the 3.4 line.  This is a
>> matter of
>> >> >>>>>>> managing
>> >> >>>>>>>>>> risk.
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Jason, your question about release cadence is a fair one.
>> If
>> >> it's
>> >> >>>>>> any
>> >> >>>>>>>>>> consolation, we are now taking the approach of trying to
>> limit
>> >> the
>> >> >>>>>>> scope
>> >> >>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us
>> to
>> >> >>>>> focus
>> >> >>>>>> on
>> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public
>> >> APIs.  I
>> >> >>>>>>> think
>> >> >>>>>>>>>> this will help us accelerate the releases into beta and
>> eventual
>> >> >>>>> GA.
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to
>> hear
>> >> >>>>>> thoughts
>> >> >>>>>>>>>> from more experienced committers and PMC members about your
>> >> >>>>> proposal
>> >> >>>>>> to
>> >> >>>>>>>>>> try to cut a stable release for a limited subset of what is
>> in
>> >> >>>>>>> branch-3.5
>> >> >>>>>>>>>> now.  My instinct is that it would be challenging to
>> cherry-pick
>> >> >>>>> out
>> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would
>> become
>> >> >>>>>>> another
>> >> >>>>>>>>>> release line for an already resource-constrained volunteer
>> >> staff to
>> >> >>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to
>> >> overall
>> >> >>>>>> 3.5
>> >> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain
>> features
>> >> >>>>>>> "vanished"
>> >> >>>>>>>>>> because of not meeting some stability criteria would be
>> >> >>>>> undesirable.
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> --Chris Nauroth
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com>
>> >> wrote:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>> Chris,
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable
>> than
>> >> >>>>> others
>> >> >>>>>>>>> (e.g.
>> >> >>>>>>>>>>> if we don't care about certain new features, is it
>> relatively
>> >> >>>>>> stable)?
>> >> >>>>>>>>>>> Would it be possible to cut out a version that only has the
>> >> bits
>> >> >>>>> we
>> >> >>>>>>>>> think
>> >> >>>>>>>>>>> are stable (and release that)?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> From that timeline, and the historic release cadence, it
>> would
>> >> >>>>> seem
>> >> >>>>>> to
>> >> >>>>>>>>> be
>> >> >>>>>>>>>>> a
>> >> >>>>>>>>>>> years away before we get to the stable release?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Thanks,
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Jason
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>> >> >>>>>>>>> cnauroth@hortonworks.com>
>> >> >>>>>>>>>>> wrote:
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>> Hello Doug,
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> At this point, I think we're still pretty far away from
>> >> >>>>> declaring a
>> >> >>>>>>>>>>>> stable
>> >> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough
>> >> that
>> >> >>>>>>> anyone
>> >> >>>>>>>>>>>> can
>> >> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
>> >> describes
>> >> >>>>> the
>> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> https://s.apache.org/ADK1
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>> >> >>>>>> resolving a
>> >> >>>>>>>>>>>> few
>> >> >>>>>>>>>>>> more blockers before we produce a release candidate.
>> >> Hopefully
>> >> >>>>>> that
>> >> >>>>>>>>>>>> will
>> >> >>>>>>>>>>>> get done in the next few weeks.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> --Chris Nauroth
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>> I know it's only been a few months, but I was wondering
>> if
>> >> there
>> >> >>>>>>> was a
>> >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or
>> is
>> >> there
>> >> >>>>>> any
>> >> >>>>>>>>>>>>> chance
>> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another
>> person
>> >> >>>>>> looking
>> >> >>>>>>>>> to
>> >> >>>>>>>>>>>>> have
>> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you do!
>> :)
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> --
>> >> >>>>>>>>>>>>> View this message in context:
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>
>> >> >>>>>>
>> >> >>>>>
>> >>
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>> >> >>>>>>>>>>>> e
>> >> >>>>>>>>>>>>> -tp7581744p7582136.html
>> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
>> >> Nabble.com.
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>
>> >> >>>>>
>> >> >
>> >>
>>
>
>

Re: Zookeeper with SSL release date

[Flavio Junqueira <fp...@apache.org>]

To recap what the goal was when we started the conversation, we were thinking of a way to move the 3.5 branch from alpha to stable faster. One suggested option was to make it possible to disable reconfiguration due to the security concerns, which is one key reason for having the 3.5 branch in alpha. We need to ask ourselves whether having a switch in any form would help us achieve our goal. If not, then having it is a moot point.

Assuming we have something to gain from such a switch, it does feel like a better option to separate concerns and not couple enabling reconfig to acls.

-Flavio
  
> On 01 Apr 2016, at 20:07, Shawn Heisey <ap...@elyograg.org> wrote:
> 
> On 4/1/2016 12:32 PM, Alexander Shraer wrote:
>> I think we have some flexibility here since reconfig is a new feature so we
>> could
>> choose to be concervative and release it first only to people that do use
>> ACLs, but
>> I don't feel strongly about it, either way.
> 
> I have no standing in this project, but if I did, I'd fight that
> proposal.  I might lose, but that wouldn't stop me from trying.  ACLs
> are a nice option, but I think they need to *remain* an option.
> 
> If I'm using feature Y, and feature X is not intimately related in a way
> that cannot be separated, X should *not* be required to use Y, or
> vice-versa.  I can't think of any good reason to require ACL enforcement
> before allowing dynamic ensemble membership.
> 
> Is there some aspect of this that I'm missing?  That could be the case. 
> I'm not deeply familiar with ZK, and even less with 3.5.
> 
> Thanks,
> Shawn
> 

Re: Zookeeper with SSL release date

[Shawn Heisey <ap...@elyograg.org>]

On 4/1/2016 12:32 PM, Alexander Shraer wrote:
> I think we have some flexibility here since reconfig is a new feature so we
> could
> choose to be concervative and release it first only to people that do use
> ACLs, but
> I don't feel strongly about it, either way.

I have no standing in this project, but if I did, I'd fight that
proposal.  I might lose, but that wouldn't stop me from trying.  ACLs
are a nice option, but I think they need to *remain* an option.

If I'm using feature Y, and feature X is not intimately related in a way
that cannot be separated, X should *not* be required to use Y, or
vice-versa.  I can't think of any good reason to require ACL enforcement
before allowing dynamic ensemble membership.

Is there some aspect of this that I'm missing?  That could be the case. 
I'm not deeply familiar with ZK, and even less with 3.5.

Thanks,
Shawn

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

Hi Shawn,

My proposal was in the following context - Flavio suggested to add new
flag(s)
to disable reconfig in order not to surprise users with new security
vulnerabilities
that arise from dynamic reconfiguration. My point was that we already have
such
a mechanism we could use - ACLs. But if we need to do that while also
allowing
unprotected use of reconfig for some users, perhaps a flag is a better
alternative.

I think we have some flexibility here since reconfig is a new feature so we
could
choose to be concervative and release it first only to people that do use
ACLs, but
I don't feel strongly about it, either way.

What do you think ?  Flavio, Patrick, what's your opinion on this ?

Cheers,
Alex

On Fri, Apr 1, 2016 at 10:16 AM, Shawn Heisey <ap...@elyograg.org> wrote:

>
> This is a potential worry even without reconfig -- a malicious person
> could change or delete the entire database ... yet many people
> (including me) run without ACLs.
>
> My ZK ensemble is in a network location that unauthorized people can't
> reach without finding and exploiting some vulnerability that has not yet
> reached my awareness.
>
> If somebody can gain access to the ZK machines, at least one of my
> public-facing servers is already compromised.  ZK will be very low on my
> list of things to worry about.  Chances

Re: Zookeeper with SSL release date

[Shawn Heisey <ap...@elyograg.org>]

On 4/1/2016 10:18 AM, Alexander Shraer wrote:
> Because using reconfig without ACLs any client can remove the servers (or
> replace them with a different set of servers
> or change their configuration parameters) and break the system.

This is a potential worry even without reconfig -- a malicious person
could change or delete the entire database ... yet many people
(including me) run without ACLs.

My ZK ensemble is in a network location that unauthorized people can't
reach without finding and exploiting some vulnerability that has not yet
reached my awareness.

If somebody can gain access to the ZK machines, at least one of my
public-facing servers is already compromised.  ZK will be very low on my
list of things to worry about.  Chances are that even if the attacker
figured out I was using ZK and where it lives, it would be extremely low
on THEIR list of priorities -- it doesn't contain any sensitive info,
and there are far more efficient ways to cause problems.

Thanks,
Shawn

Re: Zookeeper with SSL release date

[Jason Rosenberg <jb...@squareup.com>]

sure sounds like a bad policy not to allow reconfig without ACL's, but they
are essentially unrelated functionality, nonetheless....

On Fri, Apr 1, 2016 at 12:18 PM, Alexander Shraer <sh...@gmail.com> wrote:

> Because using reconfig without ACLs any client can remove the servers (or
> replace them with a different set of servers
> or change their configuration parameters) and break the system.
>
> On Fri, Apr 1, 2016 at 8:59 AM, Jason Rosenberg <jb...@squareup.com> wrote:
>
> > I think these orthogonal concerns.  Why limit reconfig to ACL users only?
> >
> > On Thu, Mar 31, 2016 at 11:37 PM, Alexander Shraer <sh...@gmail.com>
> > wrote:
> >
> > > Citing Patrick:
> > >
> > > > If you're running zk w/o security turned on and suddenly folks can do
> > > reconfig
> > > > operations it's going to potentially be a problem.
> > > ...
> > > > Rather than force people to turn on kerberos (etc...) we could
> instead
> > > > have the feature off
> > >
> > > From this I understood that the concern is mostly about users that
> DON'T
> > > use ACLs. My proposal is to disable
> > > reconfig/getconfig for all such users, forcing users who want reconfig
> to
> > > also use ACLs. Users who do use ACLs
> > > don't have to use reconfig and will have to set the ACLs on the config
> > > znode before they can use it.
> > >
> > > In preprequestprocessor where acls are checked for reconfig operation
> we
> > > can check that:
> > >
> > > skipACL = false && nodeRecord.acl != null && nodeRecord.acl.size() != 0
> > >
> > > meaning you're using ACLs, and have actually set ACLs on the config
> node.
> > >
> > > For getConfig its a bit trickier since its just a getData on the server
> > > side (for efficiency
> > > of reads, we avoided checking whether path == config znode). What we
> > could
> > > do is before sending
> > > the operation to the server check skipACL = false and maybe also issue
> a
> > > getACL call to check that
> > > nodeRecord.acl != null && nodeRecord.acl.size() != 0
> > > and only then issue a getData. This part is not air tight but its
> > probably
> > > sufficient.
> > >
> > > And of course we can emphasize the need for ACLs on this znode in the
> > > release.
> > >
> > >
> > > On Thu, Mar 31, 2016 at 1:11 PM, Flavio Junqueira <fp...@apache.org>
> > wrote:
> > >
> > > > I think Jason is saying that this is orthogonal in the following
> sense.
> > > > You set ACLs because you care about authentication/authorization in
> > your
> > > > cluster, but you may not want reconfig enabled, it just happened that
> > you
> > > > wanted to use ACLs.
> > > >
> > > > Perhaps you can elaborate a bit on how you think we can perform this
> > ACL
> > > > check? What would you check precisely?
> > > >
> > > > -Flavio
> > > >
> > > > > On 24 Mar 2016, at 21:19, Alexander Shraer <sh...@gmail.com>
> > wrote:
> > > > >
> > > > > I'm not so sure its orthogonal. The question is whether someone
> would
> > > > ever
> > > > > want to use reconfig without ACLs,
> > > > > as this allows any client to reconfigure the servers away or add a
> > > bunch
> > > > of
> > > > > servers that shouldn't be there :) and whether we should facilitate
> > > this
> > > > > knowing its insecure.
> > > > >
> > > > > Requiring ACLs solves the security concern for both reconfig and
> > > > getconfig.
> > > > > For example, if you don't want your clients to know the list of
> > > servers,
> > > > > limit their read permissions on the configuration znode.
> > > > >
> > > > > On Thu, Mar 24, 2016 at 2:11 PM, Jason Rosenberg <jbr@squareup.com
> >
> > > > wrote:
> > > > >
> > > > >> seems like an orthogonal requirement?
> > > > >>
> > > > >> On Thu, Mar 24, 2016 at 3:37 PM, Alexander Shraer <
> > shralex@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >>> How about a simpler alternative to the proposed flag for
> reconfig:
> > a
> > > > >> check
> > > > >>> in the code that requires ACLs to be set.
> > > > >>> If people want to use reconfig, they should use ACLs too.
> > > > >>>
> > > > >>> What do you think ?
> > > > >>>
> > > > >>> Alex
> > > > >>>
> > > > >>> On Mon, Mar 21, 2016 at 9:58 PM, Patrick Hunt <ph...@apache.org>
> > > > wrote:
> > > > >>>
> > > > >>>> I would say if in doubt add a safety. (a config parameter to
> turn
> > it
> > > > >>>> off). Cost is almost zero and worst case it will just give us
> > peace
> > > of
> > > > >>>> mind. ;-)
> > > > >>>>
> > > > >>>> Patrick
> > > > >>>>
> > > > >>>> On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <
> > > shralex@gmail.com>
> > > > >>>> wrote:
> > > > >>>>> ok, thanks for the suggestion, I'll look into it. For reconfig
> I
> > > > >> think
> > > > >>>> its
> > > > >>>>> pretty clear that its an admin
> > > > >>>>> functionality. I just always imagined that its controlled via
> > acls,
> > > > >>> but I
> > > > >>>>> understand
> > > > >>>>> the concerns now.
> > > > >>>>>
> > > > >>>>> getConfig returns the dynamic config (list of all servers with
> > all
> > > > >>> ports
> > > > >>>>> and quorum system if defined)
> > > > >>>>> and has an option to filter that info and just return the
> server
> > > > >>>> connection
> > > > >>>>> string (server and client port only).
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <
> phunt@apache.org>
> > > > >>> wrote:
> > > > >>>>>
> > > > >>>>>> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <
> > > > >> shralex@gmail.com>
> > > > >>>>>> wrote:
> > > > >>>>>>> I don't think that getConfig should be an admin
> functionality.
> > It
> > > > >> is
> > > > >>>>>>> essential for client-side re-balancing
> > > > >>>>>>> that we implemented (all clients shoudl be able to detect
> > > > >>>> configuration
> > > > >>>>>>> changes via getConfig). It could
> > > > >>>>>>> be hidden somewhat by defining higher-level re-balancing
> > > > >>>>>>> policies (ZOOKEEPER-2016)
> > > > >>>>>>> but there hasn't been enough progress on that. Perhaps
> instead
> > > > >>>> getConfig
> > > > >>>>>>> should be controlled
> > > > >>>>>>> by a separate flag ?
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>> I believe that the Hadoop community has something we could
> use:
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
> > > > >>>>>> (whether through annotations or just documenting it in the API
> > > > >>> javadoc)
> > > > >>>>>>
> > > > >>>>>> e.g. we could list getConfig as public/unstable for example
> and
> > > > >> still
> > > > >>>>>> ship it as GA. That would mark it as something that could
> change
> > > re
> > > > >>>>>> API policy.
> > > > >>>>>>
> > > > >>>>>> Is the entire config exposed through getConfig? If so then we
> > > might
> > > > >>>>>> want to enable/disable it with a flag similar to reconfig.
> Might
> > > be
> > > > >>>>>> safer to just do that if we're not sure.
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> Re classification - we could do the same thing with reconfig,
> > but
> > > I
> > > > >>>>>> think that would be a mistake. If we feel strongly where it
> > should
> > > > >>>>>> live long term we should just move it now.
> > > > >>>>>>
> > > > >>>>>> Patrick
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <
> > phunt@apache.org>
> > > > >>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <
> > > > >>> shralex@gmail.com
> > > > >>>>>
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>> Hi Patrick, Flavio,
> > > > >>>>>>>>>
> > > > >>>>>>>>> Since there seems to be consensus on this, I can add this
> > flag,
> > > > >>>> unless
> > > > >>>>>>>>> someone else wants to. I assume that getConfig should still
> > > > >> work
> > > > >>>>>>>> regardless
> > > > >>>>>>>>> of the flag ? is there a security concern with clients
> > knowing
> > > > >>> the
> > > > >>>>>> list
> > > > >>>>>>>> of
> > > > >>>>>>>>> servers?
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> We've always hidden that detail from users. We don't even
> > expose
> > > > >>>> which
> > > > >>>>>>>> server you're connected to today. I remember Ben (and
> perhaps
> > > > >>>> Flavio?)
> > > > >>>>>>>> highlighting this as important to maintain although I'm not
> > > super
> > > > >>>>>>>> familiar with the specifics on why. It made sense to me
> though
> > > > >> from
> > > > >>>>>>>> the perspective that we don't want clients to make
> assumptions
> > > > >> that
> > > > >>>>>>>> probably shouldn't.
> > > > >>>>>>>>
> > > > >>>>>>>> My thinking is that we should 1) add a config option to
> enable
> > > > >>>>>>>> reconfig (off by default), 2) move reconfig specific
> > > > >> functionality
> > > > >>>>>>>> from ZooKeeper.java (including getconfig) into an "admin"
> > > > >> package,
> > > > >>>>>>>> within say a class ZooKeeperAdmin, 3) document/test use of
> > ACLs
> > > > >> for
> > > > >>>>>>>> when folks do want to enable reconfig and are also worried
> > about
> > > > >>>> auth.
> > > > >>>>>>>> (e.g. turn on kerb)
> > > > >>>>>>>>
> > > > >>>>>>>> Again, I don't see any of this as a quality issue
> personally.
> > As
> > > > >>> such
> > > > >>>>>>>> I don't see why any of this (1-3) should hold up a
> 3.5.2-alpha
> > > if
> > > > >>> we
> > > > >>>>>>>> were interested in doing such a release. Adjusting the API
> > > should
> > > > >>> be
> > > > >>>>>>>> done before we move to "beta" though. Although that seems
> > like a
> > > > >>>>>>>> pretty mechanical (eclipse/idea) type refactoring?
> > > > >>>>>>>>
> > > > >>>>>>>> Patrick
> > > > >>>>>>>>
> > > > >>>>>>>>> Cheers,
> > > > >>>>>>>>> Alex
> > > > >>>>>>>>> On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org>
> > > > >>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>>> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <
> > > > >>> fpj@apache.org
> > > > >>>>>
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>>>> I gotta say that I'm not super excited about this option,
> > > > >> but
> > > > >>>> for
> > > > >>>>>> some
> > > > >>>>>>>>>> reason I ended up carrying the flag. To recap, I just
> raised
> > > > >>> this
> > > > >>>>>> option
> > > > >>>>>>>>>> because it seems that there are folks interested in
> features
> > > > >> in
> > > > >>>> 3.5
> > > > >>>>>> like
> > > > >>>>>>>>>> SSL and not necessarily in reconfiguration. SSL is
> important
> > > > >> and
> > > > >>>> to
> > > > >>>>>> take
> > > > >>>>>>>>>> Kafka as an example, it sucks that we can't have a whole
> set
> > > > >> up
> > > > >>>> using
> > > > >>>>>>>> SSL.
> > > > >>>>>>>>>> For ZK, the real questions are:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> 1- how fast can we make 3.5 stable?
> > > > >>>>>>>>>>> 2- would it be faster if we have a way of disabling
> > > > >>>>>> reconfiguration?
> > > > >>>>>>>>>>> 3- would enough users care about a stable 3.5 that has
> > > > >>>>>> reconfiguration
> > > > >>>>>>>>>> disabled?
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> It is taking a long time to get 3.5 to beta. There has
> been
> > > > >>> some
> > > > >>>>>> good
> > > > >>>>>>>>>> activity around 3.5.2 release, which is a great step, but
> it
> > > > >> is
> > > > >>>>>> unclear
> > > > >>>>>>>>>> when 3.5.3 is going to come and if we will be able to make
> > 3.5
> > > > >>>> beta
> > > > >>>>>>>> then.
> > > > >>>>>>>>>> Frankly, disabling reconfiguration sounds undesirable
> > because
> > > > >> it
> > > > >>>> is
> > > > >>>>>> an
> > > > >>>>>>>>>> important feature, but I currently don't use it in
> > production,
> > > > >>> so
> > > > >>>>>> from a
> > > > >>>>>>>>>> practical point of view, I can go both ways. Whether we go
> > > > >>> through
> > > > >>>>>> the
> > > > >>>>>>>>>> trouble of doing 2 depends on users interested in that
> > option
> > > > >>> and
> > > > >>>>>> folks
> > > > >>>>>>>>>> willing to implement it.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> To answer your question, Powell, my pseudo-proposal is
> kind
> > > > >>> of a
> > > > >>>>>> funny
> > > > >>>>>>>>>> option because once the feature is stable, then we
> wouldn't
> > > > >>> need a
> > > > >>>>>>>> switch
> > > > >>>>>>>>>> any longer, so there is not need of a deprecation path, we
> > > > >> just
> > > > >>>> start
> > > > >>>>>>>>>> ignoring it from the first beta release. Until it is beta,
> > I'd
> > > > >>> say
> > > > >>>>>> that
> > > > >>>>>>>>>> default is disabled.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> I would argue that we need this even when it does become
> > > > >> stable.
> > > > >>>> To
> > > > >>>>>> me
> > > > >>>>>>>>>> this is not a quality issue so much as it is an auth
> issue.
> > We
> > > > >>>> want
> > > > >>>>>> to
> > > > >>>>>>>>>> make it simple for folks to run a vanilla/old ZK cluster
> and
> > > > >> not
> > > > >>>>>> worry
> > > > >>>>>>>>>> about the security implications of having reconfig
> enabled.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Patrick
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> -Flavio
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>> On 17 Mar 2016, at 17:44, powell molleti
> > > > >>>>>> <powellm79@yahoo.com.INVALID
> > > > >>>>>>>>>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> Hi Flavio,
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> Generally do config options and command line args come
> > > > >> under
> > > > >>>> the
> > > > >>>>>> same
> > > > >>>>>>>>>> SLA as API?. I was assuming as such hence my question.
> > Perhaps
> > > > >>> if
> > > > >>>> the
> > > > >>>>>>>>>> expectation is that this config option is temporary from
> get
> > > > >> go
> > > > >>>> then
> > > > >>>>>>>> may be
> > > > >>>>>>>>>> it is ok. The default for re-config support will be
> enabled
> > or
> > > > >>>>>>>> disabled?.
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> I am just thinking from provisioning point of view when
> > > > >>> people
> > > > >>>>>>>> generate
> > > > >>>>>>>>>> config options etc.
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> Thanks
> > > > >>>>>>>>>>>> Powell.
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
> > > > >>>>>>>> fpj@apache.org>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>>> Hi Powell,
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> I was thinking config file and system property server
> > side.
> > > > >>>> What's
> > > > >>>>>>>> your
> > > > >>>>>>>>>> concern with compatibility? The API itself wouldn't
> change,
> > > > >> but
> > > > >>>> the
> > > > >>>>>>>> config
> > > > >>>>>>>>>> option wouldn't exist in previous versions and would not
> > exist
> > > > >>>>>> either in
> > > > >>>>>>>>>> later stable versions of 3.5.
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> -Flavio
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>> On 16 Mar 2016, at 22:08, powell molleti
> > > > >>>>>>>> <po...@yahoo.com.INVALID>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> Will this option be supplied via config file/args/API?.
> > > > >> Will
> > > > >>>> this
> > > > >>>>>>>>>> option be a temporary thing i.e what about its
> > compatibility?.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> thanks
> > > > >>>>>>>>>>>>> Powell.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira
> <
> > > > >>>>>>>> fpj@apache.org>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>> The main issue to sort out is stability of the API.
> There
> > > > >>> is a
> > > > >>>>>>>>>> security concern around the fact that clients can freely
> > > > >>>> reconfigure
> > > > >>>>>> the
> > > > >>>>>>>>>> ensemble. If we follow the plan that Pat proposed some
> time
> > > > >> ago:
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > > > >>>>>>>>>> <
> > > > >>>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> Locking the API is the main step to move it to beta.
> > > > >> Sorting
> > > > >>>> out
> > > > >>>>>>>> bugs
> > > > >>>>>>>>>> is definitely necessary, but it isn't the main thing that
> is
> > > > >>>> keeping
> > > > >>>>>>>> 3.5 in
> > > > >>>>>>>>>> alpha.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> About making it experimental, I was raising the option
> of
> > > > >>>> having
> > > > >>>>>> a
> > > > >>>>>>>>>> switch that disables the API calls, not the code. The
> reason
> > > > >> why
> > > > >>>> that
> > > > >>>>>>>> could
> > > > >>>>>>>>>> work is that anyone using 3.5 who uses the "experimental"
> > API
> > > > >>> must
> > > > >>>>>>>> explicit
> > > > >>>>>>>>>> turn on the switch and enable the calls. If they do it,
> they
> > > > >>> need
> > > > >>>> to
> > > > >>>>>> be
> > > > >>>>>>>>>> aware that the API can change.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> I must say that I haven't really looked closely into
> > doing
> > > > >>> it,
> > > > >>>>>> and
> > > > >>>>>>>> I'm
> > > > >>>>>>>>>> not even entirely convinced that this is a good idea, but
> > > > >> since
> > > > >>>> Jason
> > > > >>>>>>>>>> raised the point, I'm exploring options.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> -Flavio
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <
> > > > >>>> shralex@gmail.com>
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> Looking at the list of ~50 blocker and critical bugs
> in
> > > > >>>>>> ZooKeeper,
> > > > >>>>>>>>>> only 3-4
> > > > >>>>>>>>>>>>>> are related to reconfig. Given this, and the fact that
> > it
> > > > >>> is
> > > > >>>>>> run in
> > > > >>>>>>>>>>>>>> production since 2012 in multiple companies, I don't
> > > > >> think
> > > > >>>> its
> > > > >>>>>> more
> > > > >>>>>>>>>>>>>> unstable than any other part of ZooKeeper.
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> There are multiple reconfig-related bugs that turned
> out
> > > > >>>> really
> > > > >>>>>>>>>> difficult
> > > > >>>>>>>>>>>>>> to debug without access to the actual system or at
> least
> > > > >> to
> > > > >>>> the
> > > > >>>>>>>> Hudson
> > > > >>>>>>>>>>>>>> machines where some tests are failing. In the past,
> > Michi
> > > > >>>> and I
> > > > >>>>>>>>>> physically
> > > > >>>>>>>>>>>>>> went to Hortonworks to debug one such issue, but this
> is
> > > > >> of
> > > > >>>>>> course
> > > > >>>>>>>>>> not a
> > > > >>>>>>>>>>>>>> good method, and we weren't able to arrange such a
> visit
> > > > >>>> again.
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> Regarding making it optional - the reconfig logic has
> > > > >>> several
> > > > >>>>>>>>>> different
> > > > >>>>>>>>>>>>>> parts, some would be really difficult to disable
> using a
> > > > >>>>>>>> configuration
> > > > >>>>>>>>>>>>>> parameter. But the actual dynamic expansion of the
> > > > >> cluster
> > > > >>> is
> > > > >>>>>>>>>> triggered by
> > > > >>>>>>>>>>>>>> the reconfig command, so it should not affect users
> who
> > > > >>> don't
> > > > >>>>>>>> invoke
> > > > >>>>>>>>>> it.
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
> > > > >>>>>>>> fpj@apache.org>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> I suppose we could give it a try. How do other folks
> > > > >> feel
> > > > >>>> about
> > > > >>>>>>>> it?
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> -Flavio
> > > > >>>>>>>>>>>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <
> > > > >> jbr@squareup.com
> > > > >>>>
> > > > >>>>>> wrote:
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> So, you could enable the dynamic reconfiguration
> > > > >> feature
> > > > >>>>>> behind a
> > > > >>>>>>>>>> config
> > > > >>>>>>>>>>>>>>>> option, and document that it should only be enabled
> > > > >>>>>>>> experimentally,
> > > > >>>>>>>>>> use
> > > > >>>>>>>>>>>>>>> at
> > > > >>>>>>>>>>>>>>>> your own risk.  Keep it off by default.  Allow only
> > > > >>> static
> > > > >>>>>>>> config by
> > > > >>>>>>>>>>>>>>>> default, until it's stable?
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Jason
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
> > > > >>>>>>>> fpj@apache.org>
> > > > >>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> Hi Jason,
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> The consumer in Kafka is pretty independent from
> the
> > > > >>> core
> > > > >>>>>>>>>> (brokers),
> > > > >>>>>>>>>>>>>>>>> that's how that project manages to make such a
> > > > >>>> separation. We
> > > > >>>>>>>> don't
> > > > >>>>>>>>>>>>>>> have
> > > > >>>>>>>>>>>>>>>>> the same with ZooKeeper as the feature we are
> talking
> > > > >>>> about
> > > > >>>>>> is
> > > > >>>>>>>>>> part of
> > > > >>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>> server and the only way I see of doing what you say
> > is
> > > > >>> to
> > > > >>>>>> turn
> > > > >>>>>>>> off
> > > > >>>>>>>>>>>>>>>>> features. More specifically, we'd need to disable
> the
> > > > >>>>>> reconfig
> > > > >>>>>>>> API
> > > > >>>>>>>>>> and
> > > > >>>>>>>>>>>>>>> do
> > > > >>>>>>>>>>>>>>>>> not allow any change to the configuration, even
> > though
> > > > >>> the
> > > > >>>>>> code
> > > > >>>>>>>> is
> > > > >>>>>>>>>>>>>>> there.
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> Reconfig here refers to the ability of changing the
> > > > >>>>>>>> configuration
> > > > >>>>>>>>>> of an
> > > > >>>>>>>>>>>>>>>>> ensemble (e.g., changing the set of servers).
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> -Flavio
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <
> > > > >>>> jbr@squareup.com
> > > > >>>>>>>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> So, it would seem sensible to me to have a release
> > > > >>> where
> > > > >>>> all
> > > > >>>>>>>>>> features
> > > > >>>>>>>>>>>>>>>> are
> > > > >>>>>>>>>>>>>>>>>> stable, except where noted.  E.g. mark certain
> > > > >> features
> > > > >>>> as
> > > > >>>>>> only
> > > > >>>>>>>>>>>>>>> 'alpha
> > > > >>>>>>>>>>>>>>>>>> quality', e.g. the 're-config feature'.  (I assume
> > > > >> it's
> > > > >>>>>>>> possible
> > > > >>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>> happily
> > > > >>>>>>>>>>>>>>>>>> use 3.5.X without exercising the unstable
> re-config
> > > > >>>> bits?).
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> There's precedent for doing this sort of thing in
> > > > >> other
> > > > >>>>>>>> projects,
> > > > >>>>>>>>>>>>>>> e.g.
> > > > >>>>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>>> Kafka, they've had several release where a new
> > > > >>> "Consumer
> > > > >>>>>> API"
> > > > >>>>>>>> is
> > > > >>>>>>>>>>>>>>>> shipped
> > > > >>>>>>>>>>>>>>>>>> that is available for beta-testing, but you can
> > still
> > > > >>>> just
> > > > >>>>>> use
> > > > >>>>>>>> the
> > > > >>>>>>>>>>>>>>>> older
> > > > >>>>>>>>>>>>>>>>>> stable consumer api, etc.
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> Jason
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > > > >>>>>>>>>>>>>>>>> <powellm79@yahoo.com.invalid
> > > > >>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> Hi Doug,
> > > > >>>>>>>>>>>>>>>>>>> Is 3.5 being an alpha release preventing you from
> > > > >>> using
> > > > >>>>>> it?.
> > > > >>>>>>>> Or
> > > > >>>>>>>>>> have
> > > > >>>>>>>>>>>>>>>> you
> > > > >>>>>>>>>>>>>>>>>>> run into issues with it?. In general perhaps ZK
> 3.5
> > > > >>>> being
> > > > >>>>>>>>>> labeled as
> > > > >>>>>>>>>>>>>>>>> alpha
> > > > >>>>>>>>>>>>>>>>>>> might not be fair, since it is far more stable
> then
> > > > >>> what
> > > > >>>>>> most
> > > > >>>>>>>>>> people
> > > > >>>>>>>>>>>>>>>>>>> associate an alpha release to be.
> > > > >>>>>>>>>>>>>>>>>>> Perhaps if you do not use re-config feature may
> be
> > > > >> it
> > > > >>>> will
> > > > >>>>>>>> just
> > > > >>>>>>>>>> work
> > > > >>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>> you?.
> > > > >>>>>>>>>>>>>>>>>>> There are many examples of 3.5.X being used in
> > > > >>>> productions
> > > > >>>>>>>> from
> > > > >>>>>>>>>> my
> > > > >>>>>>>>>>>>>>>>> limited
> > > > >>>>>>>>>>>>>>>>>>> knowledge.
> > > > >>>>>>>>>>>>>>>>>>> ThanksPowell.
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio
> > > > >>> Junqueira <
> > > > >>>>>>>>>>>>>>>>> fpj@apache.org>
> > > > >>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> None of us expected the reconfig changes to take
> > > > >> this
> > > > >>>> long
> > > > >>>>>> to
> > > > >>>>>>>>>>>>>>>> stabilize.
> > > > >>>>>>>>>>>>>>>>>>> Until we get there, I don't think we can do
> > anything
> > > > >>>> else
> > > > >>>>>> with
> > > > >>>>>>>>>> 3.5.
> > > > >>>>>>>>>>>>>>>> The
> > > > >>>>>>>>>>>>>>>>>>> best bet we have is to work harder to bring 3.5
> > > > >> into a
> > > > >>>>>> stable
> > > > >>>>>>>>>> rather
> > > > >>>>>>>>>>>>>>>>> than
> > > > >>>>>>>>>>>>>>>>>>> trying to work around it.
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> There are lots of people interested in seeing 3.5
> > > > >>>> stable,
> > > > >>>>>> and
> > > > >>>>>>>> if
> > > > >>>>>>>>>> we
> > > > >>>>>>>>>>>>>>>> get
> > > > >>>>>>>>>>>>>>>>>>> everyone to contribute more patches and code
> > > > >> reviews,
> > > > >>> we
> > > > >>>>>>>> should
> > > > >>>>>>>>>> be
> > > > >>>>>>>>>>>>>>>> able
> > > > >>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>> do it sooner. After all, it is a community based
> > > > >>>> effort, so
> > > > >>>>>>>> the
> > > > >>>>>>>>>>>>>>>>> community
> > > > >>>>>>>>>>>>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> -Flavio
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> > > > >>>>>>>>>> cnauroth@hortonworks.com>
> > > > >>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> Doug, I forgot to respond to your question about
> > > > >> 3.4.
> > > > >>>>>> Since
> > > > >>>>>>>>>> 3.4 is
> > > > >>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>> stable maintenance line, we are very
> conservative
> > > > >>> about
> > > > >>>>>>>>>>>>>>> back-porting
> > > > >>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>> it.  Our policy is to limit back-ports to
> critical
> > > > >>> bug
> > > > >>>>>> fixes
> > > > >>>>>>>> and
> > > > >>>>>>>>>>>>>>> not
> > > > >>>>>>>>>>>>>>>>>>>> introduce any new features in the 3.4 line.
> This
> > > > >> is
> > > > >>> a
> > > > >>>>>>>> matter of
> > > > >>>>>>>>>>>>>>>>> managing
> > > > >>>>>>>>>>>>>>>>>>>> risk.
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> Jason, your question about release cadence is a
> > > > >> fair
> > > > >>>>>> one.  If
> > > > >>>>>>>>>> it's
> > > > >>>>>>>>>>>>>>>> any
> > > > >>>>>>>>>>>>>>>>>>>> consolation, we are now taking the approach of
> > > > >> trying
> > > > >>>> to
> > > > >>>>>>>> limit
> > > > >>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>> scope
> > > > >>>>>>>>>>>>>>>>>>>> of anything new introduced in 3.5 too.  That
> would
> > > > >>>> allow
> > > > >>>>>> us
> > > > >>>>>>>> to
> > > > >>>>>>>>>>>>>>> focus
> > > > >>>>>>>>>>>>>>>> on
> > > > >>>>>>>>>>>>>>>>>>>> stabilization: resolving blocker bugs and
> freezing
> > > > >>>> public
> > > > >>>>>>>>>> APIs.  I
> > > > >>>>>>>>>>>>>>>>> think
> > > > >>>>>>>>>>>>>>>>>>>> this will help us accelerate the releases into
> > beta
> > > > >>> and
> > > > >>>>>>>> eventual
> > > > >>>>>>>>>>>>>>> GA.
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> I am new to ZooKeeper release management, so I'd
> > > > >> like
> > > > >>>> to
> > > > >>>>>> hear
> > > > >>>>>>>>>>>>>>>> thoughts
> > > > >>>>>>>>>>>>>>>>>>>> from more experienced committers and PMC members
> > > > >>> about
> > > > >>>>>> your
> > > > >>>>>>>>>>>>>>> proposal
> > > > >>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>> try to cut a stable release for a limited subset
> > of
> > > > >>>> what
> > > > >>>>>> is
> > > > >>>>>>>> in
> > > > >>>>>>>>>>>>>>>>> branch-3.5
> > > > >>>>>>>>>>>>>>>>>>>> now.  My instinct is that it would be
> challenging
> > > > >> to
> > > > >>>>>>>> cherry-pick
> > > > >>>>>>>>>>>>>>> out
> > > > >>>>>>>>>>>>>>>>>>>> pieces of branch-3.5 piecemeal at this point.
> > This
> > > > >>>> would
> > > > >>>>>>>> become
> > > > >>>>>>>>>>>>>>>>> another
> > > > >>>>>>>>>>>>>>>>>>>> release line for an already resource-constrained
> > > > >>>> volunteer
> > > > >>>>>>>>>> staff to
> > > > >>>>>>>>>>>>>>>>>>>> manage.  I'd prefer to dedicate those limited
> > > > >>>> resources to
> > > > >>>>>>>>>> overall
> > > > >>>>>>>>>>>>>>>> 3.5
> > > > >>>>>>>>>>>>>>>>>>>> stabilization.  Also, a 3.5 release in which
> > > > >> certain
> > > > >>>>>> features
> > > > >>>>>>>>>>>>>>>>> "vanished"
> > > > >>>>>>>>>>>>>>>>>>>> because of not meeting some stability criteria
> > > > >> would
> > > > >>> be
> > > > >>>>>>>>>>>>>>> undesirable.
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> --Chris Nauroth
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <
> > > > >>>> jbr@squareup.com
> > > > >>>>>>>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> Chris,
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> Can you say whether some parts of 3.5.X are
> more
> > > > >>>> stable
> > > > >>>>>> than
> > > > >>>>>>>>>>>>>>> others
> > > > >>>>>>>>>>>>>>>>>>> (e.g.
> > > > >>>>>>>>>>>>>>>>>>>>> if we don't care about certain new features, is
> > it
> > > > >>>>>>>> relatively
> > > > >>>>>>>>>>>>>>>> stable)?
> > > > >>>>>>>>>>>>>>>>>>>>> Would it be possible to cut out a version that
> > > > >> only
> > > > >>>> has
> > > > >>>>>> the
> > > > >>>>>>>>>> bits
> > > > >>>>>>>>>>>>>>> we
> > > > >>>>>>>>>>>>>>>>>>> think
> > > > >>>>>>>>>>>>>>>>>>>>> are stable (and release that)?
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> From that timeline, and the historic release
> > > > >>> cadence,
> > > > >>>> it
> > > > >>>>>>>> would
> > > > >>>>>>>>>>>>>>> seem
> > > > >>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>> be
> > > > >>>>>>>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>> years away before we get to the stable release?
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> Jason
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth
> <
> > > > >>>>>>>>>>>>>>>>>>> cnauroth@hortonworks.com>
> > > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> Hello Doug,
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> Thanks for your interest in the SSL feature!
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> At this point, I think we're still pretty far
> > > > >> away
> > > > >>>> from
> > > > >>>>>>>>>>>>>>> declaring a
> > > > >>>>>>>>>>>>>>>>>>>>>> stable
> > > > >>>>>>>>>>>>>>>>>>>>>> release in the 3.5 line.  I don't think we're
> > > > >> close
> > > > >>>>>> enough
> > > > >>>>>>>>>> that
> > > > >>>>>>>>>>>>>>>>> anyone
> > > > >>>>>>>>>>>>>>>>>>>>>> can
> > > > >>>>>>>>>>>>>>>>>>>>>> offer a reliable ETA.  This is an earlier
> thread
> > > > >>> that
> > > > >>>>>>>>>> describes
> > > > >>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>> high-level strategy for release planning in
> the
> > > > >> 3.5
> > > > >>>>>> line:
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> https://s.apache.org/ADK1
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're
> > > > >>>> working
> > > > >>>>>> on
> > > > >>>>>>>>>>>>>>>> resolving a
> > > > >>>>>>>>>>>>>>>>>>>>>> few
> > > > >>>>>>>>>>>>>>>>>>>>>> more blockers before we produce a release
> > > > >>> candidate.
> > > > >>>>>>>>>> Hopefully
> > > > >>>>>>>>>>>>>>>> that
> > > > >>>>>>>>>>>>>>>>>>>>>> will
> > > > >>>>>>>>>>>>>>>>>>>>>> get done in the next few weeks.
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> --Chris Nauroth
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <
> > itsbehind@gmail.com
> > > > >>>
> > > > >>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> I know it's only been a few months, but I was
> > > > >>>>>> wondering if
> > > > >>>>>>>>>> there
> > > > >>>>>>>>>>>>>>>>> was a
> > > > >>>>>>>>>>>>>>>>>>>>>>> ballpark release date for a stable version of
> > > > >>> 3.5.1.
> > > > >>>>>> Or is
> > > > >>>>>>>>>> there
> > > > >>>>>>>>>>>>>>>> any
> > > > >>>>>>>>>>>>>>>>>>>>>>> chance
> > > > >>>>>>>>>>>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just
> > > > >>>> another
> > > > >>>>>>>> person
> > > > >>>>>>>>>>>>>>>> looking
> > > > >>>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>> have
> > > > >>>>>>>>>>>>>>>>>>>>>>> that feature in a stable version. Thanks for
> > all
> > > > >>> you
> > > > >>>>>> do!
> > > > >>>>>>>> :)
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>> View this message in context:
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > > > >>>>>>>>>>>>>>>>>>>>>> e
> > > > >>>>>>>>>>>>>>>>>>>>>>> -tp7581744p7582136.html
> > > > >>>>>>>>>>>>>>>>>>>>>>> Sent from the zookeeper-user mailing list
> > > > >> archive
> > > > >>> at
> > > > >>>>>>>>>> Nabble.com.
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

Because using reconfig without ACLs any client can remove the servers (or
replace them with a different set of servers
or change their configuration parameters) and break the system.

On Fri, Apr 1, 2016 at 8:59 AM, Jason Rosenberg <jb...@squareup.com> wrote:

> I think these orthogonal concerns.  Why limit reconfig to ACL users only?
>
> On Thu, Mar 31, 2016 at 11:37 PM, Alexander Shraer <sh...@gmail.com>
> wrote:
>
> > Citing Patrick:
> >
> > > If you're running zk w/o security turned on and suddenly folks can do
> > reconfig
> > > operations it's going to potentially be a problem.
> > ...
> > > Rather than force people to turn on kerberos (etc...) we could instead
> > > have the feature off
> >
> > From this I understood that the concern is mostly about users that DON'T
> > use ACLs. My proposal is to disable
> > reconfig/getconfig for all such users, forcing users who want reconfig to
> > also use ACLs. Users who do use ACLs
> > don't have to use reconfig and will have to set the ACLs on the config
> > znode before they can use it.
> >
> > In preprequestprocessor where acls are checked for reconfig operation we
> > can check that:
> >
> > skipACL = false && nodeRecord.acl != null && nodeRecord.acl.size() != 0
> >
> > meaning you're using ACLs, and have actually set ACLs on the config node.
> >
> > For getConfig its a bit trickier since its just a getData on the server
> > side (for efficiency
> > of reads, we avoided checking whether path == config znode). What we
> could
> > do is before sending
> > the operation to the server check skipACL = false and maybe also issue a
> > getACL call to check that
> > nodeRecord.acl != null && nodeRecord.acl.size() != 0
> > and only then issue a getData. This part is not air tight but its
> probably
> > sufficient.
> >
> > And of course we can emphasize the need for ACLs on this znode in the
> > release.
> >
> >
> > On Thu, Mar 31, 2016 at 1:11 PM, Flavio Junqueira <fp...@apache.org>
> wrote:
> >
> > > I think Jason is saying that this is orthogonal in the following sense.
> > > You set ACLs because you care about authentication/authorization in
> your
> > > cluster, but you may not want reconfig enabled, it just happened that
> you
> > > wanted to use ACLs.
> > >
> > > Perhaps you can elaborate a bit on how you think we can perform this
> ACL
> > > check? What would you check precisely?
> > >
> > > -Flavio
> > >
> > > > On 24 Mar 2016, at 21:19, Alexander Shraer <sh...@gmail.com>
> wrote:
> > > >
> > > > I'm not so sure its orthogonal. The question is whether someone would
> > > ever
> > > > want to use reconfig without ACLs,
> > > > as this allows any client to reconfigure the servers away or add a
> > bunch
> > > of
> > > > servers that shouldn't be there :) and whether we should facilitate
> > this
> > > > knowing its insecure.
> > > >
> > > > Requiring ACLs solves the security concern for both reconfig and
> > > getconfig.
> > > > For example, if you don't want your clients to know the list of
> > servers,
> > > > limit their read permissions on the configuration znode.
> > > >
> > > > On Thu, Mar 24, 2016 at 2:11 PM, Jason Rosenberg <jb...@squareup.com>
> > > wrote:
> > > >
> > > >> seems like an orthogonal requirement?
> > > >>
> > > >> On Thu, Mar 24, 2016 at 3:37 PM, Alexander Shraer <
> shralex@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> How about a simpler alternative to the proposed flag for reconfig:
> a
> > > >> check
> > > >>> in the code that requires ACLs to be set.
> > > >>> If people want to use reconfig, they should use ACLs too.
> > > >>>
> > > >>> What do you think ?
> > > >>>
> > > >>> Alex
> > > >>>
> > > >>> On Mon, Mar 21, 2016 at 9:58 PM, Patrick Hunt <ph...@apache.org>
> > > wrote:
> > > >>>
> > > >>>> I would say if in doubt add a safety. (a config parameter to turn
> it
> > > >>>> off). Cost is almost zero and worst case it will just give us
> peace
> > of
> > > >>>> mind. ;-)
> > > >>>>
> > > >>>> Patrick
> > > >>>>
> > > >>>> On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <
> > shralex@gmail.com>
> > > >>>> wrote:
> > > >>>>> ok, thanks for the suggestion, I'll look into it. For reconfig I
> > > >> think
> > > >>>> its
> > > >>>>> pretty clear that its an admin
> > > >>>>> functionality. I just always imagined that its controlled via
> acls,
> > > >>> but I
> > > >>>>> understand
> > > >>>>> the concerns now.
> > > >>>>>
> > > >>>>> getConfig returns the dynamic config (list of all servers with
> all
> > > >>> ports
> > > >>>>> and quorum system if defined)
> > > >>>>> and has an option to filter that info and just return the server
> > > >>>> connection
> > > >>>>> string (server and client port only).
> > > >>>>>
> > > >>>>>
> > > >>>>> On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org>
> > > >>> wrote:
> > > >>>>>
> > > >>>>>> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <
> > > >> shralex@gmail.com>
> > > >>>>>> wrote:
> > > >>>>>>> I don't think that getConfig should be an admin functionality.
> It
> > > >> is
> > > >>>>>>> essential for client-side re-balancing
> > > >>>>>>> that we implemented (all clients shoudl be able to detect
> > > >>>> configuration
> > > >>>>>>> changes via getConfig). It could
> > > >>>>>>> be hidden somewhat by defining higher-level re-balancing
> > > >>>>>>> policies (ZOOKEEPER-2016)
> > > >>>>>>> but there hasn't been enough progress on that. Perhaps instead
> > > >>>> getConfig
> > > >>>>>>> should be controlled
> > > >>>>>>> by a separate flag ?
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>> I believe that the Hadoop community has something we could use:
> > > >>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
> > > >>>>>> (whether through annotations or just documenting it in the API
> > > >>> javadoc)
> > > >>>>>>
> > > >>>>>> e.g. we could list getConfig as public/unstable for example and
> > > >> still
> > > >>>>>> ship it as GA. That would mark it as something that could change
> > re
> > > >>>>>> API policy.
> > > >>>>>>
> > > >>>>>> Is the entire config exposed through getConfig? If so then we
> > might
> > > >>>>>> want to enable/disable it with a flag similar to reconfig. Might
> > be
> > > >>>>>> safer to just do that if we're not sure.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Re classification - we could do the same thing with reconfig,
> but
> > I
> > > >>>>>> think that would be a mistake. If we feel strongly where it
> should
> > > >>>>>> live long term we should just move it now.
> > > >>>>>>
> > > >>>>>> Patrick
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <
> phunt@apache.org>
> > > >>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <
> > > >>> shralex@gmail.com
> > > >>>>>
> > > >>>>>>>> wrote:
> > > >>>>>>>>> Hi Patrick, Flavio,
> > > >>>>>>>>>
> > > >>>>>>>>> Since there seems to be consensus on this, I can add this
> flag,
> > > >>>> unless
> > > >>>>>>>>> someone else wants to. I assume that getConfig should still
> > > >> work
> > > >>>>>>>> regardless
> > > >>>>>>>>> of the flag ? is there a security concern with clients
> knowing
> > > >>> the
> > > >>>>>> list
> > > >>>>>>>> of
> > > >>>>>>>>> servers?
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> We've always hidden that detail from users. We don't even
> expose
> > > >>>> which
> > > >>>>>>>> server you're connected to today. I remember Ben (and perhaps
> > > >>>> Flavio?)
> > > >>>>>>>> highlighting this as important to maintain although I'm not
> > super
> > > >>>>>>>> familiar with the specifics on why. It made sense to me though
> > > >> from
> > > >>>>>>>> the perspective that we don't want clients to make assumptions
> > > >> that
> > > >>>>>>>> probably shouldn't.
> > > >>>>>>>>
> > > >>>>>>>> My thinking is that we should 1) add a config option to enable
> > > >>>>>>>> reconfig (off by default), 2) move reconfig specific
> > > >> functionality
> > > >>>>>>>> from ZooKeeper.java (including getconfig) into an "admin"
> > > >> package,
> > > >>>>>>>> within say a class ZooKeeperAdmin, 3) document/test use of
> ACLs
> > > >> for
> > > >>>>>>>> when folks do want to enable reconfig and are also worried
> about
> > > >>>> auth.
> > > >>>>>>>> (e.g. turn on kerb)
> > > >>>>>>>>
> > > >>>>>>>> Again, I don't see any of this as a quality issue personally.
> As
> > > >>> such
> > > >>>>>>>> I don't see why any of this (1-3) should hold up a 3.5.2-alpha
> > if
> > > >>> we
> > > >>>>>>>> were interested in doing such a release. Adjusting the API
> > should
> > > >>> be
> > > >>>>>>>> done before we move to "beta" though. Although that seems
> like a
> > > >>>>>>>> pretty mechanical (eclipse/idea) type refactoring?
> > > >>>>>>>>
> > > >>>>>>>> Patrick
> > > >>>>>>>>
> > > >>>>>>>>> Cheers,
> > > >>>>>>>>> Alex
> > > >>>>>>>>> On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org>
> > > >>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <
> > > >>> fpj@apache.org
> > > >>>>>
> > > >>>>>>>> wrote:
> > > >>>>>>>>>>> I gotta say that I'm not super excited about this option,
> > > >> but
> > > >>>> for
> > > >>>>>> some
> > > >>>>>>>>>> reason I ended up carrying the flag. To recap, I just raised
> > > >>> this
> > > >>>>>> option
> > > >>>>>>>>>> because it seems that there are folks interested in features
> > > >> in
> > > >>>> 3.5
> > > >>>>>> like
> > > >>>>>>>>>> SSL and not necessarily in reconfiguration. SSL is important
> > > >> and
> > > >>>> to
> > > >>>>>> take
> > > >>>>>>>>>> Kafka as an example, it sucks that we can't have a whole set
> > > >> up
> > > >>>> using
> > > >>>>>>>> SSL.
> > > >>>>>>>>>> For ZK, the real questions are:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> 1- how fast can we make 3.5 stable?
> > > >>>>>>>>>>> 2- would it be faster if we have a way of disabling
> > > >>>>>> reconfiguration?
> > > >>>>>>>>>>> 3- would enough users care about a stable 3.5 that has
> > > >>>>>> reconfiguration
> > > >>>>>>>>>> disabled?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> It is taking a long time to get 3.5 to beta. There has been
> > > >>> some
> > > >>>>>> good
> > > >>>>>>>>>> activity around 3.5.2 release, which is a great step, but it
> > > >> is
> > > >>>>>> unclear
> > > >>>>>>>>>> when 3.5.3 is going to come and if we will be able to make
> 3.5
> > > >>>> beta
> > > >>>>>>>> then.
> > > >>>>>>>>>> Frankly, disabling reconfiguration sounds undesirable
> because
> > > >> it
> > > >>>> is
> > > >>>>>> an
> > > >>>>>>>>>> important feature, but I currently don't use it in
> production,
> > > >>> so
> > > >>>>>> from a
> > > >>>>>>>>>> practical point of view, I can go both ways. Whether we go
> > > >>> through
> > > >>>>>> the
> > > >>>>>>>>>> trouble of doing 2 depends on users interested in that
> option
> > > >>> and
> > > >>>>>> folks
> > > >>>>>>>>>> willing to implement it.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> To answer your question, Powell, my pseudo-proposal is kind
> > > >>> of a
> > > >>>>>> funny
> > > >>>>>>>>>> option because once the feature is stable, then we wouldn't
> > > >>> need a
> > > >>>>>>>> switch
> > > >>>>>>>>>> any longer, so there is not need of a deprecation path, we
> > > >> just
> > > >>>> start
> > > >>>>>>>>>> ignoring it from the first beta release. Until it is beta,
> I'd
> > > >>> say
> > > >>>>>> that
> > > >>>>>>>>>> default is disabled.
> > > >>>>>>>>>>
> > > >>>>>>>>>> I would argue that we need this even when it does become
> > > >> stable.
> > > >>>> To
> > > >>>>>> me
> > > >>>>>>>>>> this is not a quality issue so much as it is an auth issue.
> We
> > > >>>> want
> > > >>>>>> to
> > > >>>>>>>>>> make it simple for folks to run a vanilla/old ZK cluster and
> > > >> not
> > > >>>>>> worry
> > > >>>>>>>>>> about the security implications of having reconfig enabled.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Patrick
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> -Flavio
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> On 17 Mar 2016, at 17:44, powell molleti
> > > >>>>>> <powellm79@yahoo.com.INVALID
> > > >>>>>>>>>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Hi Flavio,
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Generally do config options and command line args come
> > > >> under
> > > >>>> the
> > > >>>>>> same
> > > >>>>>>>>>> SLA as API?. I was assuming as such hence my question.
> Perhaps
> > > >>> if
> > > >>>> the
> > > >>>>>>>>>> expectation is that this config option is temporary from get
> > > >> go
> > > >>>> then
> > > >>>>>>>> may be
> > > >>>>>>>>>> it is ok. The default for re-config support will be enabled
> or
> > > >>>>>>>> disabled?.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> I am just thinking from provisioning point of view when
> > > >>> people
> > > >>>>>>>> generate
> > > >>>>>>>>>> config options etc.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Thanks
> > > >>>>>>>>>>>> Powell.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
> > > >>>>>>>> fpj@apache.org>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>>> Hi Powell,
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> I was thinking config file and system property server
> side.
> > > >>>> What's
> > > >>>>>>>> your
> > > >>>>>>>>>> concern with compatibility? The API itself wouldn't change,
> > > >> but
> > > >>>> the
> > > >>>>>>>> config
> > > >>>>>>>>>> option wouldn't exist in previous versions and would not
> exist
> > > >>>>>> either in
> > > >>>>>>>>>> later stable versions of 3.5.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> -Flavio
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> On 16 Mar 2016, at 22:08, powell molleti
> > > >>>>>>>> <po...@yahoo.com.INVALID>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Will this option be supplied via config file/args/API?.
> > > >> Will
> > > >>>> this
> > > >>>>>>>>>> option be a temporary thing i.e what about its
> compatibility?.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> thanks
> > > >>>>>>>>>>>>> Powell.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
> > > >>>>>>>> fpj@apache.org>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>>>> The main issue to sort out is stability of the API. There
> > > >>> is a
> > > >>>>>>>>>> security concern around the fact that clients can freely
> > > >>>> reconfigure
> > > >>>>>> the
> > > >>>>>>>>>> ensemble. If we follow the plan that Pat proposed some time
> > > >> ago:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > > >>>>>>>>>> <
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Locking the API is the main step to move it to beta.
> > > >> Sorting
> > > >>>> out
> > > >>>>>>>> bugs
> > > >>>>>>>>>> is definitely necessary, but it isn't the main thing that is
> > > >>>> keeping
> > > >>>>>>>> 3.5 in
> > > >>>>>>>>>> alpha.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> About making it experimental, I was raising the option of
> > > >>>> having
> > > >>>>>> a
> > > >>>>>>>>>> switch that disables the API calls, not the code. The reason
> > > >> why
> > > >>>> that
> > > >>>>>>>> could
> > > >>>>>>>>>> work is that anyone using 3.5 who uses the "experimental"
> API
> > > >>> must
> > > >>>>>>>> explicit
> > > >>>>>>>>>> turn on the switch and enable the calls. If they do it, they
> > > >>> need
> > > >>>> to
> > > >>>>>> be
> > > >>>>>>>>>> aware that the API can change.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I must say that I haven't really looked closely into
> doing
> > > >>> it,
> > > >>>>>> and
> > > >>>>>>>> I'm
> > > >>>>>>>>>> not even entirely convinced that this is a good idea, but
> > > >> since
> > > >>>> Jason
> > > >>>>>>>>>> raised the point, I'm exploring options.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> -Flavio
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <
> > > >>>> shralex@gmail.com>
> > > >>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Looking at the list of ~50 blocker and critical bugs in
> > > >>>>>> ZooKeeper,
> > > >>>>>>>>>> only 3-4
> > > >>>>>>>>>>>>>> are related to reconfig. Given this, and the fact that
> it
> > > >>> is
> > > >>>>>> run in
> > > >>>>>>>>>>>>>> production since 2012 in multiple companies, I don't
> > > >> think
> > > >>>> its
> > > >>>>>> more
> > > >>>>>>>>>>>>>> unstable than any other part of ZooKeeper.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> There are multiple reconfig-related bugs that turned out
> > > >>>> really
> > > >>>>>>>>>> difficult
> > > >>>>>>>>>>>>>> to debug without access to the actual system or at least
> > > >> to
> > > >>>> the
> > > >>>>>>>> Hudson
> > > >>>>>>>>>>>>>> machines where some tests are failing. In the past,
> Michi
> > > >>>> and I
> > > >>>>>>>>>> physically
> > > >>>>>>>>>>>>>> went to Hortonworks to debug one such issue, but this is
> > > >> of
> > > >>>>>> course
> > > >>>>>>>>>> not a
> > > >>>>>>>>>>>>>> good method, and we weren't able to arrange such a visit
> > > >>>> again.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Regarding making it optional - the reconfig logic has
> > > >>> several
> > > >>>>>>>>>> different
> > > >>>>>>>>>>>>>> parts, some would be really difficult to disable using a
> > > >>>>>>>> configuration
> > > >>>>>>>>>>>>>> parameter. But the actual dynamic expansion of the
> > > >> cluster
> > > >>> is
> > > >>>>>>>>>> triggered by
> > > >>>>>>>>>>>>>> the reconfig command, so it should not affect users who
> > > >>> don't
> > > >>>>>>>> invoke
> > > >>>>>>>>>> it.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
> > > >>>>>>>> fpj@apache.org>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> I suppose we could give it a try. How do other folks
> > > >> feel
> > > >>>> about
> > > >>>>>>>> it?
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> -Flavio
> > > >>>>>>>>>>>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <
> > > >> jbr@squareup.com
> > > >>>>
> > > >>>>>> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> So, you could enable the dynamic reconfiguration
> > > >> feature
> > > >>>>>> behind a
> > > >>>>>>>>>> config
> > > >>>>>>>>>>>>>>>> option, and document that it should only be enabled
> > > >>>>>>>> experimentally,
> > > >>>>>>>>>> use
> > > >>>>>>>>>>>>>>> at
> > > >>>>>>>>>>>>>>>> your own risk.  Keep it off by default.  Allow only
> > > >>> static
> > > >>>>>>>> config by
> > > >>>>>>>>>>>>>>>> default, until it's stable?
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Jason
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
> > > >>>>>>>> fpj@apache.org>
> > > >>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Hi Jason,
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> The consumer in Kafka is pretty independent from the
> > > >>> core
> > > >>>>>>>>>> (brokers),
> > > >>>>>>>>>>>>>>>>> that's how that project manages to make such a
> > > >>>> separation. We
> > > >>>>>>>> don't
> > > >>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>> the same with ZooKeeper as the feature we are talking
> > > >>>> about
> > > >>>>>> is
> > > >>>>>>>>>> part of
> > > >>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>> server and the only way I see of doing what you say
> is
> > > >>> to
> > > >>>>>> turn
> > > >>>>>>>> off
> > > >>>>>>>>>>>>>>>>> features. More specifically, we'd need to disable the
> > > >>>>>> reconfig
> > > >>>>>>>> API
> > > >>>>>>>>>> and
> > > >>>>>>>>>>>>>>> do
> > > >>>>>>>>>>>>>>>>> not allow any change to the configuration, even
> though
> > > >>> the
> > > >>>>>> code
> > > >>>>>>>> is
> > > >>>>>>>>>>>>>>> there.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Reconfig here refers to the ability of changing the
> > > >>>>>>>> configuration
> > > >>>>>>>>>> of an
> > > >>>>>>>>>>>>>>>>> ensemble (e.g., changing the set of servers).
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> -Flavio
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <
> > > >>>> jbr@squareup.com
> > > >>>>>>>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> So, it would seem sensible to me to have a release
> > > >>> where
> > > >>>> all
> > > >>>>>>>>>> features
> > > >>>>>>>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>> stable, except where noted.  E.g. mark certain
> > > >> features
> > > >>>> as
> > > >>>>>> only
> > > >>>>>>>>>>>>>>> 'alpha
> > > >>>>>>>>>>>>>>>>>> quality', e.g. the 're-config feature'.  (I assume
> > > >> it's
> > > >>>>>>>> possible
> > > >>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>> happily
> > > >>>>>>>>>>>>>>>>>> use 3.5.X without exercising the unstable re-config
> > > >>>> bits?).
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> There's precedent for doing this sort of thing in
> > > >> other
> > > >>>>>>>> projects,
> > > >>>>>>>>>>>>>>> e.g.
> > > >>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> Kafka, they've had several release where a new
> > > >>> "Consumer
> > > >>>>>> API"
> > > >>>>>>>> is
> > > >>>>>>>>>>>>>>>> shipped
> > > >>>>>>>>>>>>>>>>>> that is available for beta-testing, but you can
> still
> > > >>>> just
> > > >>>>>> use
> > > >>>>>>>> the
> > > >>>>>>>>>>>>>>>> older
> > > >>>>>>>>>>>>>>>>>> stable consumer api, etc.
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> Jason
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > > >>>>>>>>>>>>>>>>> <powellm79@yahoo.com.invalid
> > > >>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Hi Doug,
> > > >>>>>>>>>>>>>>>>>>> Is 3.5 being an alpha release preventing you from
> > > >>> using
> > > >>>>>> it?.
> > > >>>>>>>> Or
> > > >>>>>>>>>> have
> > > >>>>>>>>>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>>> run into issues with it?. In general perhaps ZK 3.5
> > > >>>> being
> > > >>>>>>>>>> labeled as
> > > >>>>>>>>>>>>>>>>> alpha
> > > >>>>>>>>>>>>>>>>>>> might not be fair, since it is far more stable then
> > > >>> what
> > > >>>>>> most
> > > >>>>>>>>>> people
> > > >>>>>>>>>>>>>>>>>>> associate an alpha release to be.
> > > >>>>>>>>>>>>>>>>>>> Perhaps if you do not use re-config feature may be
> > > >> it
> > > >>>> will
> > > >>>>>>>> just
> > > >>>>>>>>>> work
> > > >>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>> you?.
> > > >>>>>>>>>>>>>>>>>>> There are many examples of 3.5.X being used in
> > > >>>> productions
> > > >>>>>>>> from
> > > >>>>>>>>>> my
> > > >>>>>>>>>>>>>>>>> limited
> > > >>>>>>>>>>>>>>>>>>> knowledge.
> > > >>>>>>>>>>>>>>>>>>> ThanksPowell.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio
> > > >>> Junqueira <
> > > >>>>>>>>>>>>>>>>> fpj@apache.org>
> > > >>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> None of us expected the reconfig changes to take
> > > >> this
> > > >>>> long
> > > >>>>>> to
> > > >>>>>>>>>>>>>>>> stabilize.
> > > >>>>>>>>>>>>>>>>>>> Until we get there, I don't think we can do
> anything
> > > >>>> else
> > > >>>>>> with
> > > >>>>>>>>>> 3.5.
> > > >>>>>>>>>>>>>>>> The
> > > >>>>>>>>>>>>>>>>>>> best bet we have is to work harder to bring 3.5
> > > >> into a
> > > >>>>>> stable
> > > >>>>>>>>>> rather
> > > >>>>>>>>>>>>>>>>> than
> > > >>>>>>>>>>>>>>>>>>> trying to work around it.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> There are lots of people interested in seeing 3.5
> > > >>>> stable,
> > > >>>>>> and
> > > >>>>>>>> if
> > > >>>>>>>>>> we
> > > >>>>>>>>>>>>>>>> get
> > > >>>>>>>>>>>>>>>>>>> everyone to contribute more patches and code
> > > >> reviews,
> > > >>> we
> > > >>>>>>>> should
> > > >>>>>>>>>> be
> > > >>>>>>>>>>>>>>>> able
> > > >>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>> do it sooner. After all, it is a community based
> > > >>>> effort, so
> > > >>>>>>>> the
> > > >>>>>>>>>>>>>>>>> community
> > > >>>>>>>>>>>>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> -Flavio
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> > > >>>>>>>>>> cnauroth@hortonworks.com>
> > > >>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> Doug, I forgot to respond to your question about
> > > >> 3.4.
> > > >>>>>> Since
> > > >>>>>>>>>> 3.4 is
> > > >>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> stable maintenance line, we are very conservative
> > > >>> about
> > > >>>>>>>>>>>>>>> back-porting
> > > >>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>> it.  Our policy is to limit back-ports to critical
> > > >>> bug
> > > >>>>>> fixes
> > > >>>>>>>> and
> > > >>>>>>>>>>>>>>> not
> > > >>>>>>>>>>>>>>>>>>>> introduce any new features in the 3.4 line.  This
> > > >> is
> > > >>> a
> > > >>>>>>>> matter of
> > > >>>>>>>>>>>>>>>>> managing
> > > >>>>>>>>>>>>>>>>>>>> risk.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> Jason, your question about release cadence is a
> > > >> fair
> > > >>>>>> one.  If
> > > >>>>>>>>>> it's
> > > >>>>>>>>>>>>>>>> any
> > > >>>>>>>>>>>>>>>>>>>> consolation, we are now taking the approach of
> > > >> trying
> > > >>>> to
> > > >>>>>>>> limit
> > > >>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>> scope
> > > >>>>>>>>>>>>>>>>>>>> of anything new introduced in 3.5 too.  That would
> > > >>>> allow
> > > >>>>>> us
> > > >>>>>>>> to
> > > >>>>>>>>>>>>>>> focus
> > > >>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>> stabilization: resolving blocker bugs and freezing
> > > >>>> public
> > > >>>>>>>>>> APIs.  I
> > > >>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>> this will help us accelerate the releases into
> beta
> > > >>> and
> > > >>>>>>>> eventual
> > > >>>>>>>>>>>>>>> GA.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> I am new to ZooKeeper release management, so I'd
> > > >> like
> > > >>>> to
> > > >>>>>> hear
> > > >>>>>>>>>>>>>>>> thoughts
> > > >>>>>>>>>>>>>>>>>>>> from more experienced committers and PMC members
> > > >>> about
> > > >>>>>> your
> > > >>>>>>>>>>>>>>> proposal
> > > >>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>> try to cut a stable release for a limited subset
> of
> > > >>>> what
> > > >>>>>> is
> > > >>>>>>>> in
> > > >>>>>>>>>>>>>>>>> branch-3.5
> > > >>>>>>>>>>>>>>>>>>>> now.  My instinct is that it would be challenging
> > > >> to
> > > >>>>>>>> cherry-pick
> > > >>>>>>>>>>>>>>> out
> > > >>>>>>>>>>>>>>>>>>>> pieces of branch-3.5 piecemeal at this point.
> This
> > > >>>> would
> > > >>>>>>>> become
> > > >>>>>>>>>>>>>>>>> another
> > > >>>>>>>>>>>>>>>>>>>> release line for an already resource-constrained
> > > >>>> volunteer
> > > >>>>>>>>>> staff to
> > > >>>>>>>>>>>>>>>>>>>> manage.  I'd prefer to dedicate those limited
> > > >>>> resources to
> > > >>>>>>>>>> overall
> > > >>>>>>>>>>>>>>>> 3.5
> > > >>>>>>>>>>>>>>>>>>>> stabilization.  Also, a 3.5 release in which
> > > >> certain
> > > >>>>>> features
> > > >>>>>>>>>>>>>>>>> "vanished"
> > > >>>>>>>>>>>>>>>>>>>> because of not meeting some stability criteria
> > > >> would
> > > >>> be
> > > >>>>>>>>>>>>>>> undesirable.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> --Chris Nauroth
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <
> > > >>>> jbr@squareup.com
> > > >>>>>>>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Chris,
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Can you say whether some parts of 3.5.X are more
> > > >>>> stable
> > > >>>>>> than
> > > >>>>>>>>>>>>>>> others
> > > >>>>>>>>>>>>>>>>>>> (e.g.
> > > >>>>>>>>>>>>>>>>>>>>> if we don't care about certain new features, is
> it
> > > >>>>>>>> relatively
> > > >>>>>>>>>>>>>>>> stable)?
> > > >>>>>>>>>>>>>>>>>>>>> Would it be possible to cut out a version that
> > > >> only
> > > >>>> has
> > > >>>>>> the
> > > >>>>>>>>>> bits
> > > >>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>> are stable (and release that)?
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> From that timeline, and the historic release
> > > >>> cadence,
> > > >>>> it
> > > >>>>>>>> would
> > > >>>>>>>>>>>>>>> seem
> > > >>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>> years away before we get to the stable release?
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Jason
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> > > >>>>>>>>>>>>>>>>>>> cnauroth@hortonworks.com>
> > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Hello Doug,
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Thanks for your interest in the SSL feature!
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> At this point, I think we're still pretty far
> > > >> away
> > > >>>> from
> > > >>>>>>>>>>>>>>> declaring a
> > > >>>>>>>>>>>>>>>>>>>>>> stable
> > > >>>>>>>>>>>>>>>>>>>>>> release in the 3.5 line.  I don't think we're
> > > >> close
> > > >>>>>> enough
> > > >>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>> anyone
> > > >>>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread
> > > >>> that
> > > >>>>>>>>>> describes
> > > >>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> high-level strategy for release planning in the
> > > >> 3.5
> > > >>>>>> line:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> https://s.apache.org/ADK1
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're
> > > >>>> working
> > > >>>>>> on
> > > >>>>>>>>>>>>>>>> resolving a
> > > >>>>>>>>>>>>>>>>>>>>>> few
> > > >>>>>>>>>>>>>>>>>>>>>> more blockers before we produce a release
> > > >>> candidate.
> > > >>>>>>>>>> Hopefully
> > > >>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>>>> get done in the next few weeks.
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> --Chris Nauroth
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <
> itsbehind@gmail.com
> > > >>>
> > > >>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> I know it's only been a few months, but I was
> > > >>>>>> wondering if
> > > >>>>>>>>>> there
> > > >>>>>>>>>>>>>>>>> was a
> > > >>>>>>>>>>>>>>>>>>>>>>> ballpark release date for a stable version of
> > > >>> 3.5.1.
> > > >>>>>> Or is
> > > >>>>>>>>>> there
> > > >>>>>>>>>>>>>>>> any
> > > >>>>>>>>>>>>>>>>>>>>>>> chance
> > > >>>>>>>>>>>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just
> > > >>>> another
> > > >>>>>>>> person
> > > >>>>>>>>>>>>>>>> looking
> > > >>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>> that feature in a stable version. Thanks for
> all
> > > >>> you
> > > >>>>>> do!
> > > >>>>>>>> :)
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>> View this message in context:
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > > >>>>>>>>>>>>>>>>>>>>>> e
> > > >>>>>>>>>>>>>>>>>>>>>>> -tp7581744p7582136.html
> > > >>>>>>>>>>>>>>>>>>>>>>> Sent from the zookeeper-user mailing list
> > > >> archive
> > > >>> at
> > > >>>>>>>>>> Nabble.com.
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: Zookeeper with SSL release date

[Jason Rosenberg <jb...@squareup.com>]

I think these orthogonal concerns.  Why limit reconfig to ACL users only?

On Thu, Mar 31, 2016 at 11:37 PM, Alexander Shraer <sh...@gmail.com>
wrote:

> Citing Patrick:
>
> > If you're running zk w/o security turned on and suddenly folks can do
> reconfig
> > operations it's going to potentially be a problem.
> ...
> > Rather than force people to turn on kerberos (etc...) we could instead
> > have the feature off
>
> From this I understood that the concern is mostly about users that DON'T
> use ACLs. My proposal is to disable
> reconfig/getconfig for all such users, forcing users who want reconfig to
> also use ACLs. Users who do use ACLs
> don't have to use reconfig and will have to set the ACLs on the config
> znode before they can use it.
>
> In preprequestprocessor where acls are checked for reconfig operation we
> can check that:
>
> skipACL = false && nodeRecord.acl != null && nodeRecord.acl.size() != 0
>
> meaning you're using ACLs, and have actually set ACLs on the config node.
>
> For getConfig its a bit trickier since its just a getData on the server
> side (for efficiency
> of reads, we avoided checking whether path == config znode). What we could
> do is before sending
> the operation to the server check skipACL = false and maybe also issue a
> getACL call to check that
> nodeRecord.acl != null && nodeRecord.acl.size() != 0
> and only then issue a getData. This part is not air tight but its probably
> sufficient.
>
> And of course we can emphasize the need for ACLs on this znode in the
> release.
>
>
> On Thu, Mar 31, 2016 at 1:11 PM, Flavio Junqueira <fp...@apache.org> wrote:
>
> > I think Jason is saying that this is orthogonal in the following sense.
> > You set ACLs because you care about authentication/authorization in your
> > cluster, but you may not want reconfig enabled, it just happened that you
> > wanted to use ACLs.
> >
> > Perhaps you can elaborate a bit on how you think we can perform this ACL
> > check? What would you check precisely?
> >
> > -Flavio
> >
> > > On 24 Mar 2016, at 21:19, Alexander Shraer <sh...@gmail.com> wrote:
> > >
> > > I'm not so sure its orthogonal. The question is whether someone would
> > ever
> > > want to use reconfig without ACLs,
> > > as this allows any client to reconfigure the servers away or add a
> bunch
> > of
> > > servers that shouldn't be there :) and whether we should facilitate
> this
> > > knowing its insecure.
> > >
> > > Requiring ACLs solves the security concern for both reconfig and
> > getconfig.
> > > For example, if you don't want your clients to know the list of
> servers,
> > > limit their read permissions on the configuration znode.
> > >
> > > On Thu, Mar 24, 2016 at 2:11 PM, Jason Rosenberg <jb...@squareup.com>
> > wrote:
> > >
> > >> seems like an orthogonal requirement?
> > >>
> > >> On Thu, Mar 24, 2016 at 3:37 PM, Alexander Shraer <sh...@gmail.com>
> > >> wrote:
> > >>
> > >>> How about a simpler alternative to the proposed flag for reconfig: a
> > >> check
> > >>> in the code that requires ACLs to be set.
> > >>> If people want to use reconfig, they should use ACLs too.
> > >>>
> > >>> What do you think ?
> > >>>
> > >>> Alex
> > >>>
> > >>> On Mon, Mar 21, 2016 at 9:58 PM, Patrick Hunt <ph...@apache.org>
> > wrote:
> > >>>
> > >>>> I would say if in doubt add a safety. (a config parameter to turn it
> > >>>> off). Cost is almost zero and worst case it will just give us peace
> of
> > >>>> mind. ;-)
> > >>>>
> > >>>> Patrick
> > >>>>
> > >>>> On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <
> shralex@gmail.com>
> > >>>> wrote:
> > >>>>> ok, thanks for the suggestion, I'll look into it. For reconfig I
> > >> think
> > >>>> its
> > >>>>> pretty clear that its an admin
> > >>>>> functionality. I just always imagined that its controlled via acls,
> > >>> but I
> > >>>>> understand
> > >>>>> the concerns now.
> > >>>>>
> > >>>>> getConfig returns the dynamic config (list of all servers with all
> > >>> ports
> > >>>>> and quorum system if defined)
> > >>>>> and has an option to filter that info and just return the server
> > >>>> connection
> > >>>>> string (server and client port only).
> > >>>>>
> > >>>>>
> > >>>>> On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org>
> > >>> wrote:
> > >>>>>
> > >>>>>> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <
> > >> shralex@gmail.com>
> > >>>>>> wrote:
> > >>>>>>> I don't think that getConfig should be an admin functionality. It
> > >> is
> > >>>>>>> essential for client-side re-balancing
> > >>>>>>> that we implemented (all clients shoudl be able to detect
> > >>>> configuration
> > >>>>>>> changes via getConfig). It could
> > >>>>>>> be hidden somewhat by defining higher-level re-balancing
> > >>>>>>> policies (ZOOKEEPER-2016)
> > >>>>>>> but there hasn't been enough progress on that. Perhaps instead
> > >>>> getConfig
> > >>>>>>> should be controlled
> > >>>>>>> by a separate flag ?
> > >>>>>>>
> > >>>>>>
> > >>>>>> I believe that the Hadoop community has something we could use:
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
> > >>>>>> (whether through annotations or just documenting it in the API
> > >>> javadoc)
> > >>>>>>
> > >>>>>> e.g. we could list getConfig as public/unstable for example and
> > >> still
> > >>>>>> ship it as GA. That would mark it as something that could change
> re
> > >>>>>> API policy.
> > >>>>>>
> > >>>>>> Is the entire config exposed through getConfig? If so then we
> might
> > >>>>>> want to enable/disable it with a flag similar to reconfig. Might
> be
> > >>>>>> safer to just do that if we're not sure.
> > >>>>>>
> > >>>>>>
> > >>>>>> Re classification - we could do the same thing with reconfig, but
> I
> > >>>>>> think that would be a mistake. If we feel strongly where it should
> > >>>>>> live long term we should just move it now.
> > >>>>>>
> > >>>>>> Patrick
> > >>>>>>
> > >>>>>>>
> > >>>>>>> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org>
> > >>>> wrote:
> > >>>>>>>
> > >>>>>>>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <
> > >>> shralex@gmail.com
> > >>>>>
> > >>>>>>>> wrote:
> > >>>>>>>>> Hi Patrick, Flavio,
> > >>>>>>>>>
> > >>>>>>>>> Since there seems to be consensus on this, I can add this flag,
> > >>>> unless
> > >>>>>>>>> someone else wants to. I assume that getConfig should still
> > >> work
> > >>>>>>>> regardless
> > >>>>>>>>> of the flag ? is there a security concern with clients knowing
> > >>> the
> > >>>>>> list
> > >>>>>>>> of
> > >>>>>>>>> servers?
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>> We've always hidden that detail from users. We don't even expose
> > >>>> which
> > >>>>>>>> server you're connected to today. I remember Ben (and perhaps
> > >>>> Flavio?)
> > >>>>>>>> highlighting this as important to maintain although I'm not
> super
> > >>>>>>>> familiar with the specifics on why. It made sense to me though
> > >> from
> > >>>>>>>> the perspective that we don't want clients to make assumptions
> > >> that
> > >>>>>>>> probably shouldn't.
> > >>>>>>>>
> > >>>>>>>> My thinking is that we should 1) add a config option to enable
> > >>>>>>>> reconfig (off by default), 2) move reconfig specific
> > >> functionality
> > >>>>>>>> from ZooKeeper.java (including getconfig) into an "admin"
> > >> package,
> > >>>>>>>> within say a class ZooKeeperAdmin, 3) document/test use of ACLs
> > >> for
> > >>>>>>>> when folks do want to enable reconfig and are also worried about
> > >>>> auth.
> > >>>>>>>> (e.g. turn on kerb)
> > >>>>>>>>
> > >>>>>>>> Again, I don't see any of this as a quality issue personally. As
> > >>> such
> > >>>>>>>> I don't see why any of this (1-3) should hold up a 3.5.2-alpha
> if
> > >>> we
> > >>>>>>>> were interested in doing such a release. Adjusting the API
> should
> > >>> be
> > >>>>>>>> done before we move to "beta" though. Although that seems like a
> > >>>>>>>> pretty mechanical (eclipse/idea) type refactoring?
> > >>>>>>>>
> > >>>>>>>> Patrick
> > >>>>>>>>
> > >>>>>>>>> Cheers,
> > >>>>>>>>> Alex
> > >>>>>>>>> On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org>
> > >>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <
> > >>> fpj@apache.org
> > >>>>>
> > >>>>>>>> wrote:
> > >>>>>>>>>>> I gotta say that I'm not super excited about this option,
> > >> but
> > >>>> for
> > >>>>>> some
> > >>>>>>>>>> reason I ended up carrying the flag. To recap, I just raised
> > >>> this
> > >>>>>> option
> > >>>>>>>>>> because it seems that there are folks interested in features
> > >> in
> > >>>> 3.5
> > >>>>>> like
> > >>>>>>>>>> SSL and not necessarily in reconfiguration. SSL is important
> > >> and
> > >>>> to
> > >>>>>> take
> > >>>>>>>>>> Kafka as an example, it sucks that we can't have a whole set
> > >> up
> > >>>> using
> > >>>>>>>> SSL.
> > >>>>>>>>>> For ZK, the real questions are:
> > >>>>>>>>>>>
> > >>>>>>>>>>> 1- how fast can we make 3.5 stable?
> > >>>>>>>>>>> 2- would it be faster if we have a way of disabling
> > >>>>>> reconfiguration?
> > >>>>>>>>>>> 3- would enough users care about a stable 3.5 that has
> > >>>>>> reconfiguration
> > >>>>>>>>>> disabled?
> > >>>>>>>>>>>
> > >>>>>>>>>>> It is taking a long time to get 3.5 to beta. There has been
> > >>> some
> > >>>>>> good
> > >>>>>>>>>> activity around 3.5.2 release, which is a great step, but it
> > >> is
> > >>>>>> unclear
> > >>>>>>>>>> when 3.5.3 is going to come and if we will be able to make 3.5
> > >>>> beta
> > >>>>>>>> then.
> > >>>>>>>>>> Frankly, disabling reconfiguration sounds undesirable because
> > >> it
> > >>>> is
> > >>>>>> an
> > >>>>>>>>>> important feature, but I currently don't use it in production,
> > >>> so
> > >>>>>> from a
> > >>>>>>>>>> practical point of view, I can go both ways. Whether we go
> > >>> through
> > >>>>>> the
> > >>>>>>>>>> trouble of doing 2 depends on users interested in that option
> > >>> and
> > >>>>>> folks
> > >>>>>>>>>> willing to implement it.
> > >>>>>>>>>>>
> > >>>>>>>>>>> To answer your question, Powell, my pseudo-proposal is kind
> > >>> of a
> > >>>>>> funny
> > >>>>>>>>>> option because once the feature is stable, then we wouldn't
> > >>> need a
> > >>>>>>>> switch
> > >>>>>>>>>> any longer, so there is not need of a deprecation path, we
> > >> just
> > >>>> start
> > >>>>>>>>>> ignoring it from the first beta release. Until it is beta, I'd
> > >>> say
> > >>>>>> that
> > >>>>>>>>>> default is disabled.
> > >>>>>>>>>>
> > >>>>>>>>>> I would argue that we need this even when it does become
> > >> stable.
> > >>>> To
> > >>>>>> me
> > >>>>>>>>>> this is not a quality issue so much as it is an auth issue. We
> > >>>> want
> > >>>>>> to
> > >>>>>>>>>> make it simple for folks to run a vanilla/old ZK cluster and
> > >> not
> > >>>>>> worry
> > >>>>>>>>>> about the security implications of having reconfig enabled.
> > >>>>>>>>>>
> > >>>>>>>>>> Patrick
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> -Flavio
> > >>>>>>>>>>>
> > >>>>>>>>>>>> On 17 Mar 2016, at 17:44, powell molleti
> > >>>>>> <powellm79@yahoo.com.INVALID
> > >>>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Hi Flavio,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Generally do config options and command line args come
> > >> under
> > >>>> the
> > >>>>>> same
> > >>>>>>>>>> SLA as API?. I was assuming as such hence my question. Perhaps
> > >>> if
> > >>>> the
> > >>>>>>>>>> expectation is that this config option is temporary from get
> > >> go
> > >>>> then
> > >>>>>>>> may be
> > >>>>>>>>>> it is ok. The default for re-config support will be enabled or
> > >>>>>>>> disabled?.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I am just thinking from provisioning point of view when
> > >>> people
> > >>>>>>>> generate
> > >>>>>>>>>> config options etc.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thanks
> > >>>>>>>>>>>> Powell.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
> > >>>>>>>> fpj@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>> Hi Powell,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I was thinking config file and system property server side.
> > >>>> What's
> > >>>>>>>> your
> > >>>>>>>>>> concern with compatibility? The API itself wouldn't change,
> > >> but
> > >>>> the
> > >>>>>>>> config
> > >>>>>>>>>> option wouldn't exist in previous versions and would not exist
> > >>>>>> either in
> > >>>>>>>>>> later stable versions of 3.5.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> -Flavio
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> On 16 Mar 2016, at 22:08, powell molleti
> > >>>>>>>> <po...@yahoo.com.INVALID>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Will this option be supplied via config file/args/API?.
> > >> Will
> > >>>> this
> > >>>>>>>>>> option be a temporary thing i.e what about its compatibility?.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> thanks
> > >>>>>>>>>>>>> Powell.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
> > >>>>>>>> fpj@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>> The main issue to sort out is stability of the API. There
> > >>> is a
> > >>>>>>>>>> security concern around the fact that clients can freely
> > >>>> reconfigure
> > >>>>>> the
> > >>>>>>>>>> ensemble. If we follow the plan that Pat proposed some time
> > >> ago:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > >>>>>>>>>> <
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > >>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Locking the API is the main step to move it to beta.
> > >> Sorting
> > >>>> out
> > >>>>>>>> bugs
> > >>>>>>>>>> is definitely necessary, but it isn't the main thing that is
> > >>>> keeping
> > >>>>>>>> 3.5 in
> > >>>>>>>>>> alpha.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> About making it experimental, I was raising the option of
> > >>>> having
> > >>>>>> a
> > >>>>>>>>>> switch that disables the API calls, not the code. The reason
> > >> why
> > >>>> that
> > >>>>>>>> could
> > >>>>>>>>>> work is that anyone using 3.5 who uses the "experimental" API
> > >>> must
> > >>>>>>>> explicit
> > >>>>>>>>>> turn on the switch and enable the calls. If they do it, they
> > >>> need
> > >>>> to
> > >>>>>> be
> > >>>>>>>>>> aware that the API can change.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I must say that I haven't really looked closely into doing
> > >>> it,
> > >>>>>> and
> > >>>>>>>> I'm
> > >>>>>>>>>> not even entirely convinced that this is a good idea, but
> > >> since
> > >>>> Jason
> > >>>>>>>>>> raised the point, I'm exploring options.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> -Flavio
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <
> > >>>> shralex@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Looking at the list of ~50 blocker and critical bugs in
> > >>>>>> ZooKeeper,
> > >>>>>>>>>> only 3-4
> > >>>>>>>>>>>>>> are related to reconfig. Given this, and the fact that it
> > >>> is
> > >>>>>> run in
> > >>>>>>>>>>>>>> production since 2012 in multiple companies, I don't
> > >> think
> > >>>> its
> > >>>>>> more
> > >>>>>>>>>>>>>> unstable than any other part of ZooKeeper.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> There are multiple reconfig-related bugs that turned out
> > >>>> really
> > >>>>>>>>>> difficult
> > >>>>>>>>>>>>>> to debug without access to the actual system or at least
> > >> to
> > >>>> the
> > >>>>>>>> Hudson
> > >>>>>>>>>>>>>> machines where some tests are failing. In the past, Michi
> > >>>> and I
> > >>>>>>>>>> physically
> > >>>>>>>>>>>>>> went to Hortonworks to debug one such issue, but this is
> > >> of
> > >>>>>> course
> > >>>>>>>>>> not a
> > >>>>>>>>>>>>>> good method, and we weren't able to arrange such a visit
> > >>>> again.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Regarding making it optional - the reconfig logic has
> > >>> several
> > >>>>>>>>>> different
> > >>>>>>>>>>>>>> parts, some would be really difficult to disable using a
> > >>>>>>>> configuration
> > >>>>>>>>>>>>>> parameter. But the actual dynamic expansion of the
> > >> cluster
> > >>> is
> > >>>>>>>>>> triggered by
> > >>>>>>>>>>>>>> the reconfig command, so it should not affect users who
> > >>> don't
> > >>>>>>>> invoke
> > >>>>>>>>>> it.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
> > >>>>>>>> fpj@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I suppose we could give it a try. How do other folks
> > >> feel
> > >>>> about
> > >>>>>>>> it?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> -Flavio
> > >>>>>>>>>>>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <
> > >> jbr@squareup.com
> > >>>>
> > >>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> So, you could enable the dynamic reconfiguration
> > >> feature
> > >>>>>> behind a
> > >>>>>>>>>> config
> > >>>>>>>>>>>>>>>> option, and document that it should only be enabled
> > >>>>>>>> experimentally,
> > >>>>>>>>>> use
> > >>>>>>>>>>>>>>> at
> > >>>>>>>>>>>>>>>> your own risk.  Keep it off by default.  Allow only
> > >>> static
> > >>>>>>>> config by
> > >>>>>>>>>>>>>>>> default, until it's stable?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Jason
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
> > >>>>>>>> fpj@apache.org>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Hi Jason,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> The consumer in Kafka is pretty independent from the
> > >>> core
> > >>>>>>>>>> (brokers),
> > >>>>>>>>>>>>>>>>> that's how that project manages to make such a
> > >>>> separation. We
> > >>>>>>>> don't
> > >>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>> the same with ZooKeeper as the feature we are talking
> > >>>> about
> > >>>>>> is
> > >>>>>>>>>> part of
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> server and the only way I see of doing what you say is
> > >>> to
> > >>>>>> turn
> > >>>>>>>> off
> > >>>>>>>>>>>>>>>>> features. More specifically, we'd need to disable the
> > >>>>>> reconfig
> > >>>>>>>> API
> > >>>>>>>>>> and
> > >>>>>>>>>>>>>>> do
> > >>>>>>>>>>>>>>>>> not allow any change to the configuration, even though
> > >>> the
> > >>>>>> code
> > >>>>>>>> is
> > >>>>>>>>>>>>>>> there.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Reconfig here refers to the ability of changing the
> > >>>>>>>> configuration
> > >>>>>>>>>> of an
> > >>>>>>>>>>>>>>>>> ensemble (e.g., changing the set of servers).
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> -Flavio
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <
> > >>>> jbr@squareup.com
> > >>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> So, it would seem sensible to me to have a release
> > >>> where
> > >>>> all
> > >>>>>>>>>> features
> > >>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>> stable, except where noted.  E.g. mark certain
> > >> features
> > >>>> as
> > >>>>>> only
> > >>>>>>>>>>>>>>> 'alpha
> > >>>>>>>>>>>>>>>>>> quality', e.g. the 're-config feature'.  (I assume
> > >> it's
> > >>>>>>>> possible
> > >>>>>>>>>> to
> > >>>>>>>>>>>>>>>>> happily
> > >>>>>>>>>>>>>>>>>> use 3.5.X without exercising the unstable re-config
> > >>>> bits?).
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> There's precedent for doing this sort of thing in
> > >> other
> > >>>>>>>> projects,
> > >>>>>>>>>>>>>>> e.g.
> > >>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> Kafka, they've had several release where a new
> > >>> "Consumer
> > >>>>>> API"
> > >>>>>>>> is
> > >>>>>>>>>>>>>>>> shipped
> > >>>>>>>>>>>>>>>>>> that is available for beta-testing, but you can still
> > >>>> just
> > >>>>>> use
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>> older
> > >>>>>>>>>>>>>>>>>> stable consumer api, etc.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Jason
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > >>>>>>>>>>>>>>>>> <powellm79@yahoo.com.invalid
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Hi Doug,
> > >>>>>>>>>>>>>>>>>>> Is 3.5 being an alpha release preventing you from
> > >>> using
> > >>>>>> it?.
> > >>>>>>>> Or
> > >>>>>>>>>> have
> > >>>>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>>> run into issues with it?. In general perhaps ZK 3.5
> > >>>> being
> > >>>>>>>>>> labeled as
> > >>>>>>>>>>>>>>>>> alpha
> > >>>>>>>>>>>>>>>>>>> might not be fair, since it is far more stable then
> > >>> what
> > >>>>>> most
> > >>>>>>>>>> people
> > >>>>>>>>>>>>>>>>>>> associate an alpha release to be.
> > >>>>>>>>>>>>>>>>>>> Perhaps if you do not use re-config feature may be
> > >> it
> > >>>> will
> > >>>>>>>> just
> > >>>>>>>>>> work
> > >>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>> you?.
> > >>>>>>>>>>>>>>>>>>> There are many examples of 3.5.X being used in
> > >>>> productions
> > >>>>>>>> from
> > >>>>>>>>>> my
> > >>>>>>>>>>>>>>>>> limited
> > >>>>>>>>>>>>>>>>>>> knowledge.
> > >>>>>>>>>>>>>>>>>>> ThanksPowell.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio
> > >>> Junqueira <
> > >>>>>>>>>>>>>>>>> fpj@apache.org>
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> None of us expected the reconfig changes to take
> > >> this
> > >>>> long
> > >>>>>> to
> > >>>>>>>>>>>>>>>> stabilize.
> > >>>>>>>>>>>>>>>>>>> Until we get there, I don't think we can do anything
> > >>>> else
> > >>>>>> with
> > >>>>>>>>>> 3.5.
> > >>>>>>>>>>>>>>>> The
> > >>>>>>>>>>>>>>>>>>> best bet we have is to work harder to bring 3.5
> > >> into a
> > >>>>>> stable
> > >>>>>>>>>> rather
> > >>>>>>>>>>>>>>>>> than
> > >>>>>>>>>>>>>>>>>>> trying to work around it.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> There are lots of people interested in seeing 3.5
> > >>>> stable,
> > >>>>>> and
> > >>>>>>>> if
> > >>>>>>>>>> we
> > >>>>>>>>>>>>>>>> get
> > >>>>>>>>>>>>>>>>>>> everyone to contribute more patches and code
> > >> reviews,
> > >>> we
> > >>>>>>>> should
> > >>>>>>>>>> be
> > >>>>>>>>>>>>>>>> able
> > >>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> do it sooner. After all, it is a community based
> > >>>> effort, so
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>>> community
> > >>>>>>>>>>>>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> -Flavio
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> > >>>>>>>>>> cnauroth@hortonworks.com>
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Doug, I forgot to respond to your question about
> > >> 3.4.
> > >>>>>> Since
> > >>>>>>>>>> 3.4 is
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> stable maintenance line, we are very conservative
> > >>> about
> > >>>>>>>>>>>>>>> back-porting
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>> it.  Our policy is to limit back-ports to critical
> > >>> bug
> > >>>>>> fixes
> > >>>>>>>> and
> > >>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>> introduce any new features in the 3.4 line.  This
> > >> is
> > >>> a
> > >>>>>>>> matter of
> > >>>>>>>>>>>>>>>>> managing
> > >>>>>>>>>>>>>>>>>>>> risk.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Jason, your question about release cadence is a
> > >> fair
> > >>>>>> one.  If
> > >>>>>>>>>> it's
> > >>>>>>>>>>>>>>>> any
> > >>>>>>>>>>>>>>>>>>>> consolation, we are now taking the approach of
> > >> trying
> > >>>> to
> > >>>>>>>> limit
> > >>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> scope
> > >>>>>>>>>>>>>>>>>>>> of anything new introduced in 3.5 too.  That would
> > >>>> allow
> > >>>>>> us
> > >>>>>>>> to
> > >>>>>>>>>>>>>>> focus
> > >>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>> stabilization: resolving blocker bugs and freezing
> > >>>> public
> > >>>>>>>>>> APIs.  I
> > >>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>> this will help us accelerate the releases into beta
> > >>> and
> > >>>>>>>> eventual
> > >>>>>>>>>>>>>>> GA.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> I am new to ZooKeeper release management, so I'd
> > >> like
> > >>>> to
> > >>>>>> hear
> > >>>>>>>>>>>>>>>> thoughts
> > >>>>>>>>>>>>>>>>>>>> from more experienced committers and PMC members
> > >>> about
> > >>>>>> your
> > >>>>>>>>>>>>>>> proposal
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>> try to cut a stable release for a limited subset of
> > >>>> what
> > >>>>>> is
> > >>>>>>>> in
> > >>>>>>>>>>>>>>>>> branch-3.5
> > >>>>>>>>>>>>>>>>>>>> now.  My instinct is that it would be challenging
> > >> to
> > >>>>>>>> cherry-pick
> > >>>>>>>>>>>>>>> out
> > >>>>>>>>>>>>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This
> > >>>> would
> > >>>>>>>> become
> > >>>>>>>>>>>>>>>>> another
> > >>>>>>>>>>>>>>>>>>>> release line for an already resource-constrained
> > >>>> volunteer
> > >>>>>>>>>> staff to
> > >>>>>>>>>>>>>>>>>>>> manage.  I'd prefer to dedicate those limited
> > >>>> resources to
> > >>>>>>>>>> overall
> > >>>>>>>>>>>>>>>> 3.5
> > >>>>>>>>>>>>>>>>>>>> stabilization.  Also, a 3.5 release in which
> > >> certain
> > >>>>>> features
> > >>>>>>>>>>>>>>>>> "vanished"
> > >>>>>>>>>>>>>>>>>>>> because of not meeting some stability criteria
> > >> would
> > >>> be
> > >>>>>>>>>>>>>>> undesirable.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> --Chris Nauroth
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <
> > >>>> jbr@squareup.com
> > >>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Chris,
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Can you say whether some parts of 3.5.X are more
> > >>>> stable
> > >>>>>> than
> > >>>>>>>>>>>>>>> others
> > >>>>>>>>>>>>>>>>>>> (e.g.
> > >>>>>>>>>>>>>>>>>>>>> if we don't care about certain new features, is it
> > >>>>>>>> relatively
> > >>>>>>>>>>>>>>>> stable)?
> > >>>>>>>>>>>>>>>>>>>>> Would it be possible to cut out a version that
> > >> only
> > >>>> has
> > >>>>>> the
> > >>>>>>>>>> bits
> > >>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>> are stable (and release that)?
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> From that timeline, and the historic release
> > >>> cadence,
> > >>>> it
> > >>>>>>>> would
> > >>>>>>>>>>>>>>> seem
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>> years away before we get to the stable release?
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Jason
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> > >>>>>>>>>>>>>>>>>>> cnauroth@hortonworks.com>
> > >>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Hello Doug,
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Thanks for your interest in the SSL feature!
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> At this point, I think we're still pretty far
> > >> away
> > >>>> from
> > >>>>>>>>>>>>>>> declaring a
> > >>>>>>>>>>>>>>>>>>>>>> stable
> > >>>>>>>>>>>>>>>>>>>>>> release in the 3.5 line.  I don't think we're
> > >> close
> > >>>>>> enough
> > >>>>>>>>>> that
> > >>>>>>>>>>>>>>>>> anyone
> > >>>>>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread
> > >>> that
> > >>>>>>>>>> describes
> > >>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> high-level strategy for release planning in the
> > >> 3.5
> > >>>>>> line:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> https://s.apache.org/ADK1
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're
> > >>>> working
> > >>>>>> on
> > >>>>>>>>>>>>>>>> resolving a
> > >>>>>>>>>>>>>>>>>>>>>> few
> > >>>>>>>>>>>>>>>>>>>>>> more blockers before we produce a release
> > >>> candidate.
> > >>>>>>>>>> Hopefully
> > >>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>> get done in the next few weeks.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> --Chris Nauroth
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <itsbehind@gmail.com
> > >>>
> > >>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> I know it's only been a few months, but I was
> > >>>>>> wondering if
> > >>>>>>>>>> there
> > >>>>>>>>>>>>>>>>> was a
> > >>>>>>>>>>>>>>>>>>>>>>> ballpark release date for a stable version of
> > >>> 3.5.1.
> > >>>>>> Or is
> > >>>>>>>>>> there
> > >>>>>>>>>>>>>>>> any
> > >>>>>>>>>>>>>>>>>>>>>>> chance
> > >>>>>>>>>>>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just
> > >>>> another
> > >>>>>>>> person
> > >>>>>>>>>>>>>>>> looking
> > >>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>> that feature in a stable version. Thanks for all
> > >>> you
> > >>>>>> do!
> > >>>>>>>> :)
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>> View this message in context:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > >>>>>>>>>>>>>>>>>>>>>> e
> > >>>>>>>>>>>>>>>>>>>>>>> -tp7581744p7582136.html
> > >>>>>>>>>>>>>>>>>>>>>>> Sent from the zookeeper-user mailing list
> > >> archive
> > >>> at
> > >>>>>>>>>> Nabble.com.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

Citing Patrick:

> If you're running zk w/o security turned on and suddenly folks can do
reconfig
> operations it's going to potentially be a problem.
...
> Rather than force people to turn on kerberos (etc...) we could instead
> have the feature off

>From this I understood that the concern is mostly about users that DON'T
use ACLs. My proposal is to disable
reconfig/getconfig for all such users, forcing users who want reconfig to
also use ACLs. Users who do use ACLs
don't have to use reconfig and will have to set the ACLs on the config
znode before they can use it.

In preprequestprocessor where acls are checked for reconfig operation we
can check that:

skipACL = false && nodeRecord.acl != null && nodeRecord.acl.size() != 0

meaning you're using ACLs, and have actually set ACLs on the config node.

For getConfig its a bit trickier since its just a getData on the server
side (for efficiency
of reads, we avoided checking whether path == config znode). What we could
do is before sending
the operation to the server check skipACL = false and maybe also issue a
getACL call to check that
nodeRecord.acl != null && nodeRecord.acl.size() != 0
and only then issue a getData. This part is not air tight but its probably
sufficient.

And of course we can emphasize the need for ACLs on this znode in the
release.


On Thu, Mar 31, 2016 at 1:11 PM, Flavio Junqueira <fp...@apache.org> wrote:

> I think Jason is saying that this is orthogonal in the following sense.
> You set ACLs because you care about authentication/authorization in your
> cluster, but you may not want reconfig enabled, it just happened that you
> wanted to use ACLs.
>
> Perhaps you can elaborate a bit on how you think we can perform this ACL
> check? What would you check precisely?
>
> -Flavio
>
> > On 24 Mar 2016, at 21:19, Alexander Shraer <sh...@gmail.com> wrote:
> >
> > I'm not so sure its orthogonal. The question is whether someone would
> ever
> > want to use reconfig without ACLs,
> > as this allows any client to reconfigure the servers away or add a bunch
> of
> > servers that shouldn't be there :) and whether we should facilitate this
> > knowing its insecure.
> >
> > Requiring ACLs solves the security concern for both reconfig and
> getconfig.
> > For example, if you don't want your clients to know the list of servers,
> > limit their read permissions on the configuration znode.
> >
> > On Thu, Mar 24, 2016 at 2:11 PM, Jason Rosenberg <jb...@squareup.com>
> wrote:
> >
> >> seems like an orthogonal requirement?
> >>
> >> On Thu, Mar 24, 2016 at 3:37 PM, Alexander Shraer <sh...@gmail.com>
> >> wrote:
> >>
> >>> How about a simpler alternative to the proposed flag for reconfig: a
> >> check
> >>> in the code that requires ACLs to be set.
> >>> If people want to use reconfig, they should use ACLs too.
> >>>
> >>> What do you think ?
> >>>
> >>> Alex
> >>>
> >>> On Mon, Mar 21, 2016 at 9:58 PM, Patrick Hunt <ph...@apache.org>
> wrote:
> >>>
> >>>> I would say if in doubt add a safety. (a config parameter to turn it
> >>>> off). Cost is almost zero and worst case it will just give us peace of
> >>>> mind. ;-)
> >>>>
> >>>> Patrick
> >>>>
> >>>> On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <sh...@gmail.com>
> >>>> wrote:
> >>>>> ok, thanks for the suggestion, I'll look into it. For reconfig I
> >> think
> >>>> its
> >>>>> pretty clear that its an admin
> >>>>> functionality. I just always imagined that its controlled via acls,
> >>> but I
> >>>>> understand
> >>>>> the concerns now.
> >>>>>
> >>>>> getConfig returns the dynamic config (list of all servers with all
> >>> ports
> >>>>> and quorum system if defined)
> >>>>> and has an option to filter that info and just return the server
> >>>> connection
> >>>>> string (server and client port only).
> >>>>>
> >>>>>
> >>>>> On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org>
> >>> wrote:
> >>>>>
> >>>>>> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <
> >> shralex@gmail.com>
> >>>>>> wrote:
> >>>>>>> I don't think that getConfig should be an admin functionality. It
> >> is
> >>>>>>> essential for client-side re-balancing
> >>>>>>> that we implemented (all clients shoudl be able to detect
> >>>> configuration
> >>>>>>> changes via getConfig). It could
> >>>>>>> be hidden somewhat by defining higher-level re-balancing
> >>>>>>> policies (ZOOKEEPER-2016)
> >>>>>>> but there hasn't been enough progress on that. Perhaps instead
> >>>> getConfig
> >>>>>>> should be controlled
> >>>>>>> by a separate flag ?
> >>>>>>>
> >>>>>>
> >>>>>> I believe that the Hadoop community has something we could use:
> >>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
> >>>>>> (whether through annotations or just documenting it in the API
> >>> javadoc)
> >>>>>>
> >>>>>> e.g. we could list getConfig as public/unstable for example and
> >> still
> >>>>>> ship it as GA. That would mark it as something that could change re
> >>>>>> API policy.
> >>>>>>
> >>>>>> Is the entire config exposed through getConfig? If so then we might
> >>>>>> want to enable/disable it with a flag similar to reconfig. Might be
> >>>>>> safer to just do that if we're not sure.
> >>>>>>
> >>>>>>
> >>>>>> Re classification - we could do the same thing with reconfig, but I
> >>>>>> think that would be a mistake. If we feel strongly where it should
> >>>>>> live long term we should just move it now.
> >>>>>>
> >>>>>> Patrick
> >>>>>>
> >>>>>>>
> >>>>>>> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <
> >>> shralex@gmail.com
> >>>>>
> >>>>>>>> wrote:
> >>>>>>>>> Hi Patrick, Flavio,
> >>>>>>>>>
> >>>>>>>>> Since there seems to be consensus on this, I can add this flag,
> >>>> unless
> >>>>>>>>> someone else wants to. I assume that getConfig should still
> >> work
> >>>>>>>> regardless
> >>>>>>>>> of the flag ? is there a security concern with clients knowing
> >>> the
> >>>>>> list
> >>>>>>>> of
> >>>>>>>>> servers?
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> We've always hidden that detail from users. We don't even expose
> >>>> which
> >>>>>>>> server you're connected to today. I remember Ben (and perhaps
> >>>> Flavio?)
> >>>>>>>> highlighting this as important to maintain although I'm not super
> >>>>>>>> familiar with the specifics on why. It made sense to me though
> >> from
> >>>>>>>> the perspective that we don't want clients to make assumptions
> >> that
> >>>>>>>> probably shouldn't.
> >>>>>>>>
> >>>>>>>> My thinking is that we should 1) add a config option to enable
> >>>>>>>> reconfig (off by default), 2) move reconfig specific
> >> functionality
> >>>>>>>> from ZooKeeper.java (including getconfig) into an "admin"
> >> package,
> >>>>>>>> within say a class ZooKeeperAdmin, 3) document/test use of ACLs
> >> for
> >>>>>>>> when folks do want to enable reconfig and are also worried about
> >>>> auth.
> >>>>>>>> (e.g. turn on kerb)
> >>>>>>>>
> >>>>>>>> Again, I don't see any of this as a quality issue personally. As
> >>> such
> >>>>>>>> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if
> >>> we
> >>>>>>>> were interested in doing such a release. Adjusting the API should
> >>> be
> >>>>>>>> done before we move to "beta" though. Although that seems like a
> >>>>>>>> pretty mechanical (eclipse/idea) type refactoring?
> >>>>>>>>
> >>>>>>>> Patrick
> >>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Alex
> >>>>>>>>> On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org>
> >>> wrote:
> >>>>>>>>>
> >>>>>>>>>> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <
> >>> fpj@apache.org
> >>>>>
> >>>>>>>> wrote:
> >>>>>>>>>>> I gotta say that I'm not super excited about this option,
> >> but
> >>>> for
> >>>>>> some
> >>>>>>>>>> reason I ended up carrying the flag. To recap, I just raised
> >>> this
> >>>>>> option
> >>>>>>>>>> because it seems that there are folks interested in features
> >> in
> >>>> 3.5
> >>>>>> like
> >>>>>>>>>> SSL and not necessarily in reconfiguration. SSL is important
> >> and
> >>>> to
> >>>>>> take
> >>>>>>>>>> Kafka as an example, it sucks that we can't have a whole set
> >> up
> >>>> using
> >>>>>>>> SSL.
> >>>>>>>>>> For ZK, the real questions are:
> >>>>>>>>>>>
> >>>>>>>>>>> 1- how fast can we make 3.5 stable?
> >>>>>>>>>>> 2- would it be faster if we have a way of disabling
> >>>>>> reconfiguration?
> >>>>>>>>>>> 3- would enough users care about a stable 3.5 that has
> >>>>>> reconfiguration
> >>>>>>>>>> disabled?
> >>>>>>>>>>>
> >>>>>>>>>>> It is taking a long time to get 3.5 to beta. There has been
> >>> some
> >>>>>> good
> >>>>>>>>>> activity around 3.5.2 release, which is a great step, but it
> >> is
> >>>>>> unclear
> >>>>>>>>>> when 3.5.3 is going to come and if we will be able to make 3.5
> >>>> beta
> >>>>>>>> then.
> >>>>>>>>>> Frankly, disabling reconfiguration sounds undesirable because
> >> it
> >>>> is
> >>>>>> an
> >>>>>>>>>> important feature, but I currently don't use it in production,
> >>> so
> >>>>>> from a
> >>>>>>>>>> practical point of view, I can go both ways. Whether we go
> >>> through
> >>>>>> the
> >>>>>>>>>> trouble of doing 2 depends on users interested in that option
> >>> and
> >>>>>> folks
> >>>>>>>>>> willing to implement it.
> >>>>>>>>>>>
> >>>>>>>>>>> To answer your question, Powell, my pseudo-proposal is kind
> >>> of a
> >>>>>> funny
> >>>>>>>>>> option because once the feature is stable, then we wouldn't
> >>> need a
> >>>>>>>> switch
> >>>>>>>>>> any longer, so there is not need of a deprecation path, we
> >> just
> >>>> start
> >>>>>>>>>> ignoring it from the first beta release. Until it is beta, I'd
> >>> say
> >>>>>> that
> >>>>>>>>>> default is disabled.
> >>>>>>>>>>
> >>>>>>>>>> I would argue that we need this even when it does become
> >> stable.
> >>>> To
> >>>>>> me
> >>>>>>>>>> this is not a quality issue so much as it is an auth issue. We
> >>>> want
> >>>>>> to
> >>>>>>>>>> make it simple for folks to run a vanilla/old ZK cluster and
> >> not
> >>>>>> worry
> >>>>>>>>>> about the security implications of having reconfig enabled.
> >>>>>>>>>>
> >>>>>>>>>> Patrick
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> -Flavio
> >>>>>>>>>>>
> >>>>>>>>>>>> On 17 Mar 2016, at 17:44, powell molleti
> >>>>>> <powellm79@yahoo.com.INVALID
> >>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi Flavio,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Generally do config options and command line args come
> >> under
> >>>> the
> >>>>>> same
> >>>>>>>>>> SLA as API?. I was assuming as such hence my question. Perhaps
> >>> if
> >>>> the
> >>>>>>>>>> expectation is that this config option is temporary from get
> >> go
> >>>> then
> >>>>>>>> may be
> >>>>>>>>>> it is ok. The default for re-config support will be enabled or
> >>>>>>>> disabled?.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I am just thinking from provisioning point of view when
> >>> people
> >>>>>>>> generate
> >>>>>>>>>> config options etc.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks
> >>>>>>>>>>>> Powell.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
> >>>>>>>> fpj@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>> Hi Powell,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I was thinking config file and system property server side.
> >>>> What's
> >>>>>>>> your
> >>>>>>>>>> concern with compatibility? The API itself wouldn't change,
> >> but
> >>>> the
> >>>>>>>> config
> >>>>>>>>>> option wouldn't exist in previous versions and would not exist
> >>>>>> either in
> >>>>>>>>>> later stable versions of 3.5.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Flavio
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On 16 Mar 2016, at 22:08, powell molleti
> >>>>>>>> <po...@yahoo.com.INVALID>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Will this option be supplied via config file/args/API?.
> >> Will
> >>>> this
> >>>>>>>>>> option be a temporary thing i.e what about its compatibility?.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> thanks
> >>>>>>>>>>>>> Powell.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
> >>>>>>>> fpj@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>> The main issue to sort out is stability of the API. There
> >>> is a
> >>>>>>>>>> security concern around the fact that clients can freely
> >>>> reconfigure
> >>>>>> the
> >>>>>>>>>> ensemble. If we follow the plan that Pat proposed some time
> >> ago:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >>>>>>>>>> <
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Locking the API is the main step to move it to beta.
> >> Sorting
> >>>> out
> >>>>>>>> bugs
> >>>>>>>>>> is definitely necessary, but it isn't the main thing that is
> >>>> keeping
> >>>>>>>> 3.5 in
> >>>>>>>>>> alpha.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> About making it experimental, I was raising the option of
> >>>> having
> >>>>>> a
> >>>>>>>>>> switch that disables the API calls, not the code. The reason
> >> why
> >>>> that
> >>>>>>>> could
> >>>>>>>>>> work is that anyone using 3.5 who uses the "experimental" API
> >>> must
> >>>>>>>> explicit
> >>>>>>>>>> turn on the switch and enable the calls. If they do it, they
> >>> need
> >>>> to
> >>>>>> be
> >>>>>>>>>> aware that the API can change.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I must say that I haven't really looked closely into doing
> >>> it,
> >>>>>> and
> >>>>>>>> I'm
> >>>>>>>>>> not even entirely convinced that this is a good idea, but
> >> since
> >>>> Jason
> >>>>>>>>>> raised the point, I'm exploring options.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -Flavio
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <
> >>>> shralex@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Looking at the list of ~50 blocker and critical bugs in
> >>>>>> ZooKeeper,
> >>>>>>>>>> only 3-4
> >>>>>>>>>>>>>> are related to reconfig. Given this, and the fact that it
> >>> is
> >>>>>> run in
> >>>>>>>>>>>>>> production since 2012 in multiple companies, I don't
> >> think
> >>>> its
> >>>>>> more
> >>>>>>>>>>>>>> unstable than any other part of ZooKeeper.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There are multiple reconfig-related bugs that turned out
> >>>> really
> >>>>>>>>>> difficult
> >>>>>>>>>>>>>> to debug without access to the actual system or at least
> >> to
> >>>> the
> >>>>>>>> Hudson
> >>>>>>>>>>>>>> machines where some tests are failing. In the past, Michi
> >>>> and I
> >>>>>>>>>> physically
> >>>>>>>>>>>>>> went to Hortonworks to debug one such issue, but this is
> >> of
> >>>>>> course
> >>>>>>>>>> not a
> >>>>>>>>>>>>>> good method, and we weren't able to arrange such a visit
> >>>> again.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Regarding making it optional - the reconfig logic has
> >>> several
> >>>>>>>>>> different
> >>>>>>>>>>>>>> parts, some would be really difficult to disable using a
> >>>>>>>> configuration
> >>>>>>>>>>>>>> parameter. But the actual dynamic expansion of the
> >> cluster
> >>> is
> >>>>>>>>>> triggered by
> >>>>>>>>>>>>>> the reconfig command, so it should not affect users who
> >>> don't
> >>>>>>>> invoke
> >>>>>>>>>> it.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
> >>>>>>>> fpj@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I suppose we could give it a try. How do other folks
> >> feel
> >>>> about
> >>>>>>>> it?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -Flavio
> >>>>>>>>>>>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <
> >> jbr@squareup.com
> >>>>
> >>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> So, you could enable the dynamic reconfiguration
> >> feature
> >>>>>> behind a
> >>>>>>>>>> config
> >>>>>>>>>>>>>>>> option, and document that it should only be enabled
> >>>>>>>> experimentally,
> >>>>>>>>>> use
> >>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>> your own risk.  Keep it off by default.  Allow only
> >>> static
> >>>>>>>> config by
> >>>>>>>>>>>>>>>> default, until it's stable?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Jason
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
> >>>>>>>> fpj@apache.org>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi Jason,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The consumer in Kafka is pretty independent from the
> >>> core
> >>>>>>>>>> (brokers),
> >>>>>>>>>>>>>>>>> that's how that project manages to make such a
> >>>> separation. We
> >>>>>>>> don't
> >>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> the same with ZooKeeper as the feature we are talking
> >>>> about
> >>>>>> is
> >>>>>>>>>> part of
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> server and the only way I see of doing what you say is
> >>> to
> >>>>>> turn
> >>>>>>>> off
> >>>>>>>>>>>>>>>>> features. More specifically, we'd need to disable the
> >>>>>> reconfig
> >>>>>>>> API
> >>>>>>>>>> and
> >>>>>>>>>>>>>>> do
> >>>>>>>>>>>>>>>>> not allow any change to the configuration, even though
> >>> the
> >>>>>> code
> >>>>>>>> is
> >>>>>>>>>>>>>>> there.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Reconfig here refers to the ability of changing the
> >>>>>>>> configuration
> >>>>>>>>>> of an
> >>>>>>>>>>>>>>>>> ensemble (e.g., changing the set of servers).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> -Flavio
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <
> >>>> jbr@squareup.com
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> So, it would seem sensible to me to have a release
> >>> where
> >>>> all
> >>>>>>>>>> features
> >>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>> stable, except where noted.  E.g. mark certain
> >> features
> >>>> as
> >>>>>> only
> >>>>>>>>>>>>>>> 'alpha
> >>>>>>>>>>>>>>>>>> quality', e.g. the 're-config feature'.  (I assume
> >> it's
> >>>>>>>> possible
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>> happily
> >>>>>>>>>>>>>>>>>> use 3.5.X without exercising the unstable re-config
> >>>> bits?).
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> There's precedent for doing this sort of thing in
> >> other
> >>>>>>>> projects,
> >>>>>>>>>>>>>>> e.g.
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> Kafka, they've had several release where a new
> >>> "Consumer
> >>>>>> API"
> >>>>>>>> is
> >>>>>>>>>>>>>>>> shipped
> >>>>>>>>>>>>>>>>>> that is available for beta-testing, but you can still
> >>>> just
> >>>>>> use
> >>>>>>>> the
> >>>>>>>>>>>>>>>> older
> >>>>>>>>>>>>>>>>>> stable consumer api, etc.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Jason
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> >>>>>>>>>>>>>>>>> <powellm79@yahoo.com.invalid
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi Doug,
> >>>>>>>>>>>>>>>>>>> Is 3.5 being an alpha release preventing you from
> >>> using
> >>>>>> it?.
> >>>>>>>> Or
> >>>>>>>>>> have
> >>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>> run into issues with it?. In general perhaps ZK 3.5
> >>>> being
> >>>>>>>>>> labeled as
> >>>>>>>>>>>>>>>>> alpha
> >>>>>>>>>>>>>>>>>>> might not be fair, since it is far more stable then
> >>> what
> >>>>>> most
> >>>>>>>>>> people
> >>>>>>>>>>>>>>>>>>> associate an alpha release to be.
> >>>>>>>>>>>>>>>>>>> Perhaps if you do not use re-config feature may be
> >> it
> >>>> will
> >>>>>>>> just
> >>>>>>>>>> work
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> you?.
> >>>>>>>>>>>>>>>>>>> There are many examples of 3.5.X being used in
> >>>> productions
> >>>>>>>> from
> >>>>>>>>>> my
> >>>>>>>>>>>>>>>>> limited
> >>>>>>>>>>>>>>>>>>> knowledge.
> >>>>>>>>>>>>>>>>>>> ThanksPowell.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio
> >>> Junqueira <
> >>>>>>>>>>>>>>>>> fpj@apache.org>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> None of us expected the reconfig changes to take
> >> this
> >>>> long
> >>>>>> to
> >>>>>>>>>>>>>>>> stabilize.
> >>>>>>>>>>>>>>>>>>> Until we get there, I don't think we can do anything
> >>>> else
> >>>>>> with
> >>>>>>>>>> 3.5.
> >>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>> best bet we have is to work harder to bring 3.5
> >> into a
> >>>>>> stable
> >>>>>>>>>> rather
> >>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>> trying to work around it.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> There are lots of people interested in seeing 3.5
> >>>> stable,
> >>>>>> and
> >>>>>>>> if
> >>>>>>>>>> we
> >>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>> everyone to contribute more patches and code
> >> reviews,
> >>> we
> >>>>>>>> should
> >>>>>>>>>> be
> >>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> do it sooner. After all, it is a community based
> >>>> effort, so
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> community
> >>>>>>>>>>>>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> -Flavio
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> >>>>>>>>>> cnauroth@hortonworks.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Doug, I forgot to respond to your question about
> >> 3.4.
> >>>>>> Since
> >>>>>>>>>> 3.4 is
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> stable maintenance line, we are very conservative
> >>> about
> >>>>>>>>>>>>>>> back-porting
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> it.  Our policy is to limit back-ports to critical
> >>> bug
> >>>>>> fixes
> >>>>>>>> and
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>> introduce any new features in the 3.4 line.  This
> >> is
> >>> a
> >>>>>>>> matter of
> >>>>>>>>>>>>>>>>> managing
> >>>>>>>>>>>>>>>>>>>> risk.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Jason, your question about release cadence is a
> >> fair
> >>>>>> one.  If
> >>>>>>>>>> it's
> >>>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>>>> consolation, we are now taking the approach of
> >> trying
> >>>> to
> >>>>>>>> limit
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>> scope
> >>>>>>>>>>>>>>>>>>>> of anything new introduced in 3.5 too.  That would
> >>>> allow
> >>>>>> us
> >>>>>>>> to
> >>>>>>>>>>>>>>> focus
> >>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>> stabilization: resolving blocker bugs and freezing
> >>>> public
> >>>>>>>>>> APIs.  I
> >>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>> this will help us accelerate the releases into beta
> >>> and
> >>>>>>>> eventual
> >>>>>>>>>>>>>>> GA.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I am new to ZooKeeper release management, so I'd
> >> like
> >>>> to
> >>>>>> hear
> >>>>>>>>>>>>>>>> thoughts
> >>>>>>>>>>>>>>>>>>>> from more experienced committers and PMC members
> >>> about
> >>>>>> your
> >>>>>>>>>>>>>>> proposal
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> try to cut a stable release for a limited subset of
> >>>> what
> >>>>>> is
> >>>>>>>> in
> >>>>>>>>>>>>>>>>> branch-3.5
> >>>>>>>>>>>>>>>>>>>> now.  My instinct is that it would be challenging
> >> to
> >>>>>>>> cherry-pick
> >>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This
> >>>> would
> >>>>>>>> become
> >>>>>>>>>>>>>>>>> another
> >>>>>>>>>>>>>>>>>>>> release line for an already resource-constrained
> >>>> volunteer
> >>>>>>>>>> staff to
> >>>>>>>>>>>>>>>>>>>> manage.  I'd prefer to dedicate those limited
> >>>> resources to
> >>>>>>>>>> overall
> >>>>>>>>>>>>>>>> 3.5
> >>>>>>>>>>>>>>>>>>>> stabilization.  Also, a 3.5 release in which
> >> certain
> >>>>>> features
> >>>>>>>>>>>>>>>>> "vanished"
> >>>>>>>>>>>>>>>>>>>> because of not meeting some stability criteria
> >> would
> >>> be
> >>>>>>>>>>>>>>> undesirable.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --Chris Nauroth
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <
> >>>> jbr@squareup.com
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Chris,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Can you say whether some parts of 3.5.X are more
> >>>> stable
> >>>>>> than
> >>>>>>>>>>>>>>> others
> >>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>> if we don't care about certain new features, is it
> >>>>>>>> relatively
> >>>>>>>>>>>>>>>> stable)?
> >>>>>>>>>>>>>>>>>>>>> Would it be possible to cut out a version that
> >> only
> >>>> has
> >>>>>> the
> >>>>>>>>>> bits
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>> are stable (and release that)?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> From that timeline, and the historic release
> >>> cadence,
> >>>> it
> >>>>>>>> would
> >>>>>>>>>>>>>>> seem
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>> years away before we get to the stable release?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Jason
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> >>>>>>>>>>>>>>>>>>> cnauroth@hortonworks.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hello Doug,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks for your interest in the SSL feature!
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> At this point, I think we're still pretty far
> >> away
> >>>> from
> >>>>>>>>>>>>>>> declaring a
> >>>>>>>>>>>>>>>>>>>>>> stable
> >>>>>>>>>>>>>>>>>>>>>> release in the 3.5 line.  I don't think we're
> >> close
> >>>>>> enough
> >>>>>>>>>> that
> >>>>>>>>>>>>>>>>> anyone
> >>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread
> >>> that
> >>>>>>>>>> describes
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> high-level strategy for release planning in the
> >> 3.5
> >>>>>> line:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> https://s.apache.org/ADK1
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're
> >>>> working
> >>>>>> on
> >>>>>>>>>>>>>>>> resolving a
> >>>>>>>>>>>>>>>>>>>>>> few
> >>>>>>>>>>>>>>>>>>>>>> more blockers before we produce a release
> >>> candidate.
> >>>>>>>>>> Hopefully
> >>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>> get done in the next few weeks.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --Chris Nauroth
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <itsbehind@gmail.com
> >>>
> >>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I know it's only been a few months, but I was
> >>>>>> wondering if
> >>>>>>>>>> there
> >>>>>>>>>>>>>>>>> was a
> >>>>>>>>>>>>>>>>>>>>>>> ballpark release date for a stable version of
> >>> 3.5.1.
> >>>>>> Or is
> >>>>>>>>>> there
> >>>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>>>>>>> chance
> >>>>>>>>>>>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just
> >>>> another
> >>>>>>>> person
> >>>>>>>>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>> that feature in a stable version. Thanks for all
> >>> you
> >>>>>> do!
> >>>>>>>> :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> View this message in context:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >>>>>>>>>>>>>>>>>>>>>> e
> >>>>>>>>>>>>>>>>>>>>>>> -tp7581744p7582136.html
> >>>>>>>>>>>>>>>>>>>>>>> Sent from the zookeeper-user mailing list
> >> archive
> >>> at
> >>>>>>>>>> Nabble.com.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
>
>

Re: Zookeeper with SSL release date

[Flavio Junqueira <fp...@apache.org>]

I think Jason is saying that this is orthogonal in the following sense. You set ACLs because you care about authentication/authorization in your cluster, but you may not want reconfig enabled, it just happened that you wanted to use ACLs.

Perhaps you can elaborate a bit on how you think we can perform this ACL check? What would you check precisely?

-Flavio

> On 24 Mar 2016, at 21:19, Alexander Shraer <sh...@gmail.com> wrote:
> 
> I'm not so sure its orthogonal. The question is whether someone would ever
> want to use reconfig without ACLs,
> as this allows any client to reconfigure the servers away or add a bunch of
> servers that shouldn't be there :) and whether we should facilitate this
> knowing its insecure.
> 
> Requiring ACLs solves the security concern for both reconfig and getconfig.
> For example, if you don't want your clients to know the list of servers,
> limit their read permissions on the configuration znode.
> 
> On Thu, Mar 24, 2016 at 2:11 PM, Jason Rosenberg <jb...@squareup.com> wrote:
> 
>> seems like an orthogonal requirement?
>> 
>> On Thu, Mar 24, 2016 at 3:37 PM, Alexander Shraer <sh...@gmail.com>
>> wrote:
>> 
>>> How about a simpler alternative to the proposed flag for reconfig: a
>> check
>>> in the code that requires ACLs to be set.
>>> If people want to use reconfig, they should use ACLs too.
>>> 
>>> What do you think ?
>>> 
>>> Alex
>>> 
>>> On Mon, Mar 21, 2016 at 9:58 PM, Patrick Hunt <ph...@apache.org> wrote:
>>> 
>>>> I would say if in doubt add a safety. (a config parameter to turn it
>>>> off). Cost is almost zero and worst case it will just give us peace of
>>>> mind. ;-)
>>>> 
>>>> Patrick
>>>> 
>>>> On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <sh...@gmail.com>
>>>> wrote:
>>>>> ok, thanks for the suggestion, I'll look into it. For reconfig I
>> think
>>>> its
>>>>> pretty clear that its an admin
>>>>> functionality. I just always imagined that its controlled via acls,
>>> but I
>>>>> understand
>>>>> the concerns now.
>>>>> 
>>>>> getConfig returns the dynamic config (list of all servers with all
>>> ports
>>>>> and quorum system if defined)
>>>>> and has an option to filter that info and just return the server
>>>> connection
>>>>> string (server and client port only).
>>>>> 
>>>>> 
>>>>> On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org>
>>> wrote:
>>>>> 
>>>>>> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <
>> shralex@gmail.com>
>>>>>> wrote:
>>>>>>> I don't think that getConfig should be an admin functionality. It
>> is
>>>>>>> essential for client-side re-balancing
>>>>>>> that we implemented (all clients shoudl be able to detect
>>>> configuration
>>>>>>> changes via getConfig). It could
>>>>>>> be hidden somewhat by defining higher-level re-balancing
>>>>>>> policies (ZOOKEEPER-2016)
>>>>>>> but there hasn't been enough progress on that. Perhaps instead
>>>> getConfig
>>>>>>> should be controlled
>>>>>>> by a separate flag ?
>>>>>>> 
>>>>>> 
>>>>>> I believe that the Hadoop community has something we could use:
>>>>>> 
>>>>>> 
>>>> 
>>> 
>> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
>>>>>> (whether through annotations or just documenting it in the API
>>> javadoc)
>>>>>> 
>>>>>> e.g. we could list getConfig as public/unstable for example and
>> still
>>>>>> ship it as GA. That would mark it as something that could change re
>>>>>> API policy.
>>>>>> 
>>>>>> Is the entire config exposed through getConfig? If so then we might
>>>>>> want to enable/disable it with a flag similar to reconfig. Might be
>>>>>> safer to just do that if we're not sure.
>>>>>> 
>>>>>> 
>>>>>> Re classification - we could do the same thing with reconfig, but I
>>>>>> think that would be a mistake. If we feel strongly where it should
>>>>>> live long term we should just move it now.
>>>>>> 
>>>>>> Patrick
>>>>>> 
>>>>>>> 
>>>>>>> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org>
>>>> wrote:
>>>>>>> 
>>>>>>>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <
>>> shralex@gmail.com
>>>>> 
>>>>>>>> wrote:
>>>>>>>>> Hi Patrick, Flavio,
>>>>>>>>> 
>>>>>>>>> Since there seems to be consensus on this, I can add this flag,
>>>> unless
>>>>>>>>> someone else wants to. I assume that getConfig should still
>> work
>>>>>>>> regardless
>>>>>>>>> of the flag ? is there a security concern with clients knowing
>>> the
>>>>>> list
>>>>>>>> of
>>>>>>>>> servers?
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> We've always hidden that detail from users. We don't even expose
>>>> which
>>>>>>>> server you're connected to today. I remember Ben (and perhaps
>>>> Flavio?)
>>>>>>>> highlighting this as important to maintain although I'm not super
>>>>>>>> familiar with the specifics on why. It made sense to me though
>> from
>>>>>>>> the perspective that we don't want clients to make assumptions
>> that
>>>>>>>> probably shouldn't.
>>>>>>>> 
>>>>>>>> My thinking is that we should 1) add a config option to enable
>>>>>>>> reconfig (off by default), 2) move reconfig specific
>> functionality
>>>>>>>> from ZooKeeper.java (including getconfig) into an "admin"
>> package,
>>>>>>>> within say a class ZooKeeperAdmin, 3) document/test use of ACLs
>> for
>>>>>>>> when folks do want to enable reconfig and are also worried about
>>>> auth.
>>>>>>>> (e.g. turn on kerb)
>>>>>>>> 
>>>>>>>> Again, I don't see any of this as a quality issue personally. As
>>> such
>>>>>>>> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if
>>> we
>>>>>>>> were interested in doing such a release. Adjusting the API should
>>> be
>>>>>>>> done before we move to "beta" though. Although that seems like a
>>>>>>>> pretty mechanical (eclipse/idea) type refactoring?
>>>>>>>> 
>>>>>>>> Patrick
>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> Alex
>>>>>>>>> On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org>
>>> wrote:
>>>>>>>>> 
>>>>>>>>>> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <
>>> fpj@apache.org
>>>>> 
>>>>>>>> wrote:
>>>>>>>>>>> I gotta say that I'm not super excited about this option,
>> but
>>>> for
>>>>>> some
>>>>>>>>>> reason I ended up carrying the flag. To recap, I just raised
>>> this
>>>>>> option
>>>>>>>>>> because it seems that there are folks interested in features
>> in
>>>> 3.5
>>>>>> like
>>>>>>>>>> SSL and not necessarily in reconfiguration. SSL is important
>> and
>>>> to
>>>>>> take
>>>>>>>>>> Kafka as an example, it sucks that we can't have a whole set
>> up
>>>> using
>>>>>>>> SSL.
>>>>>>>>>> For ZK, the real questions are:
>>>>>>>>>>> 
>>>>>>>>>>> 1- how fast can we make 3.5 stable?
>>>>>>>>>>> 2- would it be faster if we have a way of disabling
>>>>>> reconfiguration?
>>>>>>>>>>> 3- would enough users care about a stable 3.5 that has
>>>>>> reconfiguration
>>>>>>>>>> disabled?
>>>>>>>>>>> 
>>>>>>>>>>> It is taking a long time to get 3.5 to beta. There has been
>>> some
>>>>>> good
>>>>>>>>>> activity around 3.5.2 release, which is a great step, but it
>> is
>>>>>> unclear
>>>>>>>>>> when 3.5.3 is going to come and if we will be able to make 3.5
>>>> beta
>>>>>>>> then.
>>>>>>>>>> Frankly, disabling reconfiguration sounds undesirable because
>> it
>>>> is
>>>>>> an
>>>>>>>>>> important feature, but I currently don't use it in production,
>>> so
>>>>>> from a
>>>>>>>>>> practical point of view, I can go both ways. Whether we go
>>> through
>>>>>> the
>>>>>>>>>> trouble of doing 2 depends on users interested in that option
>>> and
>>>>>> folks
>>>>>>>>>> willing to implement it.
>>>>>>>>>>> 
>>>>>>>>>>> To answer your question, Powell, my pseudo-proposal is kind
>>> of a
>>>>>> funny
>>>>>>>>>> option because once the feature is stable, then we wouldn't
>>> need a
>>>>>>>> switch
>>>>>>>>>> any longer, so there is not need of a deprecation path, we
>> just
>>>> start
>>>>>>>>>> ignoring it from the first beta release. Until it is beta, I'd
>>> say
>>>>>> that
>>>>>>>>>> default is disabled.
>>>>>>>>>> 
>>>>>>>>>> I would argue that we need this even when it does become
>> stable.
>>>> To
>>>>>> me
>>>>>>>>>> this is not a quality issue so much as it is an auth issue. We
>>>> want
>>>>>> to
>>>>>>>>>> make it simple for folks to run a vanilla/old ZK cluster and
>> not
>>>>>> worry
>>>>>>>>>> about the security implications of having reconfig enabled.
>>>>>>>>>> 
>>>>>>>>>> Patrick
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> -Flavio
>>>>>>>>>>> 
>>>>>>>>>>>> On 17 Mar 2016, at 17:44, powell molleti
>>>>>> <powellm79@yahoo.com.INVALID
>>>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi Flavio,
>>>>>>>>>>>> 
>>>>>>>>>>>> Generally do config options and command line args come
>> under
>>>> the
>>>>>> same
>>>>>>>>>> SLA as API?. I was assuming as such hence my question. Perhaps
>>> if
>>>> the
>>>>>>>>>> expectation is that this config option is temporary from get
>> go
>>>> then
>>>>>>>> may be
>>>>>>>>>> it is ok. The default for re-config support will be enabled or
>>>>>>>> disabled?.
>>>>>>>>>>>> 
>>>>>>>>>>>> I am just thinking from provisioning point of view when
>>> people
>>>>>>>> generate
>>>>>>>>>> config options etc.
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks
>>>>>>>>>>>> Powell.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
>>>>>>>> fpj@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>> Hi Powell,
>>>>>>>>>>>> 
>>>>>>>>>>>> I was thinking config file and system property server side.
>>>> What's
>>>>>>>> your
>>>>>>>>>> concern with compatibility? The API itself wouldn't change,
>> but
>>>> the
>>>>>>>> config
>>>>>>>>>> option wouldn't exist in previous versions and would not exist
>>>>>> either in
>>>>>>>>>> later stable versions of 3.5.
>>>>>>>>>>>> 
>>>>>>>>>>>> -Flavio
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> On 16 Mar 2016, at 22:08, powell molleti
>>>>>>>> <po...@yahoo.com.INVALID>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Will this option be supplied via config file/args/API?.
>> Will
>>>> this
>>>>>>>>>> option be a temporary thing i.e what about its compatibility?.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> thanks
>>>>>>>>>>>>> Powell.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
>>>>>>>> fpj@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> The main issue to sort out is stability of the API. There
>>> is a
>>>>>>>>>> security concern around the fact that clients can freely
>>>> reconfigure
>>>>>> the
>>>>>>>>>> ensemble. If we follow the plan that Pat proposed some time
>> ago:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> 
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>>>>>>>>>> <
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> 
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Locking the API is the main step to move it to beta.
>> Sorting
>>>> out
>>>>>>>> bugs
>>>>>>>>>> is definitely necessary, but it isn't the main thing that is
>>>> keeping
>>>>>>>> 3.5 in
>>>>>>>>>> alpha.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> About making it experimental, I was raising the option of
>>>> having
>>>>>> a
>>>>>>>>>> switch that disables the API calls, not the code. The reason
>> why
>>>> that
>>>>>>>> could
>>>>>>>>>> work is that anyone using 3.5 who uses the "experimental" API
>>> must
>>>>>>>> explicit
>>>>>>>>>> turn on the switch and enable the calls. If they do it, they
>>> need
>>>> to
>>>>>> be
>>>>>>>>>> aware that the API can change.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I must say that I haven't really looked closely into doing
>>> it,
>>>>>> and
>>>>>>>> I'm
>>>>>>>>>> not even entirely convinced that this is a good idea, but
>> since
>>>> Jason
>>>>>>>>>> raised the point, I'm exploring options.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -Flavio
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <
>>>> shralex@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Looking at the list of ~50 blocker and critical bugs in
>>>>>> ZooKeeper,
>>>>>>>>>> only 3-4
>>>>>>>>>>>>>> are related to reconfig. Given this, and the fact that it
>>> is
>>>>>> run in
>>>>>>>>>>>>>> production since 2012 in multiple companies, I don't
>> think
>>>> its
>>>>>> more
>>>>>>>>>>>>>> unstable than any other part of ZooKeeper.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> There are multiple reconfig-related bugs that turned out
>>>> really
>>>>>>>>>> difficult
>>>>>>>>>>>>>> to debug without access to the actual system or at least
>> to
>>>> the
>>>>>>>> Hudson
>>>>>>>>>>>>>> machines where some tests are failing. In the past, Michi
>>>> and I
>>>>>>>>>> physically
>>>>>>>>>>>>>> went to Hortonworks to debug one such issue, but this is
>> of
>>>>>> course
>>>>>>>>>> not a
>>>>>>>>>>>>>> good method, and we weren't able to arrange such a visit
>>>> again.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regarding making it optional - the reconfig logic has
>>> several
>>>>>>>>>> different
>>>>>>>>>>>>>> parts, some would be really difficult to disable using a
>>>>>>>> configuration
>>>>>>>>>>>>>> parameter. But the actual dynamic expansion of the
>> cluster
>>> is
>>>>>>>>>> triggered by
>>>>>>>>>>>>>> the reconfig command, so it should not affect users who
>>> don't
>>>>>>>> invoke
>>>>>>>>>> it.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
>>>>>>>> fpj@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I suppose we could give it a try. How do other folks
>> feel
>>>> about
>>>>>>>> it?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> -Flavio
>>>>>>>>>>>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <
>> jbr@squareup.com
>>>> 
>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> So, you could enable the dynamic reconfiguration
>> feature
>>>>>> behind a
>>>>>>>>>> config
>>>>>>>>>>>>>>>> option, and document that it should only be enabled
>>>>>>>> experimentally,
>>>>>>>>>> use
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> your own risk.  Keep it off by default.  Allow only
>>> static
>>>>>>>> config by
>>>>>>>>>>>>>>>> default, until it's stable?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Jason
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
>>>>>>>> fpj@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Hi Jason,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> The consumer in Kafka is pretty independent from the
>>> core
>>>>>>>>>> (brokers),
>>>>>>>>>>>>>>>>> that's how that project manages to make such a
>>>> separation. We
>>>>>>>> don't
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> the same with ZooKeeper as the feature we are talking
>>>> about
>>>>>> is
>>>>>>>>>> part of
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> server and the only way I see of doing what you say is
>>> to
>>>>>> turn
>>>>>>>> off
>>>>>>>>>>>>>>>>> features. More specifically, we'd need to disable the
>>>>>> reconfig
>>>>>>>> API
>>>>>>>>>> and
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>> not allow any change to the configuration, even though
>>> the
>>>>>> code
>>>>>>>> is
>>>>>>>>>>>>>>> there.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Reconfig here refers to the ability of changing the
>>>>>>>> configuration
>>>>>>>>>> of an
>>>>>>>>>>>>>>>>> ensemble (e.g., changing the set of servers).
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> -Flavio
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <
>>>> jbr@squareup.com
>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> So, it would seem sensible to me to have a release
>>> where
>>>> all
>>>>>>>>>> features
>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> stable, except where noted.  E.g. mark certain
>> features
>>>> as
>>>>>> only
>>>>>>>>>>>>>>> 'alpha
>>>>>>>>>>>>>>>>>> quality', e.g. the 're-config feature'.  (I assume
>> it's
>>>>>>>> possible
>>>>>>>>>> to
>>>>>>>>>>>>>>>>> happily
>>>>>>>>>>>>>>>>>> use 3.5.X without exercising the unstable re-config
>>>> bits?).
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> There's precedent for doing this sort of thing in
>> other
>>>>>>>> projects,
>>>>>>>>>>>>>>> e.g.
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> Kafka, they've had several release where a new
>>> "Consumer
>>>>>> API"
>>>>>>>> is
>>>>>>>>>>>>>>>> shipped
>>>>>>>>>>>>>>>>>> that is available for beta-testing, but you can still
>>>> just
>>>>>> use
>>>>>>>> the
>>>>>>>>>>>>>>>> older
>>>>>>>>>>>>>>>>>> stable consumer api, etc.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Jason
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>>>>>>>>>>>>>> <powellm79@yahoo.com.invalid
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hi Doug,
>>>>>>>>>>>>>>>>>>> Is 3.5 being an alpha release preventing you from
>>> using
>>>>>> it?.
>>>>>>>> Or
>>>>>>>>>> have
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> run into issues with it?. In general perhaps ZK 3.5
>>>> being
>>>>>>>>>> labeled as
>>>>>>>>>>>>>>>>> alpha
>>>>>>>>>>>>>>>>>>> might not be fair, since it is far more stable then
>>> what
>>>>>> most
>>>>>>>>>> people
>>>>>>>>>>>>>>>>>>> associate an alpha release to be.
>>>>>>>>>>>>>>>>>>> Perhaps if you do not use re-config feature may be
>> it
>>>> will
>>>>>>>> just
>>>>>>>>>> work
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> you?.
>>>>>>>>>>>>>>>>>>> There are many examples of 3.5.X being used in
>>>> productions
>>>>>>>> from
>>>>>>>>>> my
>>>>>>>>>>>>>>>>> limited
>>>>>>>>>>>>>>>>>>> knowledge.
>>>>>>>>>>>>>>>>>>> ThanksPowell.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio
>>> Junqueira <
>>>>>>>>>>>>>>>>> fpj@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> None of us expected the reconfig changes to take
>> this
>>>> long
>>>>>> to
>>>>>>>>>>>>>>>> stabilize.
>>>>>>>>>>>>>>>>>>> Until we get there, I don't think we can do anything
>>>> else
>>>>>> with
>>>>>>>>>> 3.5.
>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>> best bet we have is to work harder to bring 3.5
>> into a
>>>>>> stable
>>>>>>>>>> rather
>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>> trying to work around it.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> There are lots of people interested in seeing 3.5
>>>> stable,
>>>>>> and
>>>>>>>> if
>>>>>>>>>> we
>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>> everyone to contribute more patches and code
>> reviews,
>>> we
>>>>>>>> should
>>>>>>>>>> be
>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> do it sooner. After all, it is a community based
>>>> effort, so
>>>>>>>> the
>>>>>>>>>>>>>>>>> community
>>>>>>>>>>>>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> -Flavio
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
>>>>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Doug, I forgot to respond to your question about
>> 3.4.
>>>>>> Since
>>>>>>>>>> 3.4 is
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> stable maintenance line, we are very conservative
>>> about
>>>>>>>>>>>>>>> back-porting
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> it.  Our policy is to limit back-ports to critical
>>> bug
>>>>>> fixes
>>>>>>>> and
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>> introduce any new features in the 3.4 line.  This
>> is
>>> a
>>>>>>>> matter of
>>>>>>>>>>>>>>>>> managing
>>>>>>>>>>>>>>>>>>>> risk.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Jason, your question about release cadence is a
>> fair
>>>>>> one.  If
>>>>>>>>>> it's
>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>> consolation, we are now taking the approach of
>> trying
>>>> to
>>>>>>>> limit
>>>>>>>>>> the
>>>>>>>>>>>>>>>>> scope
>>>>>>>>>>>>>>>>>>>> of anything new introduced in 3.5 too.  That would
>>>> allow
>>>>>> us
>>>>>>>> to
>>>>>>>>>>>>>>> focus
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>> stabilization: resolving blocker bugs and freezing
>>>> public
>>>>>>>>>> APIs.  I
>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>> this will help us accelerate the releases into beta
>>> and
>>>>>>>> eventual
>>>>>>>>>>>>>>> GA.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I am new to ZooKeeper release management, so I'd
>> like
>>>> to
>>>>>> hear
>>>>>>>>>>>>>>>> thoughts
>>>>>>>>>>>>>>>>>>>> from more experienced committers and PMC members
>>> about
>>>>>> your
>>>>>>>>>>>>>>> proposal
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> try to cut a stable release for a limited subset of
>>>> what
>>>>>> is
>>>>>>>> in
>>>>>>>>>>>>>>>>> branch-3.5
>>>>>>>>>>>>>>>>>>>> now.  My instinct is that it would be challenging
>> to
>>>>>>>> cherry-pick
>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This
>>>> would
>>>>>>>> become
>>>>>>>>>>>>>>>>> another
>>>>>>>>>>>>>>>>>>>> release line for an already resource-constrained
>>>> volunteer
>>>>>>>>>> staff to
>>>>>>>>>>>>>>>>>>>> manage.  I'd prefer to dedicate those limited
>>>> resources to
>>>>>>>>>> overall
>>>>>>>>>>>>>>>> 3.5
>>>>>>>>>>>>>>>>>>>> stabilization.  Also, a 3.5 release in which
>> certain
>>>>>> features
>>>>>>>>>>>>>>>>> "vanished"
>>>>>>>>>>>>>>>>>>>> because of not meeting some stability criteria
>> would
>>> be
>>>>>>>>>>>>>>> undesirable.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <
>>>> jbr@squareup.com
>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Chris,
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Can you say whether some parts of 3.5.X are more
>>>> stable
>>>>>> than
>>>>>>>>>>>>>>> others
>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>> if we don't care about certain new features, is it
>>>>>>>> relatively
>>>>>>>>>>>>>>>> stable)?
>>>>>>>>>>>>>>>>>>>>> Would it be possible to cut out a version that
>> only
>>>> has
>>>>>> the
>>>>>>>>>> bits
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>> are stable (and release that)?
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> From that timeline, and the historic release
>>> cadence,
>>>> it
>>>>>>>> would
>>>>>>>>>>>>>>> seem
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> years away before we get to the stable release?
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Jason
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>>>>>>>>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Hello Doug,
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> At this point, I think we're still pretty far
>> away
>>>> from
>>>>>>>>>>>>>>> declaring a
>>>>>>>>>>>>>>>>>>>>>> stable
>>>>>>>>>>>>>>>>>>>>>> release in the 3.5 line.  I don't think we're
>> close
>>>>>> enough
>>>>>>>>>> that
>>>>>>>>>>>>>>>>> anyone
>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread
>>> that
>>>>>>>>>> describes
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> high-level strategy for release planning in the
>> 3.5
>>>>>> line:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're
>>>> working
>>>>>> on
>>>>>>>>>>>>>>>> resolving a
>>>>>>>>>>>>>>>>>>>>>> few
>>>>>>>>>>>>>>>>>>>>>> more blockers before we produce a release
>>> candidate.
>>>>>>>>>> Hopefully
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <itsbehind@gmail.com
>>> 
>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I know it's only been a few months, but I was
>>>>>> wondering if
>>>>>>>>>> there
>>>>>>>>>>>>>>>>> was a
>>>>>>>>>>>>>>>>>>>>>>> ballpark release date for a stable version of
>>> 3.5.1.
>>>>>> Or is
>>>>>>>>>> there
>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>>> chance
>>>>>>>>>>>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just
>>>> another
>>>>>>>> person
>>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>> that feature in a stable version. Thanks for all
>>> you
>>>>>> do!
>>>>>>>> :)
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> 
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>>>>>>>>>>>>>> e
>>>>>>>>>>>>>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>>>>>>>>>>>>>> Sent from the zookeeper-user mailing list
>> archive
>>> at
>>>>>>>>>> Nabble.com.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> 
>> 

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

I'm not so sure its orthogonal. The question is whether someone would ever
want to use reconfig without ACLs,
as this allows any client to reconfigure the servers away or add a bunch of
servers that shouldn't be there :) and whether we should facilitate this
knowing its insecure.

Requiring ACLs solves the security concern for both reconfig and getconfig.
For example, if you don't want your clients to know the list of servers,
limit their read permissions on the configuration znode.

On Thu, Mar 24, 2016 at 2:11 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> seems like an orthogonal requirement?
>
> On Thu, Mar 24, 2016 at 3:37 PM, Alexander Shraer <sh...@gmail.com>
> wrote:
>
> > How about a simpler alternative to the proposed flag for reconfig: a
> check
> > in the code that requires ACLs to be set.
> > If people want to use reconfig, they should use ACLs too.
> >
> > What do you think ?
> >
> > Alex
> >
> > On Mon, Mar 21, 2016 at 9:58 PM, Patrick Hunt <ph...@apache.org> wrote:
> >
> > > I would say if in doubt add a safety. (a config parameter to turn it
> > > off). Cost is almost zero and worst case it will just give us peace of
> > > mind. ;-)
> > >
> > > Patrick
> > >
> > > On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <sh...@gmail.com>
> > > wrote:
> > > > ok, thanks for the suggestion, I'll look into it. For reconfig I
> think
> > > its
> > > > pretty clear that its an admin
> > > > functionality. I just always imagined that its controlled via acls,
> > but I
> > > > understand
> > > > the concerns now.
> > > >
> > > > getConfig returns the dynamic config (list of all servers with all
> > ports
> > > > and quorum system if defined)
> > > > and has an option to filter that info and just return the server
> > > connection
> > > > string (server and client port only).
> > > >
> > > >
> > > > On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org>
> > wrote:
> > > >
> > > >> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <
> shralex@gmail.com>
> > > >> wrote:
> > > >> > I don't think that getConfig should be an admin functionality. It
> is
> > > >> > essential for client-side re-balancing
> > > >> > that we implemented (all clients shoudl be able to detect
> > > configuration
> > > >> > changes via getConfig). It could
> > > >> > be hidden somewhat by defining higher-level re-balancing
> > > >> > policies (ZOOKEEPER-2016)
> > > >> > but there hasn't been enough progress on that. Perhaps instead
> > > getConfig
> > > >> > should be controlled
> > > >> > by a separate flag ?
> > > >> >
> > > >>
> > > >> I believe that the Hadoop community has something we could use:
> > > >>
> > > >>
> > >
> >
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
> > > >> (whether through annotations or just documenting it in the API
> > javadoc)
> > > >>
> > > >> e.g. we could list getConfig as public/unstable for example and
> still
> > > >> ship it as GA. That would mark it as something that could change re
> > > >> API policy.
> > > >>
> > > >> Is the entire config exposed through getConfig? If so then we might
> > > >> want to enable/disable it with a flag similar to reconfig. Might be
> > > >> safer to just do that if we're not sure.
> > > >>
> > > >>
> > > >> Re classification - we could do the same thing with reconfig, but I
> > > >> think that would be a mistake. If we feel strongly where it should
> > > >> live long term we should just move it now.
> > > >>
> > > >> Patrick
> > > >>
> > > >> >
> > > >> > On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org>
> > > wrote:
> > > >> >
> > > >> >> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <
> > shralex@gmail.com
> > > >
> > > >> >> wrote:
> > > >> >> > Hi Patrick, Flavio,
> > > >> >> >
> > > >> >> > Since there seems to be consensus on this, I can add this flag,
> > > unless
> > > >> >> > someone else wants to. I assume that getConfig should still
> work
> > > >> >> regardless
> > > >> >> > of the flag ? is there a security concern with clients knowing
> > the
> > > >> list
> > > >> >> of
> > > >> >> > servers?
> > > >> >> >
> > > >> >>
> > > >> >> We've always hidden that detail from users. We don't even expose
> > > which
> > > >> >> server you're connected to today. I remember Ben (and perhaps
> > > Flavio?)
> > > >> >> highlighting this as important to maintain although I'm not super
> > > >> >> familiar with the specifics on why. It made sense to me though
> from
> > > >> >> the perspective that we don't want clients to make assumptions
> that
> > > >> >> probably shouldn't.
> > > >> >>
> > > >> >> My thinking is that we should 1) add a config option to enable
> > > >> >> reconfig (off by default), 2) move reconfig specific
> functionality
> > > >> >> from ZooKeeper.java (including getconfig) into an "admin"
> package,
> > > >> >> within say a class ZooKeeperAdmin, 3) document/test use of ACLs
> for
> > > >> >> when folks do want to enable reconfig and are also worried about
> > > auth.
> > > >> >> (e.g. turn on kerb)
> > > >> >>
> > > >> >> Again, I don't see any of this as a quality issue personally. As
> > such
> > > >> >> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if
> > we
> > > >> >> were interested in doing such a release. Adjusting the API should
> > be
> > > >> >> done before we move to "beta" though. Although that seems like a
> > > >> >> pretty mechanical (eclipse/idea) type refactoring?
> > > >> >>
> > > >> >> Patrick
> > > >> >>
> > > >> >> > Cheers,
> > > >> >> > Alex
> > > >> >> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org>
> > wrote:
> > > >> >> >
> > > >> >> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <
> > fpj@apache.org
> > > >
> > > >> >> wrote:
> > > >> >> >> > I gotta say that I'm not super excited about this option,
> but
> > > for
> > > >> some
> > > >> >> >> reason I ended up carrying the flag. To recap, I just raised
> > this
> > > >> option
> > > >> >> >> because it seems that there are folks interested in features
> in
> > > 3.5
> > > >> like
> > > >> >> >> SSL and not necessarily in reconfiguration. SSL is important
> and
> > > to
> > > >> take
> > > >> >> >> Kafka as an example, it sucks that we can't have a whole set
> up
> > > using
> > > >> >> SSL.
> > > >> >> >> For ZK, the real questions are:
> > > >> >> >> >
> > > >> >> >> > 1- how fast can we make 3.5 stable?
> > > >> >> >> > 2- would it be faster if we have a way of disabling
> > > >> reconfiguration?
> > > >> >> >> > 3- would enough users care about a stable 3.5 that has
> > > >> reconfiguration
> > > >> >> >> disabled?
> > > >> >> >> >
> > > >> >> >> > It is taking a long time to get 3.5 to beta. There has been
> > some
> > > >> good
> > > >> >> >> activity around 3.5.2 release, which is a great step, but it
> is
> > > >> unclear
> > > >> >> >> when 3.5.3 is going to come and if we will be able to make 3.5
> > > beta
> > > >> >> then.
> > > >> >> >> Frankly, disabling reconfiguration sounds undesirable because
> it
> > > is
> > > >> an
> > > >> >> >> important feature, but I currently don't use it in production,
> > so
> > > >> from a
> > > >> >> >> practical point of view, I can go both ways. Whether we go
> > through
> > > >> the
> > > >> >> >> trouble of doing 2 depends on users interested in that option
> > and
> > > >> folks
> > > >> >> >> willing to implement it.
> > > >> >> >> >
> > > >> >> >> > To answer your question, Powell, my pseudo-proposal is kind
> > of a
> > > >> funny
> > > >> >> >> option because once the feature is stable, then we wouldn't
> > need a
> > > >> >> switch
> > > >> >> >> any longer, so there is not need of a deprecation path, we
> just
> > > start
> > > >> >> >> ignoring it from the first beta release. Until it is beta, I'd
> > say
> > > >> that
> > > >> >> >> default is disabled.
> > > >> >> >>
> > > >> >> >> I would argue that we need this even when it does become
> stable.
> > > To
> > > >> me
> > > >> >> >> this is not a quality issue so much as it is an auth issue. We
> > > want
> > > >> to
> > > >> >> >> make it simple for folks to run a vanilla/old ZK cluster and
> not
> > > >> worry
> > > >> >> >> about the security implications of having reconfig enabled.
> > > >> >> >>
> > > >> >> >> Patrick
> > > >> >> >>
> > > >> >> >> >
> > > >> >> >> > -Flavio
> > > >> >> >> >
> > > >> >> >> >> On 17 Mar 2016, at 17:44, powell molleti
> > > >> <powellm79@yahoo.com.INVALID
> > > >> >> >
> > > >> >> >> wrote:
> > > >> >> >> >>
> > > >> >> >> >> Hi Flavio,
> > > >> >> >> >>
> > > >> >> >> >> Generally do config options and command line args come
> under
> > > the
> > > >> same
> > > >> >> >> SLA as API?. I was assuming as such hence my question. Perhaps
> > if
> > > the
> > > >> >> >> expectation is that this config option is temporary from get
> go
> > > then
> > > >> >> may be
> > > >> >> >> it is ok. The default for re-config support will be enabled or
> > > >> >> disabled?.
> > > >> >> >> >>
> > > >> >> >> >> I am just thinking from provisioning point of view when
> > people
> > > >> >> generate
> > > >> >> >> config options etc.
> > > >> >> >> >>
> > > >> >> >> >> Thanks
> > > >> >> >> >> Powell.
> > > >> >> >> >>
> > > >> >> >> >>
> > > >> >> >> >>
> > > >> >> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
> > > >> >> fpj@apache.org>
> > > >> >> >> wrote:
> > > >> >> >> >> Hi Powell,
> > > >> >> >> >>
> > > >> >> >> >> I was thinking config file and system property server side.
> > > What's
> > > >> >> your
> > > >> >> >> concern with compatibility? The API itself wouldn't change,
> but
> > > the
> > > >> >> config
> > > >> >> >> option wouldn't exist in previous versions and would not exist
> > > >> either in
> > > >> >> >> later stable versions of 3.5.
> > > >> >> >> >>
> > > >> >> >> >> -Flavio
> > > >> >> >> >>
> > > >> >> >> >>
> > > >> >> >> >>
> > > >> >> >> >>> On 16 Mar 2016, at 22:08, powell molleti
> > > >> >> <po...@yahoo.com.INVALID>
> > > >> >> >> wrote:
> > > >> >> >> >>>
> > > >> >> >> >>> Will this option be supplied via config file/args/API?.
> Will
> > > this
> > > >> >> >> option be a temporary thing i.e what about its compatibility?.
> > > >> >> >> >>>
> > > >> >> >> >>> thanks
> > > >> >> >> >>> Powell.
> > > >> >> >> >>>
> > > >> >> >> >>>
> > > >> >> >> >>>
> > > >> >> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
> > > >> >> fpj@apache.org>
> > > >> >> >> wrote:
> > > >> >> >> >>> The main issue to sort out is stability of the API. There
> > is a
> > > >> >> >> security concern around the fact that clients can freely
> > > reconfigure
> > > >> the
> > > >> >> >> ensemble. If we follow the plan that Pat proposed some time
> ago:
> > > >> >> >> >>>
> > > >> >> >> >>>
> > > >> >> >>
> > > >> >>
> > > >>
> > >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > > >> >> >> <
> > > >> >> >>
> > > >> >>
> > > >>
> > >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > > >> >> >> >
> > > >> >> >> >>>
> > > >> >> >> >>> Locking the API is the main step to move it to beta.
> Sorting
> > > out
> > > >> >> bugs
> > > >> >> >> is definitely necessary, but it isn't the main thing that is
> > > keeping
> > > >> >> 3.5 in
> > > >> >> >> alpha.
> > > >> >> >> >>>
> > > >> >> >> >>> About making it experimental, I was raising the option of
> > > having
> > > >> a
> > > >> >> >> switch that disables the API calls, not the code. The reason
> why
> > > that
> > > >> >> could
> > > >> >> >> work is that anyone using 3.5 who uses the "experimental" API
> > must
> > > >> >> explicit
> > > >> >> >> turn on the switch and enable the calls. If they do it, they
> > need
> > > to
> > > >> be
> > > >> >> >> aware that the API can change.
> > > >> >> >> >>>
> > > >> >> >> >>> I must say that I haven't really looked closely into doing
> > it,
> > > >> and
> > > >> >> I'm
> > > >> >> >> not even entirely convinced that this is a good idea, but
> since
> > > Jason
> > > >> >> >> raised the point, I'm exploring options.
> > > >> >> >> >>>
> > > >> >> >> >>> -Flavio
> > > >> >> >> >>>
> > > >> >> >> >>>
> > > >> >> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <
> > > shralex@gmail.com>
> > > >> >> wrote:
> > > >> >> >> >>>>
> > > >> >> >> >>>> Looking at the list of ~50 blocker and critical bugs in
> > > >> ZooKeeper,
> > > >> >> >> only 3-4
> > > >> >> >> >>>> are related to reconfig. Given this, and the fact that it
> > is
> > > >> run in
> > > >> >> >> >>>> production since 2012 in multiple companies, I don't
> think
> > > its
> > > >> more
> > > >> >> >> >>>> unstable than any other part of ZooKeeper.
> > > >> >> >> >>>>
> > > >> >> >> >>>> There are multiple reconfig-related bugs that turned out
> > > really
> > > >> >> >> difficult
> > > >> >> >> >>>> to debug without access to the actual system or at least
> to
> > > the
> > > >> >> Hudson
> > > >> >> >> >>>> machines where some tests are failing. In the past, Michi
> > > and I
> > > >> >> >> physically
> > > >> >> >> >>>> went to Hortonworks to debug one such issue, but this is
> of
> > > >> course
> > > >> >> >> not a
> > > >> >> >> >>>> good method, and we weren't able to arrange such a visit
> > > again.
> > > >> >> >> >>>>
> > > >> >> >> >>>> Regarding making it optional - the reconfig logic has
> > several
> > > >> >> >> different
> > > >> >> >> >>>> parts, some would be really difficult to disable using a
> > > >> >> configuration
> > > >> >> >> >>>> parameter. But the actual dynamic expansion of the
> cluster
> > is
> > > >> >> >> triggered by
> > > >> >> >> >>>> the reconfig command, so it should not affect users who
> > don't
> > > >> >> invoke
> > > >> >> >> it.
> > > >> >> >> >>>>
> > > >> >> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
> > > >> >> fpj@apache.org>
> > > >> >> >> wrote:
> > > >> >> >> >>>>
> > > >> >> >> >>>>> I suppose we could give it a try. How do other folks
> feel
> > > about
> > > >> >> it?
> > > >> >> >> >>>>>
> > > >> >> >> >>>>> -Flavio
> > > >> >> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <
> jbr@squareup.com
> > >
> > > >> wrote:
> > > >> >> >> >>>>>
> > > >> >> >> >>>>>> So, you could enable the dynamic reconfiguration
> feature
> > > >> behind a
> > > >> >> >> config
> > > >> >> >> >>>>>> option, and document that it should only be enabled
> > > >> >> experimentally,
> > > >> >> >> use
> > > >> >> >> >>>>> at
> > > >> >> >> >>>>>> your own risk.  Keep it off by default.  Allow only
> > static
> > > >> >> config by
> > > >> >> >> >>>>>> default, until it's stable?
> > > >> >> >> >>>>>>
> > > >> >> >> >>>>>> Jason
> > > >> >> >> >>>>>>
> > > >> >> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
> > > >> >> fpj@apache.org>
> > > >> >> >> >>>>> wrote:
> > > >> >> >> >>>>>>
> > > >> >> >> >>>>>>> Hi Jason,
> > > >> >> >> >>>>>>>
> > > >> >> >> >>>>>>> The consumer in Kafka is pretty independent from the
> > core
> > > >> >> >> (brokers),
> > > >> >> >> >>>>>>> that's how that project manages to make such a
> > > separation. We
> > > >> >> don't
> > > >> >> >> >>>>> have
> > > >> >> >> >>>>>>> the same with ZooKeeper as the feature we are talking
> > > about
> > > >> is
> > > >> >> >> part of
> > > >> >> >> >>>>>> the
> > > >> >> >> >>>>>>> server and the only way I see of doing what you say is
> > to
> > > >> turn
> > > >> >> off
> > > >> >> >> >>>>>>> features. More specifically, we'd need to disable the
> > > >> reconfig
> > > >> >> API
> > > >> >> >> and
> > > >> >> >> >>>>> do
> > > >> >> >> >>>>>>> not allow any change to the configuration, even though
> > the
> > > >> code
> > > >> >> is
> > > >> >> >> >>>>> there.
> > > >> >> >> >>>>>>>
> > > >> >> >> >>>>>>> Reconfig here refers to the ability of changing the
> > > >> >> configuration
> > > >> >> >> of an
> > > >> >> >> >>>>>>> ensemble (e.g., changing the set of servers).
> > > >> >> >> >>>>>>>
> > > >> >> >> >>>>>>> -Flavio
> > > >> >> >> >>>>>>>
> > > >> >> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <
> > > jbr@squareup.com
> > > >> >
> > > >> >> >> wrote:
> > > >> >> >> >>>>>>>>
> > > >> >> >> >>>>>>>> So, it would seem sensible to me to have a release
> > where
> > > all
> > > >> >> >> features
> > > >> >> >> >>>>>> are
> > > >> >> >> >>>>>>>> stable, except where noted.  E.g. mark certain
> features
> > > as
> > > >> only
> > > >> >> >> >>>>> 'alpha
> > > >> >> >> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume
> it's
> > > >> >> possible
> > > >> >> >> to
> > > >> >> >> >>>>>>> happily
> > > >> >> >> >>>>>>>> use 3.5.X without exercising the unstable re-config
> > > bits?).
> > > >> >> >> >>>>>>>>
> > > >> >> >> >>>>>>>> There's precedent for doing this sort of thing in
> other
> > > >> >> projects,
> > > >> >> >> >>>>> e.g.
> > > >> >> >> >>>>>> in
> > > >> >> >> >>>>>>>> Kafka, they've had several release where a new
> > "Consumer
> > > >> API"
> > > >> >> is
> > > >> >> >> >>>>>> shipped
> > > >> >> >> >>>>>>>> that is available for beta-testing, but you can still
> > > just
> > > >> use
> > > >> >> the
> > > >> >> >> >>>>>> older
> > > >> >> >> >>>>>>>> stable consumer api, etc.
> > > >> >> >> >>>>>>>>
> > > >> >> >> >>>>>>>> Jason
> > > >> >> >> >>>>>>>>
> > > >> >> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > > >> >> >> >>>>>>> <powellm79@yahoo.com.invalid
> > > >> >> >> >>>>>>>>> wrote:
> > > >> >> >> >>>>>>>>
> > > >> >> >> >>>>>>>>> Hi Doug,
> > > >> >> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from
> > using
> > > >> it?.
> > > >> >> Or
> > > >> >> >> have
> > > >> >> >> >>>>>> you
> > > >> >> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5
> > > being
> > > >> >> >> labeled as
> > > >> >> >> >>>>>>> alpha
> > > >> >> >> >>>>>>>>> might not be fair, since it is far more stable then
> > what
> > > >> most
> > > >> >> >> people
> > > >> >> >> >>>>>>>>> associate an alpha release to be.
> > > >> >> >> >>>>>>>>> Perhaps if you do not use re-config feature may be
> it
> > > will
> > > >> >> just
> > > >> >> >> work
> > > >> >> >> >>>>>> for
> > > >> >> >> >>>>>>>>> you?.
> > > >> >> >> >>>>>>>>> There are many examples of 3.5.X being used in
> > > productions
> > > >> >> from
> > > >> >> >> my
> > > >> >> >> >>>>>>> limited
> > > >> >> >> >>>>>>>>> knowledge.
> > > >> >> >> >>>>>>>>> ThanksPowell.
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio
> > Junqueira <
> > > >> >> >> >>>>>>> fpj@apache.org>
> > > >> >> >> >>>>>>>>> wrote:
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>>> None of us expected the reconfig changes to take
> this
> > > long
> > > >> to
> > > >> >> >> >>>>>> stabilize.
> > > >> >> >> >>>>>>>>> Until we get there, I don't think we can do anything
> > > else
> > > >> with
> > > >> >> >> 3.5.
> > > >> >> >> >>>>>> The
> > > >> >> >> >>>>>>>>> best bet we have is to work harder to bring 3.5
> into a
> > > >> stable
> > > >> >> >> rather
> > > >> >> >> >>>>>>> than
> > > >> >> >> >>>>>>>>> trying to work around it.
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>>> There are lots of people interested in seeing 3.5
> > > stable,
> > > >> and
> > > >> >> if
> > > >> >> >> we
> > > >> >> >> >>>>>> get
> > > >> >> >> >>>>>>>>> everyone to contribute more patches and code
> reviews,
> > we
> > > >> >> should
> > > >> >> >> be
> > > >> >> >> >>>>>> able
> > > >> >> >> >>>>>>> to
> > > >> >> >> >>>>>>>>> do it sooner. After all, it is a community based
> > > effort, so
> > > >> >> the
> > > >> >> >> >>>>>>> community
> > > >> >> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>>> -Flavio
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> > > >> >> >> cnauroth@hortonworks.com>
> > > >> >> >> >>>>>>>>> wrote:
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>> Doug, I forgot to respond to your question about
> 3.4.
> > > >> Since
> > > >> >> >> 3.4 is
> > > >> >> >> >>>>>> the
> > > >> >> >> >>>>>>>>>> stable maintenance line, we are very conservative
> > about
> > > >> >> >> >>>>> back-porting
> > > >> >> >> >>>>>> to
> > > >> >> >> >>>>>>>>>> it.  Our policy is to limit back-ports to critical
> > bug
> > > >> fixes
> > > >> >> and
> > > >> >> >> >>>>> not
> > > >> >> >> >>>>>>>>>> introduce any new features in the 3.4 line.  This
> is
> > a
> > > >> >> matter of
> > > >> >> >> >>>>>>> managing
> > > >> >> >> >>>>>>>>>> risk.
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>> Jason, your question about release cadence is a
> fair
> > > >> one.  If
> > > >> >> >> it's
> > > >> >> >> >>>>>> any
> > > >> >> >> >>>>>>>>>> consolation, we are now taking the approach of
> trying
> > > to
> > > >> >> limit
> > > >> >> >> the
> > > >> >> >> >>>>>>> scope
> > > >> >> >> >>>>>>>>>> of anything new introduced in 3.5 too.  That would
> > > allow
> > > >> us
> > > >> >> to
> > > >> >> >> >>>>> focus
> > > >> >> >> >>>>>> on
> > > >> >> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing
> > > public
> > > >> >> >> APIs.  I
> > > >> >> >> >>>>>>> think
> > > >> >> >> >>>>>>>>>> this will help us accelerate the releases into beta
> > and
> > > >> >> eventual
> > > >> >> >> >>>>> GA.
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd
> like
> > > to
> > > >> hear
> > > >> >> >> >>>>>> thoughts
> > > >> >> >> >>>>>>>>>> from more experienced committers and PMC members
> > about
> > > >> your
> > > >> >> >> >>>>> proposal
> > > >> >> >> >>>>>> to
> > > >> >> >> >>>>>>>>>> try to cut a stable release for a limited subset of
> > > what
> > > >> is
> > > >> >> in
> > > >> >> >> >>>>>>> branch-3.5
> > > >> >> >> >>>>>>>>>> now.  My instinct is that it would be challenging
> to
> > > >> >> cherry-pick
> > > >> >> >> >>>>> out
> > > >> >> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This
> > > would
> > > >> >> become
> > > >> >> >> >>>>>>> another
> > > >> >> >> >>>>>>>>>> release line for an already resource-constrained
> > > volunteer
> > > >> >> >> staff to
> > > >> >> >> >>>>>>>>>> manage.  I'd prefer to dedicate those limited
> > > resources to
> > > >> >> >> overall
> > > >> >> >> >>>>>> 3.5
> > > >> >> >> >>>>>>>>>> stabilization.  Also, a 3.5 release in which
> certain
> > > >> features
> > > >> >> >> >>>>>>> "vanished"
> > > >> >> >> >>>>>>>>>> because of not meeting some stability criteria
> would
> > be
> > > >> >> >> >>>>> undesirable.
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>> --Chris Nauroth
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <
> > > jbr@squareup.com
> > > >> >
> > > >> >> >> wrote:
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>>> Chris,
> > > >> >> >> >>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more
> > > stable
> > > >> than
> > > >> >> >> >>>>> others
> > > >> >> >> >>>>>>>>> (e.g.
> > > >> >> >> >>>>>>>>>>> if we don't care about certain new features, is it
> > > >> >> relatively
> > > >> >> >> >>>>>> stable)?
> > > >> >> >> >>>>>>>>>>> Would it be possible to cut out a version that
> only
> > > has
> > > >> the
> > > >> >> >> bits
> > > >> >> >> >>>>> we
> > > >> >> >> >>>>>>>>> think
> > > >> >> >> >>>>>>>>>>> are stable (and release that)?
> > > >> >> >> >>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>> From that timeline, and the historic release
> > cadence,
> > > it
> > > >> >> would
> > > >> >> >> >>>>> seem
> > > >> >> >> >>>>>> to
> > > >> >> >> >>>>>>>>> be
> > > >> >> >> >>>>>>>>>>> a
> > > >> >> >> >>>>>>>>>>> years away before we get to the stable release?
> > > >> >> >> >>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>> Thanks,
> > > >> >> >> >>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>> Jason
> > > >> >> >> >>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> > > >> >> >> >>>>>>>>> cnauroth@hortonworks.com>
> > > >> >> >> >>>>>>>>>>> wrote:
> > > >> >> >> >>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>> Hello Doug,
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>> At this point, I think we're still pretty far
> away
> > > from
> > > >> >> >> >>>>> declaring a
> > > >> >> >> >>>>>>>>>>>> stable
> > > >> >> >> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're
> close
> > > >> enough
> > > >> >> >> that
> > > >> >> >> >>>>>>> anyone
> > > >> >> >> >>>>>>>>>>>> can
> > > >> >> >> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread
> > that
> > > >> >> >> describes
> > > >> >> >> >>>>> the
> > > >> >> >> >>>>>>>>>>>> high-level strategy for release planning in the
> 3.5
> > > >> line:
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>> https://s.apache.org/ADK1
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're
> > > working
> > > >> on
> > > >> >> >> >>>>>> resolving a
> > > >> >> >> >>>>>>>>>>>> few
> > > >> >> >> >>>>>>>>>>>> more blockers before we produce a release
> > candidate.
> > > >> >> >> Hopefully
> > > >> >> >> >>>>>> that
> > > >> >> >> >>>>>>>>>>>> will
> > > >> >> >> >>>>>>>>>>>> get done in the next few weeks.
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>> --Chris Nauroth
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <itsbehind@gmail.com
> >
> > > >> wrote:
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>> I know it's only been a few months, but I was
> > > >> wondering if
> > > >> >> >> there
> > > >> >> >> >>>>>>> was a
> > > >> >> >> >>>>>>>>>>>>> ballpark release date for a stable version of
> > 3.5.1.
> > > >> Or is
> > > >> >> >> there
> > > >> >> >> >>>>>> any
> > > >> >> >> >>>>>>>>>>>>> chance
> > > >> >> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just
> > > another
> > > >> >> person
> > > >> >> >> >>>>>> looking
> > > >> >> >> >>>>>>>>> to
> > > >> >> >> >>>>>>>>>>>>> have
> > > >> >> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all
> > you
> > > >> do!
> > > >> >> :)
> > > >> >> >> >>>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>> --
> > > >> >> >> >>>>>>>>>>>>> View this message in context:
> > > >> >> >> >>>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>
> > > >> >> >> >>>>>>
> > > >> >> >> >>>>>
> > > >> >> >>
> > > >> >>
> > > >>
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > > >> >> >> >>>>>>>>>>>> e
> > > >> >> >> >>>>>>>>>>>>> -tp7581744p7582136.html
> > > >> >> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list
> archive
> > at
> > > >> >> >> Nabble.com.
> > > >> >> >> >>>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>>>
> > > >> >> >> >>>>>>>>>>
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>>>
> > > >> >> >> >>>>>>>
> > > >> >> >> >>>>>>>
> > > >> >> >> >>>>>>
> > > >> >> >> >>>>>
> > > >> >> >> >
> > > >> >> >>
> > > >> >>
> > > >>
> > >
> >
>

Re: Zookeeper with SSL release date

[Jason Rosenberg <jb...@squareup.com>]

seems like an orthogonal requirement?

On Thu, Mar 24, 2016 at 3:37 PM, Alexander Shraer <sh...@gmail.com> wrote:

> How about a simpler alternative to the proposed flag for reconfig: a check
> in the code that requires ACLs to be set.
> If people want to use reconfig, they should use ACLs too.
>
> What do you think ?
>
> Alex
>
> On Mon, Mar 21, 2016 at 9:58 PM, Patrick Hunt <ph...@apache.org> wrote:
>
> > I would say if in doubt add a safety. (a config parameter to turn it
> > off). Cost is almost zero and worst case it will just give us peace of
> > mind. ;-)
> >
> > Patrick
> >
> > On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <sh...@gmail.com>
> > wrote:
> > > ok, thanks for the suggestion, I'll look into it. For reconfig I think
> > its
> > > pretty clear that its an admin
> > > functionality. I just always imagined that its controlled via acls,
> but I
> > > understand
> > > the concerns now.
> > >
> > > getConfig returns the dynamic config (list of all servers with all
> ports
> > > and quorum system if defined)
> > > and has an option to filter that info and just return the server
> > connection
> > > string (server and client port only).
> > >
> > >
> > > On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org>
> wrote:
> > >
> > >> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <sh...@gmail.com>
> > >> wrote:
> > >> > I don't think that getConfig should be an admin functionality. It is
> > >> > essential for client-side re-balancing
> > >> > that we implemented (all clients shoudl be able to detect
> > configuration
> > >> > changes via getConfig). It could
> > >> > be hidden somewhat by defining higher-level re-balancing
> > >> > policies (ZOOKEEPER-2016)
> > >> > but there hasn't been enough progress on that. Perhaps instead
> > getConfig
> > >> > should be controlled
> > >> > by a separate flag ?
> > >> >
> > >>
> > >> I believe that the Hadoop community has something we could use:
> > >>
> > >>
> >
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
> > >> (whether through annotations or just documenting it in the API
> javadoc)
> > >>
> > >> e.g. we could list getConfig as public/unstable for example and still
> > >> ship it as GA. That would mark it as something that could change re
> > >> API policy.
> > >>
> > >> Is the entire config exposed through getConfig? If so then we might
> > >> want to enable/disable it with a flag similar to reconfig. Might be
> > >> safer to just do that if we're not sure.
> > >>
> > >>
> > >> Re classification - we could do the same thing with reconfig, but I
> > >> think that would be a mistake. If we feel strongly where it should
> > >> live long term we should just move it now.
> > >>
> > >> Patrick
> > >>
> > >> >
> > >> > On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org>
> > wrote:
> > >> >
> > >> >> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <
> shralex@gmail.com
> > >
> > >> >> wrote:
> > >> >> > Hi Patrick, Flavio,
> > >> >> >
> > >> >> > Since there seems to be consensus on this, I can add this flag,
> > unless
> > >> >> > someone else wants to. I assume that getConfig should still work
> > >> >> regardless
> > >> >> > of the flag ? is there a security concern with clients knowing
> the
> > >> list
> > >> >> of
> > >> >> > servers?
> > >> >> >
> > >> >>
> > >> >> We've always hidden that detail from users. We don't even expose
> > which
> > >> >> server you're connected to today. I remember Ben (and perhaps
> > Flavio?)
> > >> >> highlighting this as important to maintain although I'm not super
> > >> >> familiar with the specifics on why. It made sense to me though from
> > >> >> the perspective that we don't want clients to make assumptions that
> > >> >> probably shouldn't.
> > >> >>
> > >> >> My thinking is that we should 1) add a config option to enable
> > >> >> reconfig (off by default), 2) move reconfig specific functionality
> > >> >> from ZooKeeper.java (including getconfig) into an "admin" package,
> > >> >> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
> > >> >> when folks do want to enable reconfig and are also worried about
> > auth.
> > >> >> (e.g. turn on kerb)
> > >> >>
> > >> >> Again, I don't see any of this as a quality issue personally. As
> such
> > >> >> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if
> we
> > >> >> were interested in doing such a release. Adjusting the API should
> be
> > >> >> done before we move to "beta" though. Although that seems like a
> > >> >> pretty mechanical (eclipse/idea) type refactoring?
> > >> >>
> > >> >> Patrick
> > >> >>
> > >> >> > Cheers,
> > >> >> > Alex
> > >> >> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org>
> wrote:
> > >> >> >
> > >> >> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <
> fpj@apache.org
> > >
> > >> >> wrote:
> > >> >> >> > I gotta say that I'm not super excited about this option, but
> > for
> > >> some
> > >> >> >> reason I ended up carrying the flag. To recap, I just raised
> this
> > >> option
> > >> >> >> because it seems that there are folks interested in features in
> > 3.5
> > >> like
> > >> >> >> SSL and not necessarily in reconfiguration. SSL is important and
> > to
> > >> take
> > >> >> >> Kafka as an example, it sucks that we can't have a whole set up
> > using
> > >> >> SSL.
> > >> >> >> For ZK, the real questions are:
> > >> >> >> >
> > >> >> >> > 1- how fast can we make 3.5 stable?
> > >> >> >> > 2- would it be faster if we have a way of disabling
> > >> reconfiguration?
> > >> >> >> > 3- would enough users care about a stable 3.5 that has
> > >> reconfiguration
> > >> >> >> disabled?
> > >> >> >> >
> > >> >> >> > It is taking a long time to get 3.5 to beta. There has been
> some
> > >> good
> > >> >> >> activity around 3.5.2 release, which is a great step, but it is
> > >> unclear
> > >> >> >> when 3.5.3 is going to come and if we will be able to make 3.5
> > beta
> > >> >> then.
> > >> >> >> Frankly, disabling reconfiguration sounds undesirable because it
> > is
> > >> an
> > >> >> >> important feature, but I currently don't use it in production,
> so
> > >> from a
> > >> >> >> practical point of view, I can go both ways. Whether we go
> through
> > >> the
> > >> >> >> trouble of doing 2 depends on users interested in that option
> and
> > >> folks
> > >> >> >> willing to implement it.
> > >> >> >> >
> > >> >> >> > To answer your question, Powell, my pseudo-proposal is kind
> of a
> > >> funny
> > >> >> >> option because once the feature is stable, then we wouldn't
> need a
> > >> >> switch
> > >> >> >> any longer, so there is not need of a deprecation path, we just
> > start
> > >> >> >> ignoring it from the first beta release. Until it is beta, I'd
> say
> > >> that
> > >> >> >> default is disabled.
> > >> >> >>
> > >> >> >> I would argue that we need this even when it does become stable.
> > To
> > >> me
> > >> >> >> this is not a quality issue so much as it is an auth issue. We
> > want
> > >> to
> > >> >> >> make it simple for folks to run a vanilla/old ZK cluster and not
> > >> worry
> > >> >> >> about the security implications of having reconfig enabled.
> > >> >> >>
> > >> >> >> Patrick
> > >> >> >>
> > >> >> >> >
> > >> >> >> > -Flavio
> > >> >> >> >
> > >> >> >> >> On 17 Mar 2016, at 17:44, powell molleti
> > >> <powellm79@yahoo.com.INVALID
> > >> >> >
> > >> >> >> wrote:
> > >> >> >> >>
> > >> >> >> >> Hi Flavio,
> > >> >> >> >>
> > >> >> >> >> Generally do config options and command line args come under
> > the
> > >> same
> > >> >> >> SLA as API?. I was assuming as such hence my question. Perhaps
> if
> > the
> > >> >> >> expectation is that this config option is temporary from get go
> > then
> > >> >> may be
> > >> >> >> it is ok. The default for re-config support will be enabled or
> > >> >> disabled?.
> > >> >> >> >>
> > >> >> >> >> I am just thinking from provisioning point of view when
> people
> > >> >> generate
> > >> >> >> config options etc.
> > >> >> >> >>
> > >> >> >> >> Thanks
> > >> >> >> >> Powell.
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
> > >> >> fpj@apache.org>
> > >> >> >> wrote:
> > >> >> >> >> Hi Powell,
> > >> >> >> >>
> > >> >> >> >> I was thinking config file and system property server side.
> > What's
> > >> >> your
> > >> >> >> concern with compatibility? The API itself wouldn't change, but
> > the
> > >> >> config
> > >> >> >> option wouldn't exist in previous versions and would not exist
> > >> either in
> > >> >> >> later stable versions of 3.5.
> > >> >> >> >>
> > >> >> >> >> -Flavio
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >>> On 16 Mar 2016, at 22:08, powell molleti
> > >> >> <po...@yahoo.com.INVALID>
> > >> >> >> wrote:
> > >> >> >> >>>
> > >> >> >> >>> Will this option be supplied via config file/args/API?. Will
> > this
> > >> >> >> option be a temporary thing i.e what about its compatibility?.
> > >> >> >> >>>
> > >> >> >> >>> thanks
> > >> >> >> >>> Powell.
> > >> >> >> >>>
> > >> >> >> >>>
> > >> >> >> >>>
> > >> >> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
> > >> >> fpj@apache.org>
> > >> >> >> wrote:
> > >> >> >> >>> The main issue to sort out is stability of the API. There
> is a
> > >> >> >> security concern around the fact that clients can freely
> > reconfigure
> > >> the
> > >> >> >> ensemble. If we follow the plan that Pat proposed some time ago:
> > >> >> >> >>>
> > >> >> >> >>>
> > >> >> >>
> > >> >>
> > >>
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > >> >> >> <
> > >> >> >>
> > >> >>
> > >>
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > >> >> >> >
> > >> >> >> >>>
> > >> >> >> >>> Locking the API is the main step to move it to beta. Sorting
> > out
> > >> >> bugs
> > >> >> >> is definitely necessary, but it isn't the main thing that is
> > keeping
> > >> >> 3.5 in
> > >> >> >> alpha.
> > >> >> >> >>>
> > >> >> >> >>> About making it experimental, I was raising the option of
> > having
> > >> a
> > >> >> >> switch that disables the API calls, not the code. The reason why
> > that
> > >> >> could
> > >> >> >> work is that anyone using 3.5 who uses the "experimental" API
> must
> > >> >> explicit
> > >> >> >> turn on the switch and enable the calls. If they do it, they
> need
> > to
> > >> be
> > >> >> >> aware that the API can change.
> > >> >> >> >>>
> > >> >> >> >>> I must say that I haven't really looked closely into doing
> it,
> > >> and
> > >> >> I'm
> > >> >> >> not even entirely convinced that this is a good idea, but since
> > Jason
> > >> >> >> raised the point, I'm exploring options.
> > >> >> >> >>>
> > >> >> >> >>> -Flavio
> > >> >> >> >>>
> > >> >> >> >>>
> > >> >> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <
> > shralex@gmail.com>
> > >> >> wrote:
> > >> >> >> >>>>
> > >> >> >> >>>> Looking at the list of ~50 blocker and critical bugs in
> > >> ZooKeeper,
> > >> >> >> only 3-4
> > >> >> >> >>>> are related to reconfig. Given this, and the fact that it
> is
> > >> run in
> > >> >> >> >>>> production since 2012 in multiple companies, I don't think
> > its
> > >> more
> > >> >> >> >>>> unstable than any other part of ZooKeeper.
> > >> >> >> >>>>
> > >> >> >> >>>> There are multiple reconfig-related bugs that turned out
> > really
> > >> >> >> difficult
> > >> >> >> >>>> to debug without access to the actual system or at least to
> > the
> > >> >> Hudson
> > >> >> >> >>>> machines where some tests are failing. In the past, Michi
> > and I
> > >> >> >> physically
> > >> >> >> >>>> went to Hortonworks to debug one such issue, but this is of
> > >> course
> > >> >> >> not a
> > >> >> >> >>>> good method, and we weren't able to arrange such a visit
> > again.
> > >> >> >> >>>>
> > >> >> >> >>>> Regarding making it optional - the reconfig logic has
> several
> > >> >> >> different
> > >> >> >> >>>> parts, some would be really difficult to disable using a
> > >> >> configuration
> > >> >> >> >>>> parameter. But the actual dynamic expansion of the cluster
> is
> > >> >> >> triggered by
> > >> >> >> >>>> the reconfig command, so it should not affect users who
> don't
> > >> >> invoke
> > >> >> >> it.
> > >> >> >> >>>>
> > >> >> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
> > >> >> fpj@apache.org>
> > >> >> >> wrote:
> > >> >> >> >>>>
> > >> >> >> >>>>> I suppose we could give it a try. How do other folks feel
> > about
> > >> >> it?
> > >> >> >> >>>>>
> > >> >> >> >>>>> -Flavio
> > >> >> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jbr@squareup.com
> >
> > >> wrote:
> > >> >> >> >>>>>
> > >> >> >> >>>>>> So, you could enable the dynamic reconfiguration feature
> > >> behind a
> > >> >> >> config
> > >> >> >> >>>>>> option, and document that it should only be enabled
> > >> >> experimentally,
> > >> >> >> use
> > >> >> >> >>>>> at
> > >> >> >> >>>>>> your own risk.  Keep it off by default.  Allow only
> static
> > >> >> config by
> > >> >> >> >>>>>> default, until it's stable?
> > >> >> >> >>>>>>
> > >> >> >> >>>>>> Jason
> > >> >> >> >>>>>>
> > >> >> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
> > >> >> fpj@apache.org>
> > >> >> >> >>>>> wrote:
> > >> >> >> >>>>>>
> > >> >> >> >>>>>>> Hi Jason,
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>> The consumer in Kafka is pretty independent from the
> core
> > >> >> >> (brokers),
> > >> >> >> >>>>>>> that's how that project manages to make such a
> > separation. We
> > >> >> don't
> > >> >> >> >>>>> have
> > >> >> >> >>>>>>> the same with ZooKeeper as the feature we are talking
> > about
> > >> is
> > >> >> >> part of
> > >> >> >> >>>>>> the
> > >> >> >> >>>>>>> server and the only way I see of doing what you say is
> to
> > >> turn
> > >> >> off
> > >> >> >> >>>>>>> features. More specifically, we'd need to disable the
> > >> reconfig
> > >> >> API
> > >> >> >> and
> > >> >> >> >>>>> do
> > >> >> >> >>>>>>> not allow any change to the configuration, even though
> the
> > >> code
> > >> >> is
> > >> >> >> >>>>> there.
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>> Reconfig here refers to the ability of changing the
> > >> >> configuration
> > >> >> >> of an
> > >> >> >> >>>>>>> ensemble (e.g., changing the set of servers).
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>> -Flavio
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <
> > jbr@squareup.com
> > >> >
> > >> >> >> wrote:
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>> So, it would seem sensible to me to have a release
> where
> > all
> > >> >> >> features
> > >> >> >> >>>>>> are
> > >> >> >> >>>>>>>> stable, except where noted.  E.g. mark certain features
> > as
> > >> only
> > >> >> >> >>>>> 'alpha
> > >> >> >> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's
> > >> >> possible
> > >> >> >> to
> > >> >> >> >>>>>>> happily
> > >> >> >> >>>>>>>> use 3.5.X without exercising the unstable re-config
> > bits?).
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>> There's precedent for doing this sort of thing in other
> > >> >> projects,
> > >> >> >> >>>>> e.g.
> > >> >> >> >>>>>> in
> > >> >> >> >>>>>>>> Kafka, they've had several release where a new
> "Consumer
> > >> API"
> > >> >> is
> > >> >> >> >>>>>> shipped
> > >> >> >> >>>>>>>> that is available for beta-testing, but you can still
> > just
> > >> use
> > >> >> the
> > >> >> >> >>>>>> older
> > >> >> >> >>>>>>>> stable consumer api, etc.
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>> Jason
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > >> >> >> >>>>>>> <powellm79@yahoo.com.invalid
> > >> >> >> >>>>>>>>> wrote:
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>>> Hi Doug,
> > >> >> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from
> using
> > >> it?.
> > >> >> Or
> > >> >> >> have
> > >> >> >> >>>>>> you
> > >> >> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5
> > being
> > >> >> >> labeled as
> > >> >> >> >>>>>>> alpha
> > >> >> >> >>>>>>>>> might not be fair, since it is far more stable then
> what
> > >> most
> > >> >> >> people
> > >> >> >> >>>>>>>>> associate an alpha release to be.
> > >> >> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it
> > will
> > >> >> just
> > >> >> >> work
> > >> >> >> >>>>>> for
> > >> >> >> >>>>>>>>> you?.
> > >> >> >> >>>>>>>>> There are many examples of 3.5.X being used in
> > productions
> > >> >> from
> > >> >> >> my
> > >> >> >> >>>>>>> limited
> > >> >> >> >>>>>>>>> knowledge.
> > >> >> >> >>>>>>>>> ThanksPowell.
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio
> Junqueira <
> > >> >> >> >>>>>>> fpj@apache.org>
> > >> >> >> >>>>>>>>> wrote:
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>> None of us expected the reconfig changes to take this
> > long
> > >> to
> > >> >> >> >>>>>> stabilize.
> > >> >> >> >>>>>>>>> Until we get there, I don't think we can do anything
> > else
> > >> with
> > >> >> >> 3.5.
> > >> >> >> >>>>>> The
> > >> >> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a
> > >> stable
> > >> >> >> rather
> > >> >> >> >>>>>>> than
> > >> >> >> >>>>>>>>> trying to work around it.
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>> There are lots of people interested in seeing 3.5
> > stable,
> > >> and
> > >> >> if
> > >> >> >> we
> > >> >> >> >>>>>> get
> > >> >> >> >>>>>>>>> everyone to contribute more patches and code reviews,
> we
> > >> >> should
> > >> >> >> be
> > >> >> >> >>>>>> able
> > >> >> >> >>>>>>> to
> > >> >> >> >>>>>>>>> do it sooner. After all, it is a community based
> > effort, so
> > >> >> the
> > >> >> >> >>>>>>> community
> > >> >> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>> -Flavio
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> > >> >> >> cnauroth@hortonworks.com>
> > >> >> >> >>>>>>>>> wrote:
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.
> > >> Since
> > >> >> >> 3.4 is
> > >> >> >> >>>>>> the
> > >> >> >> >>>>>>>>>> stable maintenance line, we are very conservative
> about
> > >> >> >> >>>>> back-porting
> > >> >> >> >>>>>> to
> > >> >> >> >>>>>>>>>> it.  Our policy is to limit back-ports to critical
> bug
> > >> fixes
> > >> >> and
> > >> >> >> >>>>> not
> > >> >> >> >>>>>>>>>> introduce any new features in the 3.4 line.  This is
> a
> > >> >> matter of
> > >> >> >> >>>>>>> managing
> > >> >> >> >>>>>>>>>> risk.
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>> Jason, your question about release cadence is a fair
> > >> one.  If
> > >> >> >> it's
> > >> >> >> >>>>>> any
> > >> >> >> >>>>>>>>>> consolation, we are now taking the approach of trying
> > to
> > >> >> limit
> > >> >> >> the
> > >> >> >> >>>>>>> scope
> > >> >> >> >>>>>>>>>> of anything new introduced in 3.5 too.  That would
> > allow
> > >> us
> > >> >> to
> > >> >> >> >>>>> focus
> > >> >> >> >>>>>> on
> > >> >> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing
> > public
> > >> >> >> APIs.  I
> > >> >> >> >>>>>>> think
> > >> >> >> >>>>>>>>>> this will help us accelerate the releases into beta
> and
> > >> >> eventual
> > >> >> >> >>>>> GA.
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like
> > to
> > >> hear
> > >> >> >> >>>>>> thoughts
> > >> >> >> >>>>>>>>>> from more experienced committers and PMC members
> about
> > >> your
> > >> >> >> >>>>> proposal
> > >> >> >> >>>>>> to
> > >> >> >> >>>>>>>>>> try to cut a stable release for a limited subset of
> > what
> > >> is
> > >> >> in
> > >> >> >> >>>>>>> branch-3.5
> > >> >> >> >>>>>>>>>> now.  My instinct is that it would be challenging to
> > >> >> cherry-pick
> > >> >> >> >>>>> out
> > >> >> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This
> > would
> > >> >> become
> > >> >> >> >>>>>>> another
> > >> >> >> >>>>>>>>>> release line for an already resource-constrained
> > volunteer
> > >> >> >> staff to
> > >> >> >> >>>>>>>>>> manage.  I'd prefer to dedicate those limited
> > resources to
> > >> >> >> overall
> > >> >> >> >>>>>> 3.5
> > >> >> >> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain
> > >> features
> > >> >> >> >>>>>>> "vanished"
> > >> >> >> >>>>>>>>>> because of not meeting some stability criteria would
> be
> > >> >> >> >>>>> undesirable.
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>> --Chris Nauroth
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <
> > jbr@squareup.com
> > >> >
> > >> >> >> wrote:
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>>> Chris,
> > >> >> >> >>>>>>>>>>>
> > >> >> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more
> > stable
> > >> than
> > >> >> >> >>>>> others
> > >> >> >> >>>>>>>>> (e.g.
> > >> >> >> >>>>>>>>>>> if we don't care about certain new features, is it
> > >> >> relatively
> > >> >> >> >>>>>> stable)?
> > >> >> >> >>>>>>>>>>> Would it be possible to cut out a version that only
> > has
> > >> the
> > >> >> >> bits
> > >> >> >> >>>>> we
> > >> >> >> >>>>>>>>> think
> > >> >> >> >>>>>>>>>>> are stable (and release that)?
> > >> >> >> >>>>>>>>>>>
> > >> >> >> >>>>>>>>>>> From that timeline, and the historic release
> cadence,
> > it
> > >> >> would
> > >> >> >> >>>>> seem
> > >> >> >> >>>>>> to
> > >> >> >> >>>>>>>>> be
> > >> >> >> >>>>>>>>>>> a
> > >> >> >> >>>>>>>>>>> years away before we get to the stable release?
> > >> >> >> >>>>>>>>>>>
> > >> >> >> >>>>>>>>>>> Thanks,
> > >> >> >> >>>>>>>>>>>
> > >> >> >> >>>>>>>>>>> Jason
> > >> >> >> >>>>>>>>>>>
> > >> >> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> > >> >> >> >>>>>>>>> cnauroth@hortonworks.com>
> > >> >> >> >>>>>>>>>>> wrote:
> > >> >> >> >>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>> Hello Doug,
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>> At this point, I think we're still pretty far away
> > from
> > >> >> >> >>>>> declaring a
> > >> >> >> >>>>>>>>>>>> stable
> > >> >> >> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close
> > >> enough
> > >> >> >> that
> > >> >> >> >>>>>>> anyone
> > >> >> >> >>>>>>>>>>>> can
> > >> >> >> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread
> that
> > >> >> >> describes
> > >> >> >> >>>>> the
> > >> >> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5
> > >> line:
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>> https://s.apache.org/ADK1
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're
> > working
> > >> on
> > >> >> >> >>>>>> resolving a
> > >> >> >> >>>>>>>>>>>> few
> > >> >> >> >>>>>>>>>>>> more blockers before we produce a release
> candidate.
> > >> >> >> Hopefully
> > >> >> >> >>>>>> that
> > >> >> >> >>>>>>>>>>>> will
> > >> >> >> >>>>>>>>>>>> get done in the next few weeks.
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>> --Chris Nauroth
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com>
> > >> wrote:
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>> I know it's only been a few months, but I was
> > >> wondering if
> > >> >> >> there
> > >> >> >> >>>>>>> was a
> > >> >> >> >>>>>>>>>>>>> ballpark release date for a stable version of
> 3.5.1.
> > >> Or is
> > >> >> >> there
> > >> >> >> >>>>>> any
> > >> >> >> >>>>>>>>>>>>> chance
> > >> >> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just
> > another
> > >> >> person
> > >> >> >> >>>>>> looking
> > >> >> >> >>>>>>>>> to
> > >> >> >> >>>>>>>>>>>>> have
> > >> >> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all
> you
> > >> do!
> > >> >> :)
> > >> >> >> >>>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>> --
> > >> >> >> >>>>>>>>>>>>> View this message in context:
> > >> >> >> >>>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>
> > >> >> >> >>>>>
> > >> >> >>
> > >> >>
> > >>
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > >> >> >> >>>>>>>>>>>> e
> > >> >> >> >>>>>>>>>>>>> -tp7581744p7582136.html
> > >> >> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive
> at
> > >> >> >> Nabble.com.
> > >> >> >> >>>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>>>
> > >> >> >> >>>>>>>>>>
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>
> > >> >> >> >>>>>
> > >> >> >> >
> > >> >> >>
> > >> >>
> > >>
> >
>

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

How about a simpler alternative to the proposed flag for reconfig: a check
in the code that requires ACLs to be set.
If people want to use reconfig, they should use ACLs too.

What do you think ?

Alex

On Mon, Mar 21, 2016 at 9:58 PM, Patrick Hunt <ph...@apache.org> wrote:

> I would say if in doubt add a safety. (a config parameter to turn it
> off). Cost is almost zero and worst case it will just give us peace of
> mind. ;-)
>
> Patrick
>
> On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <sh...@gmail.com>
> wrote:
> > ok, thanks for the suggestion, I'll look into it. For reconfig I think
> its
> > pretty clear that its an admin
> > functionality. I just always imagined that its controlled via acls, but I
> > understand
> > the concerns now.
> >
> > getConfig returns the dynamic config (list of all servers with all ports
> > and quorum system if defined)
> > and has an option to filter that info and just return the server
> connection
> > string (server and client port only).
> >
> >
> > On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org> wrote:
> >
> >> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <sh...@gmail.com>
> >> wrote:
> >> > I don't think that getConfig should be an admin functionality. It is
> >> > essential for client-side re-balancing
> >> > that we implemented (all clients shoudl be able to detect
> configuration
> >> > changes via getConfig). It could
> >> > be hidden somewhat by defining higher-level re-balancing
> >> > policies (ZOOKEEPER-2016)
> >> > but there hasn't been enough progress on that. Perhaps instead
> getConfig
> >> > should be controlled
> >> > by a separate flag ?
> >> >
> >>
> >> I believe that the Hadoop community has something we could use:
> >>
> >>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
> >> (whether through annotations or just documenting it in the API javadoc)
> >>
> >> e.g. we could list getConfig as public/unstable for example and still
> >> ship it as GA. That would mark it as something that could change re
> >> API policy.
> >>
> >> Is the entire config exposed through getConfig? If so then we might
> >> want to enable/disable it with a flag similar to reconfig. Might be
> >> safer to just do that if we're not sure.
> >>
> >>
> >> Re classification - we could do the same thing with reconfig, but I
> >> think that would be a mistake. If we feel strongly where it should
> >> live long term we should just move it now.
> >>
> >> Patrick
> >>
> >> >
> >> > On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org>
> wrote:
> >> >
> >> >> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <shralex@gmail.com
> >
> >> >> wrote:
> >> >> > Hi Patrick, Flavio,
> >> >> >
> >> >> > Since there seems to be consensus on this, I can add this flag,
> unless
> >> >> > someone else wants to. I assume that getConfig should still work
> >> >> regardless
> >> >> > of the flag ? is there a security concern with clients knowing the
> >> list
> >> >> of
> >> >> > servers?
> >> >> >
> >> >>
> >> >> We've always hidden that detail from users. We don't even expose
> which
> >> >> server you're connected to today. I remember Ben (and perhaps
> Flavio?)
> >> >> highlighting this as important to maintain although I'm not super
> >> >> familiar with the specifics on why. It made sense to me though from
> >> >> the perspective that we don't want clients to make assumptions that
> >> >> probably shouldn't.
> >> >>
> >> >> My thinking is that we should 1) add a config option to enable
> >> >> reconfig (off by default), 2) move reconfig specific functionality
> >> >> from ZooKeeper.java (including getconfig) into an "admin" package,
> >> >> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
> >> >> when folks do want to enable reconfig and are also worried about
> auth.
> >> >> (e.g. turn on kerb)
> >> >>
> >> >> Again, I don't see any of this as a quality issue personally. As such
> >> >> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
> >> >> were interested in doing such a release. Adjusting the API should be
> >> >> done before we move to "beta" though. Although that seems like a
> >> >> pretty mechanical (eclipse/idea) type refactoring?
> >> >>
> >> >> Patrick
> >> >>
> >> >> > Cheers,
> >> >> > Alex
> >> >> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org> wrote:
> >> >> >
> >> >> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fpj@apache.org
> >
> >> >> wrote:
> >> >> >> > I gotta say that I'm not super excited about this option, but
> for
> >> some
> >> >> >> reason I ended up carrying the flag. To recap, I just raised this
> >> option
> >> >> >> because it seems that there are folks interested in features in
> 3.5
> >> like
> >> >> >> SSL and not necessarily in reconfiguration. SSL is important and
> to
> >> take
> >> >> >> Kafka as an example, it sucks that we can't have a whole set up
> using
> >> >> SSL.
> >> >> >> For ZK, the real questions are:
> >> >> >> >
> >> >> >> > 1- how fast can we make 3.5 stable?
> >> >> >> > 2- would it be faster if we have a way of disabling
> >> reconfiguration?
> >> >> >> > 3- would enough users care about a stable 3.5 that has
> >> reconfiguration
> >> >> >> disabled?
> >> >> >> >
> >> >> >> > It is taking a long time to get 3.5 to beta. There has been some
> >> good
> >> >> >> activity around 3.5.2 release, which is a great step, but it is
> >> unclear
> >> >> >> when 3.5.3 is going to come and if we will be able to make 3.5
> beta
> >> >> then.
> >> >> >> Frankly, disabling reconfiguration sounds undesirable because it
> is
> >> an
> >> >> >> important feature, but I currently don't use it in production, so
> >> from a
> >> >> >> practical point of view, I can go both ways. Whether we go through
> >> the
> >> >> >> trouble of doing 2 depends on users interested in that option and
> >> folks
> >> >> >> willing to implement it.
> >> >> >> >
> >> >> >> > To answer your question, Powell, my pseudo-proposal is kind of a
> >> funny
> >> >> >> option because once the feature is stable, then we wouldn't need a
> >> >> switch
> >> >> >> any longer, so there is not need of a deprecation path, we just
> start
> >> >> >> ignoring it from the first beta release. Until it is beta, I'd say
> >> that
> >> >> >> default is disabled.
> >> >> >>
> >> >> >> I would argue that we need this even when it does become stable.
> To
> >> me
> >> >> >> this is not a quality issue so much as it is an auth issue. We
> want
> >> to
> >> >> >> make it simple for folks to run a vanilla/old ZK cluster and not
> >> worry
> >> >> >> about the security implications of having reconfig enabled.
> >> >> >>
> >> >> >> Patrick
> >> >> >>
> >> >> >> >
> >> >> >> > -Flavio
> >> >> >> >
> >> >> >> >> On 17 Mar 2016, at 17:44, powell molleti
> >> <powellm79@yahoo.com.INVALID
> >> >> >
> >> >> >> wrote:
> >> >> >> >>
> >> >> >> >> Hi Flavio,
> >> >> >> >>
> >> >> >> >> Generally do config options and command line args come under
> the
> >> same
> >> >> >> SLA as API?. I was assuming as such hence my question. Perhaps if
> the
> >> >> >> expectation is that this config option is temporary from get go
> then
> >> >> may be
> >> >> >> it is ok. The default for re-config support will be enabled or
> >> >> disabled?.
> >> >> >> >>
> >> >> >> >> I am just thinking from provisioning point of view when people
> >> >> generate
> >> >> >> config options etc.
> >> >> >> >>
> >> >> >> >> Thanks
> >> >> >> >> Powell.
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
> >> >> fpj@apache.org>
> >> >> >> wrote:
> >> >> >> >> Hi Powell,
> >> >> >> >>
> >> >> >> >> I was thinking config file and system property server side.
> What's
> >> >> your
> >> >> >> concern with compatibility? The API itself wouldn't change, but
> the
> >> >> config
> >> >> >> option wouldn't exist in previous versions and would not exist
> >> either in
> >> >> >> later stable versions of 3.5.
> >> >> >> >>
> >> >> >> >> -Flavio
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>> On 16 Mar 2016, at 22:08, powell molleti
> >> >> <po...@yahoo.com.INVALID>
> >> >> >> wrote:
> >> >> >> >>>
> >> >> >> >>> Will this option be supplied via config file/args/API?. Will
> this
> >> >> >> option be a temporary thing i.e what about its compatibility?.
> >> >> >> >>>
> >> >> >> >>> thanks
> >> >> >> >>> Powell.
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
> >> >> fpj@apache.org>
> >> >> >> wrote:
> >> >> >> >>> The main issue to sort out is stability of the API. There is a
> >> >> >> security concern around the fact that clients can freely
> reconfigure
> >> the
> >> >> >> ensemble. If we follow the plan that Pat proposed some time ago:
> >> >> >> >>>
> >> >> >> >>>
> >> >> >>
> >> >>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >> >> >> <
> >> >> >>
> >> >>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >> >> >> >
> >> >> >> >>>
> >> >> >> >>> Locking the API is the main step to move it to beta. Sorting
> out
> >> >> bugs
> >> >> >> is definitely necessary, but it isn't the main thing that is
> keeping
> >> >> 3.5 in
> >> >> >> alpha.
> >> >> >> >>>
> >> >> >> >>> About making it experimental, I was raising the option of
> having
> >> a
> >> >> >> switch that disables the API calls, not the code. The reason why
> that
> >> >> could
> >> >> >> work is that anyone using 3.5 who uses the "experimental" API must
> >> >> explicit
> >> >> >> turn on the switch and enable the calls. If they do it, they need
> to
> >> be
> >> >> >> aware that the API can change.
> >> >> >> >>>
> >> >> >> >>> I must say that I haven't really looked closely into doing it,
> >> and
> >> >> I'm
> >> >> >> not even entirely convinced that this is a good idea, but since
> Jason
> >> >> >> raised the point, I'm exploring options.
> >> >> >> >>>
> >> >> >> >>> -Flavio
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <
> shralex@gmail.com>
> >> >> wrote:
> >> >> >> >>>>
> >> >> >> >>>> Looking at the list of ~50 blocker and critical bugs in
> >> ZooKeeper,
> >> >> >> only 3-4
> >> >> >> >>>> are related to reconfig. Given this, and the fact that it is
> >> run in
> >> >> >> >>>> production since 2012 in multiple companies, I don't think
> its
> >> more
> >> >> >> >>>> unstable than any other part of ZooKeeper.
> >> >> >> >>>>
> >> >> >> >>>> There are multiple reconfig-related bugs that turned out
> really
> >> >> >> difficult
> >> >> >> >>>> to debug without access to the actual system or at least to
> the
> >> >> Hudson
> >> >> >> >>>> machines where some tests are failing. In the past, Michi
> and I
> >> >> >> physically
> >> >> >> >>>> went to Hortonworks to debug one such issue, but this is of
> >> course
> >> >> >> not a
> >> >> >> >>>> good method, and we weren't able to arrange such a visit
> again.
> >> >> >> >>>>
> >> >> >> >>>> Regarding making it optional - the reconfig logic has several
> >> >> >> different
> >> >> >> >>>> parts, some would be really difficult to disable using a
> >> >> configuration
> >> >> >> >>>> parameter. But the actual dynamic expansion of the cluster is
> >> >> >> triggered by
> >> >> >> >>>> the reconfig command, so it should not affect users who don't
> >> >> invoke
> >> >> >> it.
> >> >> >> >>>>
> >> >> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
> >> >> fpj@apache.org>
> >> >> >> wrote:
> >> >> >> >>>>
> >> >> >> >>>>> I suppose we could give it a try. How do other folks feel
> about
> >> >> it?
> >> >> >> >>>>>
> >> >> >> >>>>> -Flavio
> >> >> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com>
> >> wrote:
> >> >> >> >>>>>
> >> >> >> >>>>>> So, you could enable the dynamic reconfiguration feature
> >> behind a
> >> >> >> config
> >> >> >> >>>>>> option, and document that it should only be enabled
> >> >> experimentally,
> >> >> >> use
> >> >> >> >>>>> at
> >> >> >> >>>>>> your own risk.  Keep it off by default.  Allow only static
> >> >> config by
> >> >> >> >>>>>> default, until it's stable?
> >> >> >> >>>>>>
> >> >> >> >>>>>> Jason
> >> >> >> >>>>>>
> >> >> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
> >> >> fpj@apache.org>
> >> >> >> >>>>> wrote:
> >> >> >> >>>>>>
> >> >> >> >>>>>>> Hi Jason,
> >> >> >> >>>>>>>
> >> >> >> >>>>>>> The consumer in Kafka is pretty independent from the core
> >> >> >> (brokers),
> >> >> >> >>>>>>> that's how that project manages to make such a
> separation. We
> >> >> don't
> >> >> >> >>>>> have
> >> >> >> >>>>>>> the same with ZooKeeper as the feature we are talking
> about
> >> is
> >> >> >> part of
> >> >> >> >>>>>> the
> >> >> >> >>>>>>> server and the only way I see of doing what you say is to
> >> turn
> >> >> off
> >> >> >> >>>>>>> features. More specifically, we'd need to disable the
> >> reconfig
> >> >> API
> >> >> >> and
> >> >> >> >>>>> do
> >> >> >> >>>>>>> not allow any change to the configuration, even though the
> >> code
> >> >> is
> >> >> >> >>>>> there.
> >> >> >> >>>>>>>
> >> >> >> >>>>>>> Reconfig here refers to the ability of changing the
> >> >> configuration
> >> >> >> of an
> >> >> >> >>>>>>> ensemble (e.g., changing the set of servers).
> >> >> >> >>>>>>>
> >> >> >> >>>>>>> -Flavio
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <
> jbr@squareup.com
> >> >
> >> >> >> wrote:
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>> So, it would seem sensible to me to have a release where
> all
> >> >> >> features
> >> >> >> >>>>>> are
> >> >> >> >>>>>>>> stable, except where noted.  E.g. mark certain features
> as
> >> only
> >> >> >> >>>>> 'alpha
> >> >> >> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's
> >> >> possible
> >> >> >> to
> >> >> >> >>>>>>> happily
> >> >> >> >>>>>>>> use 3.5.X without exercising the unstable re-config
> bits?).
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>> There's precedent for doing this sort of thing in other
> >> >> projects,
> >> >> >> >>>>> e.g.
> >> >> >> >>>>>> in
> >> >> >> >>>>>>>> Kafka, they've had several release where a new "Consumer
> >> API"
> >> >> is
> >> >> >> >>>>>> shipped
> >> >> >> >>>>>>>> that is available for beta-testing, but you can still
> just
> >> use
> >> >> the
> >> >> >> >>>>>> older
> >> >> >> >>>>>>>> stable consumer api, etc.
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>> Jason
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> >> >> >> >>>>>>> <powellm79@yahoo.com.invalid
> >> >> >> >>>>>>>>> wrote:
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>>> Hi Doug,
> >> >> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using
> >> it?.
> >> >> Or
> >> >> >> have
> >> >> >> >>>>>> you
> >> >> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5
> being
> >> >> >> labeled as
> >> >> >> >>>>>>> alpha
> >> >> >> >>>>>>>>> might not be fair, since it is far more stable then what
> >> most
> >> >> >> people
> >> >> >> >>>>>>>>> associate an alpha release to be.
> >> >> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it
> will
> >> >> just
> >> >> >> work
> >> >> >> >>>>>> for
> >> >> >> >>>>>>>>> you?.
> >> >> >> >>>>>>>>> There are many examples of 3.5.X being used in
> productions
> >> >> from
> >> >> >> my
> >> >> >> >>>>>>> limited
> >> >> >> >>>>>>>>> knowledge.
> >> >> >> >>>>>>>>> ThanksPowell.
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> >> >> >> >>>>>>> fpj@apache.org>
> >> >> >> >>>>>>>>> wrote:
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>> None of us expected the reconfig changes to take this
> long
> >> to
> >> >> >> >>>>>> stabilize.
> >> >> >> >>>>>>>>> Until we get there, I don't think we can do anything
> else
> >> with
> >> >> >> 3.5.
> >> >> >> >>>>>> The
> >> >> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a
> >> stable
> >> >> >> rather
> >> >> >> >>>>>>> than
> >> >> >> >>>>>>>>> trying to work around it.
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>> There are lots of people interested in seeing 3.5
> stable,
> >> and
> >> >> if
> >> >> >> we
> >> >> >> >>>>>> get
> >> >> >> >>>>>>>>> everyone to contribute more patches and code reviews, we
> >> >> should
> >> >> >> be
> >> >> >> >>>>>> able
> >> >> >> >>>>>>> to
> >> >> >> >>>>>>>>> do it sooner. After all, it is a community based
> effort, so
> >> >> the
> >> >> >> >>>>>>> community
> >> >> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>> -Flavio
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> >> >> >> cnauroth@hortonworks.com>
> >> >> >> >>>>>>>>> wrote:
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.
> >> Since
> >> >> >> 3.4 is
> >> >> >> >>>>>> the
> >> >> >> >>>>>>>>>> stable maintenance line, we are very conservative about
> >> >> >> >>>>> back-porting
> >> >> >> >>>>>> to
> >> >> >> >>>>>>>>>> it.  Our policy is to limit back-ports to critical bug
> >> fixes
> >> >> and
> >> >> >> >>>>> not
> >> >> >> >>>>>>>>>> introduce any new features in the 3.4 line.  This is a
> >> >> matter of
> >> >> >> >>>>>>> managing
> >> >> >> >>>>>>>>>> risk.
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>> Jason, your question about release cadence is a fair
> >> one.  If
> >> >> >> it's
> >> >> >> >>>>>> any
> >> >> >> >>>>>>>>>> consolation, we are now taking the approach of trying
> to
> >> >> limit
> >> >> >> the
> >> >> >> >>>>>>> scope
> >> >> >> >>>>>>>>>> of anything new introduced in 3.5 too.  That would
> allow
> >> us
> >> >> to
> >> >> >> >>>>> focus
> >> >> >> >>>>>> on
> >> >> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing
> public
> >> >> >> APIs.  I
> >> >> >> >>>>>>> think
> >> >> >> >>>>>>>>>> this will help us accelerate the releases into beta and
> >> >> eventual
> >> >> >> >>>>> GA.
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like
> to
> >> hear
> >> >> >> >>>>>> thoughts
> >> >> >> >>>>>>>>>> from more experienced committers and PMC members about
> >> your
> >> >> >> >>>>> proposal
> >> >> >> >>>>>> to
> >> >> >> >>>>>>>>>> try to cut a stable release for a limited subset of
> what
> >> is
> >> >> in
> >> >> >> >>>>>>> branch-3.5
> >> >> >> >>>>>>>>>> now.  My instinct is that it would be challenging to
> >> >> cherry-pick
> >> >> >> >>>>> out
> >> >> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This
> would
> >> >> become
> >> >> >> >>>>>>> another
> >> >> >> >>>>>>>>>> release line for an already resource-constrained
> volunteer
> >> >> >> staff to
> >> >> >> >>>>>>>>>> manage.  I'd prefer to dedicate those limited
> resources to
> >> >> >> overall
> >> >> >> >>>>>> 3.5
> >> >> >> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain
> >> features
> >> >> >> >>>>>>> "vanished"
> >> >> >> >>>>>>>>>> because of not meeting some stability criteria would be
> >> >> >> >>>>> undesirable.
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>> --Chris Nauroth
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <
> jbr@squareup.com
> >> >
> >> >> >> wrote:
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>>> Chris,
> >> >> >> >>>>>>>>>>>
> >> >> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more
> stable
> >> than
> >> >> >> >>>>> others
> >> >> >> >>>>>>>>> (e.g.
> >> >> >> >>>>>>>>>>> if we don't care about certain new features, is it
> >> >> relatively
> >> >> >> >>>>>> stable)?
> >> >> >> >>>>>>>>>>> Would it be possible to cut out a version that only
> has
> >> the
> >> >> >> bits
> >> >> >> >>>>> we
> >> >> >> >>>>>>>>> think
> >> >> >> >>>>>>>>>>> are stable (and release that)?
> >> >> >> >>>>>>>>>>>
> >> >> >> >>>>>>>>>>> From that timeline, and the historic release cadence,
> it
> >> >> would
> >> >> >> >>>>> seem
> >> >> >> >>>>>> to
> >> >> >> >>>>>>>>> be
> >> >> >> >>>>>>>>>>> a
> >> >> >> >>>>>>>>>>> years away before we get to the stable release?
> >> >> >> >>>>>>>>>>>
> >> >> >> >>>>>>>>>>> Thanks,
> >> >> >> >>>>>>>>>>>
> >> >> >> >>>>>>>>>>> Jason
> >> >> >> >>>>>>>>>>>
> >> >> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> >> >> >> >>>>>>>>> cnauroth@hortonworks.com>
> >> >> >> >>>>>>>>>>> wrote:
> >> >> >> >>>>>>>>>>>
> >> >> >> >>>>>>>>>>>> Hello Doug,
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>> At this point, I think we're still pretty far away
> from
> >> >> >> >>>>> declaring a
> >> >> >> >>>>>>>>>>>> stable
> >> >> >> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close
> >> enough
> >> >> >> that
> >> >> >> >>>>>>> anyone
> >> >> >> >>>>>>>>>>>> can
> >> >> >> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
> >> >> >> describes
> >> >> >> >>>>> the
> >> >> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5
> >> line:
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>> https://s.apache.org/ADK1
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're
> working
> >> on
> >> >> >> >>>>>> resolving a
> >> >> >> >>>>>>>>>>>> few
> >> >> >> >>>>>>>>>>>> more blockers before we produce a release candidate.
> >> >> >> Hopefully
> >> >> >> >>>>>> that
> >> >> >> >>>>>>>>>>>> will
> >> >> >> >>>>>>>>>>>> get done in the next few weeks.
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>> --Chris Nauroth
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com>
> >> wrote:
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>> I know it's only been a few months, but I was
> >> wondering if
> >> >> >> there
> >> >> >> >>>>>>> was a
> >> >> >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1.
> >> Or is
> >> >> >> there
> >> >> >> >>>>>> any
> >> >> >> >>>>>>>>>>>>> chance
> >> >> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just
> another
> >> >> person
> >> >> >> >>>>>> looking
> >> >> >> >>>>>>>>> to
> >> >> >> >>>>>>>>>>>>> have
> >> >> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you
> >> do!
> >> >> :)
> >> >> >> >>>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>> --
> >> >> >> >>>>>>>>>>>>> View this message in context:
> >> >> >> >>>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>
> >> >> >> >>>>>>
> >> >> >> >>>>>
> >> >> >>
> >> >>
> >>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >> >> >> >>>>>>>>>>>> e
> >> >> >> >>>>>>>>>>>>> -tp7581744p7582136.html
> >> >> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
> >> >> >> Nabble.com.
> >> >> >> >>>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>>>
> >> >> >> >>>>>>>>>>
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>
> >> >> >> >>>>>>
> >> >> >> >>>>>
> >> >> >> >
> >> >> >>
> >> >>
> >>
>

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

I would say if in doubt add a safety. (a config parameter to turn it
off). Cost is almost zero and worst case it will just give us peace of
mind. ;-)

Patrick

On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <sh...@gmail.com> wrote:
> ok, thanks for the suggestion, I'll look into it. For reconfig I think its
> pretty clear that its an admin
> functionality. I just always imagined that its controlled via acls, but I
> understand
> the concerns now.
>
> getConfig returns the dynamic config (list of all servers with all ports
> and quorum system if defined)
> and has an option to filter that info and just return the server connection
> string (server and client port only).
>
>
> On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org> wrote:
>
>> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <sh...@gmail.com>
>> wrote:
>> > I don't think that getConfig should be an admin functionality. It is
>> > essential for client-side re-balancing
>> > that we implemented (all clients shoudl be able to detect configuration
>> > changes via getConfig). It could
>> > be hidden somewhat by defining higher-level re-balancing
>> > policies (ZOOKEEPER-2016)
>> > but there hasn't been enough progress on that. Perhaps instead getConfig
>> > should be controlled
>> > by a separate flag ?
>> >
>>
>> I believe that the Hadoop community has something we could use:
>>
>> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
>> (whether through annotations or just documenting it in the API javadoc)
>>
>> e.g. we could list getConfig as public/unstable for example and still
>> ship it as GA. That would mark it as something that could change re
>> API policy.
>>
>> Is the entire config exposed through getConfig? If so then we might
>> want to enable/disable it with a flag similar to reconfig. Might be
>> safer to just do that if we're not sure.
>>
>>
>> Re classification - we could do the same thing with reconfig, but I
>> think that would be a mistake. If we feel strongly where it should
>> live long term we should just move it now.
>>
>> Patrick
>>
>> >
>> > On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org> wrote:
>> >
>> >> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <sh...@gmail.com>
>> >> wrote:
>> >> > Hi Patrick, Flavio,
>> >> >
>> >> > Since there seems to be consensus on this, I can add this flag, unless
>> >> > someone else wants to. I assume that getConfig should still work
>> >> regardless
>> >> > of the flag ? is there a security concern with clients knowing the
>> list
>> >> of
>> >> > servers?
>> >> >
>> >>
>> >> We've always hidden that detail from users. We don't even expose which
>> >> server you're connected to today. I remember Ben (and perhaps Flavio?)
>> >> highlighting this as important to maintain although I'm not super
>> >> familiar with the specifics on why. It made sense to me though from
>> >> the perspective that we don't want clients to make assumptions that
>> >> probably shouldn't.
>> >>
>> >> My thinking is that we should 1) add a config option to enable
>> >> reconfig (off by default), 2) move reconfig specific functionality
>> >> from ZooKeeper.java (including getconfig) into an "admin" package,
>> >> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
>> >> when folks do want to enable reconfig and are also worried about auth.
>> >> (e.g. turn on kerb)
>> >>
>> >> Again, I don't see any of this as a quality issue personally. As such
>> >> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
>> >> were interested in doing such a release. Adjusting the API should be
>> >> done before we move to "beta" though. Although that seems like a
>> >> pretty mechanical (eclipse/idea) type refactoring?
>> >>
>> >> Patrick
>> >>
>> >> > Cheers,
>> >> > Alex
>> >> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>> >> >
>> >> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fp...@apache.org>
>> >> wrote:
>> >> >> > I gotta say that I'm not super excited about this option, but for
>> some
>> >> >> reason I ended up carrying the flag. To recap, I just raised this
>> option
>> >> >> because it seems that there are folks interested in features in 3.5
>> like
>> >> >> SSL and not necessarily in reconfiguration. SSL is important and to
>> take
>> >> >> Kafka as an example, it sucks that we can't have a whole set up using
>> >> SSL.
>> >> >> For ZK, the real questions are:
>> >> >> >
>> >> >> > 1- how fast can we make 3.5 stable?
>> >> >> > 2- would it be faster if we have a way of disabling
>> reconfiguration?
>> >> >> > 3- would enough users care about a stable 3.5 that has
>> reconfiguration
>> >> >> disabled?
>> >> >> >
>> >> >> > It is taking a long time to get 3.5 to beta. There has been some
>> good
>> >> >> activity around 3.5.2 release, which is a great step, but it is
>> unclear
>> >> >> when 3.5.3 is going to come and if we will be able to make 3.5 beta
>> >> then.
>> >> >> Frankly, disabling reconfiguration sounds undesirable because it is
>> an
>> >> >> important feature, but I currently don't use it in production, so
>> from a
>> >> >> practical point of view, I can go both ways. Whether we go through
>> the
>> >> >> trouble of doing 2 depends on users interested in that option and
>> folks
>> >> >> willing to implement it.
>> >> >> >
>> >> >> > To answer your question, Powell, my pseudo-proposal is kind of a
>> funny
>> >> >> option because once the feature is stable, then we wouldn't need a
>> >> switch
>> >> >> any longer, so there is not need of a deprecation path, we just start
>> >> >> ignoring it from the first beta release. Until it is beta, I'd say
>> that
>> >> >> default is disabled.
>> >> >>
>> >> >> I would argue that we need this even when it does become stable. To
>> me
>> >> >> this is not a quality issue so much as it is an auth issue. We want
>> to
>> >> >> make it simple for folks to run a vanilla/old ZK cluster and not
>> worry
>> >> >> about the security implications of having reconfig enabled.
>> >> >>
>> >> >> Patrick
>> >> >>
>> >> >> >
>> >> >> > -Flavio
>> >> >> >
>> >> >> >> On 17 Mar 2016, at 17:44, powell molleti
>> <powellm79@yahoo.com.INVALID
>> >> >
>> >> >> wrote:
>> >> >> >>
>> >> >> >> Hi Flavio,
>> >> >> >>
>> >> >> >> Generally do config options and command line args come under the
>> same
>> >> >> SLA as API?. I was assuming as such hence my question. Perhaps if the
>> >> >> expectation is that this config option is temporary from get go then
>> >> may be
>> >> >> it is ok. The default for re-config support will be enabled or
>> >> disabled?.
>> >> >> >>
>> >> >> >> I am just thinking from provisioning point of view when people
>> >> generate
>> >> >> config options etc.
>> >> >> >>
>> >> >> >> Thanks
>> >> >> >> Powell.
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
>> >> fpj@apache.org>
>> >> >> wrote:
>> >> >> >> Hi Powell,
>> >> >> >>
>> >> >> >> I was thinking config file and system property server side. What's
>> >> your
>> >> >> concern with compatibility? The API itself wouldn't change, but the
>> >> config
>> >> >> option wouldn't exist in previous versions and would not exist
>> either in
>> >> >> later stable versions of 3.5.
>> >> >> >>
>> >> >> >> -Flavio
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>> On 16 Mar 2016, at 22:08, powell molleti
>> >> <po...@yahoo.com.INVALID>
>> >> >> wrote:
>> >> >> >>>
>> >> >> >>> Will this option be supplied via config file/args/API?. Will this
>> >> >> option be a temporary thing i.e what about its compatibility?.
>> >> >> >>>
>> >> >> >>> thanks
>> >> >> >>> Powell.
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
>> >> fpj@apache.org>
>> >> >> wrote:
>> >> >> >>> The main issue to sort out is stability of the API. There is a
>> >> >> security concern around the fact that clients can freely reconfigure
>> the
>> >> >> ensemble. If we follow the plan that Pat proposed some time ago:
>> >> >> >>>
>> >> >> >>>
>> >> >>
>> >>
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> >> >> <
>> >> >>
>> >>
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> >> >> >
>> >> >> >>>
>> >> >> >>> Locking the API is the main step to move it to beta. Sorting out
>> >> bugs
>> >> >> is definitely necessary, but it isn't the main thing that is keeping
>> >> 3.5 in
>> >> >> alpha.
>> >> >> >>>
>> >> >> >>> About making it experimental, I was raising the option of having
>> a
>> >> >> switch that disables the API calls, not the code. The reason why that
>> >> could
>> >> >> work is that anyone using 3.5 who uses the "experimental" API must
>> >> explicit
>> >> >> turn on the switch and enable the calls. If they do it, they need to
>> be
>> >> >> aware that the API can change.
>> >> >> >>>
>> >> >> >>> I must say that I haven't really looked closely into doing it,
>> and
>> >> I'm
>> >> >> not even entirely convinced that this is a good idea, but since Jason
>> >> >> raised the point, I'm exploring options.
>> >> >> >>>
>> >> >> >>> -Flavio
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com>
>> >> wrote:
>> >> >> >>>>
>> >> >> >>>> Looking at the list of ~50 blocker and critical bugs in
>> ZooKeeper,
>> >> >> only 3-4
>> >> >> >>>> are related to reconfig. Given this, and the fact that it is
>> run in
>> >> >> >>>> production since 2012 in multiple companies, I don't think its
>> more
>> >> >> >>>> unstable than any other part of ZooKeeper.
>> >> >> >>>>
>> >> >> >>>> There are multiple reconfig-related bugs that turned out really
>> >> >> difficult
>> >> >> >>>> to debug without access to the actual system or at least to the
>> >> Hudson
>> >> >> >>>> machines where some tests are failing. In the past, Michi and I
>> >> >> physically
>> >> >> >>>> went to Hortonworks to debug one such issue, but this is of
>> course
>> >> >> not a
>> >> >> >>>> good method, and we weren't able to arrange such a visit again.
>> >> >> >>>>
>> >> >> >>>> Regarding making it optional - the reconfig logic has several
>> >> >> different
>> >> >> >>>> parts, some would be really difficult to disable using a
>> >> configuration
>> >> >> >>>> parameter. But the actual dynamic expansion of the cluster is
>> >> >> triggered by
>> >> >> >>>> the reconfig command, so it should not affect users who don't
>> >> invoke
>> >> >> it.
>> >> >> >>>>
>> >> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
>> >> fpj@apache.org>
>> >> >> wrote:
>> >> >> >>>>
>> >> >> >>>>> I suppose we could give it a try. How do other folks feel about
>> >> it?
>> >> >> >>>>>
>> >> >> >>>>> -Flavio
>> >> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com>
>> wrote:
>> >> >> >>>>>
>> >> >> >>>>>> So, you could enable the dynamic reconfiguration feature
>> behind a
>> >> >> config
>> >> >> >>>>>> option, and document that it should only be enabled
>> >> experimentally,
>> >> >> use
>> >> >> >>>>> at
>> >> >> >>>>>> your own risk.  Keep it off by default.  Allow only static
>> >> config by
>> >> >> >>>>>> default, until it's stable?
>> >> >> >>>>>>
>> >> >> >>>>>> Jason
>> >> >> >>>>>>
>> >> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
>> >> fpj@apache.org>
>> >> >> >>>>> wrote:
>> >> >> >>>>>>
>> >> >> >>>>>>> Hi Jason,
>> >> >> >>>>>>>
>> >> >> >>>>>>> The consumer in Kafka is pretty independent from the core
>> >> >> (brokers),
>> >> >> >>>>>>> that's how that project manages to make such a separation. We
>> >> don't
>> >> >> >>>>> have
>> >> >> >>>>>>> the same with ZooKeeper as the feature we are talking about
>> is
>> >> >> part of
>> >> >> >>>>>> the
>> >> >> >>>>>>> server and the only way I see of doing what you say is to
>> turn
>> >> off
>> >> >> >>>>>>> features. More specifically, we'd need to disable the
>> reconfig
>> >> API
>> >> >> and
>> >> >> >>>>> do
>> >> >> >>>>>>> not allow any change to the configuration, even though the
>> code
>> >> is
>> >> >> >>>>> there.
>> >> >> >>>>>>>
>> >> >> >>>>>>> Reconfig here refers to the ability of changing the
>> >> configuration
>> >> >> of an
>> >> >> >>>>>>> ensemble (e.g., changing the set of servers).
>> >> >> >>>>>>>
>> >> >> >>>>>>> -Flavio
>> >> >> >>>>>>>
>> >> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jbr@squareup.com
>> >
>> >> >> wrote:
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> So, it would seem sensible to me to have a release where all
>> >> >> features
>> >> >> >>>>>> are
>> >> >> >>>>>>>> stable, except where noted.  E.g. mark certain features as
>> only
>> >> >> >>>>> 'alpha
>> >> >> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's
>> >> possible
>> >> >> to
>> >> >> >>>>>>> happily
>> >> >> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> There's precedent for doing this sort of thing in other
>> >> projects,
>> >> >> >>>>> e.g.
>> >> >> >>>>>> in
>> >> >> >>>>>>>> Kafka, they've had several release where a new "Consumer
>> API"
>> >> is
>> >> >> >>>>>> shipped
>> >> >> >>>>>>>> that is available for beta-testing, but you can still just
>> use
>> >> the
>> >> >> >>>>>> older
>> >> >> >>>>>>>> stable consumer api, etc.
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> Jason
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>> >> >> >>>>>>> <powellm79@yahoo.com.invalid
>> >> >> >>>>>>>>> wrote:
>> >> >> >>>>>>>>
>> >> >> >>>>>>>>> Hi Doug,
>> >> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using
>> it?.
>> >> Or
>> >> >> have
>> >> >> >>>>>> you
>> >> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being
>> >> >> labeled as
>> >> >> >>>>>>> alpha
>> >> >> >>>>>>>>> might not be fair, since it is far more stable then what
>> most
>> >> >> people
>> >> >> >>>>>>>>> associate an alpha release to be.
>> >> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it will
>> >> just
>> >> >> work
>> >> >> >>>>>> for
>> >> >> >>>>>>>>> you?.
>> >> >> >>>>>>>>> There are many examples of 3.5.X being used in productions
>> >> from
>> >> >> my
>> >> >> >>>>>>> limited
>> >> >> >>>>>>>>> knowledge.
>> >> >> >>>>>>>>> ThanksPowell.
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>> >> >> >>>>>>> fpj@apache.org>
>> >> >> >>>>>>>>> wrote:
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> None of us expected the reconfig changes to take this long
>> to
>> >> >> >>>>>> stabilize.
>> >> >> >>>>>>>>> Until we get there, I don't think we can do anything else
>> with
>> >> >> 3.5.
>> >> >> >>>>>> The
>> >> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a
>> stable
>> >> >> rather
>> >> >> >>>>>>> than
>> >> >> >>>>>>>>> trying to work around it.
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> There are lots of people interested in seeing 3.5 stable,
>> and
>> >> if
>> >> >> we
>> >> >> >>>>>> get
>> >> >> >>>>>>>>> everyone to contribute more patches and code reviews, we
>> >> should
>> >> >> be
>> >> >> >>>>>> able
>> >> >> >>>>>>> to
>> >> >> >>>>>>>>> do it sooner. After all, it is a community based effort, so
>> >> the
>> >> >> >>>>>>> community
>> >> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> -Flavio
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
>> >> >> cnauroth@hortonworks.com>
>> >> >> >>>>>>>>> wrote:
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.
>> Since
>> >> >> 3.4 is
>> >> >> >>>>>> the
>> >> >> >>>>>>>>>> stable maintenance line, we are very conservative about
>> >> >> >>>>> back-porting
>> >> >> >>>>>> to
>> >> >> >>>>>>>>>> it.  Our policy is to limit back-ports to critical bug
>> fixes
>> >> and
>> >> >> >>>>> not
>> >> >> >>>>>>>>>> introduce any new features in the 3.4 line.  This is a
>> >> matter of
>> >> >> >>>>>>> managing
>> >> >> >>>>>>>>>> risk.
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> Jason, your question about release cadence is a fair
>> one.  If
>> >> >> it's
>> >> >> >>>>>> any
>> >> >> >>>>>>>>>> consolation, we are now taking the approach of trying to
>> >> limit
>> >> >> the
>> >> >> >>>>>>> scope
>> >> >> >>>>>>>>>> of anything new introduced in 3.5 too.  That would allow
>> us
>> >> to
>> >> >> >>>>> focus
>> >> >> >>>>>> on
>> >> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public
>> >> >> APIs.  I
>> >> >> >>>>>>> think
>> >> >> >>>>>>>>>> this will help us accelerate the releases into beta and
>> >> eventual
>> >> >> >>>>> GA.
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to
>> hear
>> >> >> >>>>>> thoughts
>> >> >> >>>>>>>>>> from more experienced committers and PMC members about
>> your
>> >> >> >>>>> proposal
>> >> >> >>>>>> to
>> >> >> >>>>>>>>>> try to cut a stable release for a limited subset of what
>> is
>> >> in
>> >> >> >>>>>>> branch-3.5
>> >> >> >>>>>>>>>> now.  My instinct is that it would be challenging to
>> >> cherry-pick
>> >> >> >>>>> out
>> >> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would
>> >> become
>> >> >> >>>>>>> another
>> >> >> >>>>>>>>>> release line for an already resource-constrained volunteer
>> >> >> staff to
>> >> >> >>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to
>> >> >> overall
>> >> >> >>>>>> 3.5
>> >> >> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain
>> features
>> >> >> >>>>>>> "vanished"
>> >> >> >>>>>>>>>> because of not meeting some stability criteria would be
>> >> >> >>>>> undesirable.
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> --Chris Nauroth
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jbr@squareup.com
>> >
>> >> >> wrote:
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>> Chris,
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable
>> than
>> >> >> >>>>> others
>> >> >> >>>>>>>>> (e.g.
>> >> >> >>>>>>>>>>> if we don't care about certain new features, is it
>> >> relatively
>> >> >> >>>>>> stable)?
>> >> >> >>>>>>>>>>> Would it be possible to cut out a version that only has
>> the
>> >> >> bits
>> >> >> >>>>> we
>> >> >> >>>>>>>>> think
>> >> >> >>>>>>>>>>> are stable (and release that)?
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> From that timeline, and the historic release cadence, it
>> >> would
>> >> >> >>>>> seem
>> >> >> >>>>>> to
>> >> >> >>>>>>>>> be
>> >> >> >>>>>>>>>>> a
>> >> >> >>>>>>>>>>> years away before we get to the stable release?
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> Thanks,
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> Jason
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>> >> >> >>>>>>>>> cnauroth@hortonworks.com>
>> >> >> >>>>>>>>>>> wrote:
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>>> Hello Doug,
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> At this point, I think we're still pretty far away from
>> >> >> >>>>> declaring a
>> >> >> >>>>>>>>>>>> stable
>> >> >> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close
>> enough
>> >> >> that
>> >> >> >>>>>>> anyone
>> >> >> >>>>>>>>>>>> can
>> >> >> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
>> >> >> describes
>> >> >> >>>>> the
>> >> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5
>> line:
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> https://s.apache.org/ADK1
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working
>> on
>> >> >> >>>>>> resolving a
>> >> >> >>>>>>>>>>>> few
>> >> >> >>>>>>>>>>>> more blockers before we produce a release candidate.
>> >> >> Hopefully
>> >> >> >>>>>> that
>> >> >> >>>>>>>>>>>> will
>> >> >> >>>>>>>>>>>> get done in the next few weeks.
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> --Chris Nauroth
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com>
>> wrote:
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>> I know it's only been a few months, but I was
>> wondering if
>> >> >> there
>> >> >> >>>>>>> was a
>> >> >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1.
>> Or is
>> >> >> there
>> >> >> >>>>>> any
>> >> >> >>>>>>>>>>>>> chance
>> >> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another
>> >> person
>> >> >> >>>>>> looking
>> >> >> >>>>>>>>> to
>> >> >> >>>>>>>>>>>>> have
>> >> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you
>> do!
>> >> :)
>> >> >> >>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>> --
>> >> >> >>>>>>>>>>>>> View this message in context:
>> >> >> >>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>
>> >> >>
>> >>
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>> >> >> >>>>>>>>>>>> e
>> >> >> >>>>>>>>>>>>> -tp7581744p7582136.html
>> >> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
>> >> >> Nabble.com.
>> >> >> >>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>
>> >> >> >
>> >> >>
>> >>
>>

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

ok, thanks for the suggestion, I'll look into it. For reconfig I think its
pretty clear that its an admin
functionality. I just always imagined that its controlled via acls, but I
understand
the concerns now.

getConfig returns the dynamic config (list of all servers with all ports
and quorum system if defined)
and has an option to filter that info and just return the server connection
string (server and client port only).


On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org> wrote:

> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <sh...@gmail.com>
> wrote:
> > I don't think that getConfig should be an admin functionality. It is
> > essential for client-side re-balancing
> > that we implemented (all clients shoudl be able to detect configuration
> > changes via getConfig). It could
> > be hidden somewhat by defining higher-level re-balancing
> > policies (ZOOKEEPER-2016)
> > but there hasn't been enough progress on that. Perhaps instead getConfig
> > should be controlled
> > by a separate flag ?
> >
>
> I believe that the Hadoop community has something we could use:
>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
> (whether through annotations or just documenting it in the API javadoc)
>
> e.g. we could list getConfig as public/unstable for example and still
> ship it as GA. That would mark it as something that could change re
> API policy.
>
> Is the entire config exposed through getConfig? If so then we might
> want to enable/disable it with a flag similar to reconfig. Might be
> safer to just do that if we're not sure.
>
>
> Re classification - we could do the same thing with reconfig, but I
> think that would be a mistake. If we feel strongly where it should
> live long term we should just move it now.
>
> Patrick
>
> >
> > On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org> wrote:
> >
> >> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <sh...@gmail.com>
> >> wrote:
> >> > Hi Patrick, Flavio,
> >> >
> >> > Since there seems to be consensus on this, I can add this flag, unless
> >> > someone else wants to. I assume that getConfig should still work
> >> regardless
> >> > of the flag ? is there a security concern with clients knowing the
> list
> >> of
> >> > servers?
> >> >
> >>
> >> We've always hidden that detail from users. We don't even expose which
> >> server you're connected to today. I remember Ben (and perhaps Flavio?)
> >> highlighting this as important to maintain although I'm not super
> >> familiar with the specifics on why. It made sense to me though from
> >> the perspective that we don't want clients to make assumptions that
> >> probably shouldn't.
> >>
> >> My thinking is that we should 1) add a config option to enable
> >> reconfig (off by default), 2) move reconfig specific functionality
> >> from ZooKeeper.java (including getconfig) into an "admin" package,
> >> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
> >> when folks do want to enable reconfig and are also worried about auth.
> >> (e.g. turn on kerb)
> >>
> >> Again, I don't see any of this as a quality issue personally. As such
> >> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
> >> were interested in doing such a release. Adjusting the API should be
> >> done before we move to "beta" though. Although that seems like a
> >> pretty mechanical (eclipse/idea) type refactoring?
> >>
> >> Patrick
> >>
> >> > Cheers,
> >> > Alex
> >> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org> wrote:
> >> >
> >> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fp...@apache.org>
> >> wrote:
> >> >> > I gotta say that I'm not super excited about this option, but for
> some
> >> >> reason I ended up carrying the flag. To recap, I just raised this
> option
> >> >> because it seems that there are folks interested in features in 3.5
> like
> >> >> SSL and not necessarily in reconfiguration. SSL is important and to
> take
> >> >> Kafka as an example, it sucks that we can't have a whole set up using
> >> SSL.
> >> >> For ZK, the real questions are:
> >> >> >
> >> >> > 1- how fast can we make 3.5 stable?
> >> >> > 2- would it be faster if we have a way of disabling
> reconfiguration?
> >> >> > 3- would enough users care about a stable 3.5 that has
> reconfiguration
> >> >> disabled?
> >> >> >
> >> >> > It is taking a long time to get 3.5 to beta. There has been some
> good
> >> >> activity around 3.5.2 release, which is a great step, but it is
> unclear
> >> >> when 3.5.3 is going to come and if we will be able to make 3.5 beta
> >> then.
> >> >> Frankly, disabling reconfiguration sounds undesirable because it is
> an
> >> >> important feature, but I currently don't use it in production, so
> from a
> >> >> practical point of view, I can go both ways. Whether we go through
> the
> >> >> trouble of doing 2 depends on users interested in that option and
> folks
> >> >> willing to implement it.
> >> >> >
> >> >> > To answer your question, Powell, my pseudo-proposal is kind of a
> funny
> >> >> option because once the feature is stable, then we wouldn't need a
> >> switch
> >> >> any longer, so there is not need of a deprecation path, we just start
> >> >> ignoring it from the first beta release. Until it is beta, I'd say
> that
> >> >> default is disabled.
> >> >>
> >> >> I would argue that we need this even when it does become stable. To
> me
> >> >> this is not a quality issue so much as it is an auth issue. We want
> to
> >> >> make it simple for folks to run a vanilla/old ZK cluster and not
> worry
> >> >> about the security implications of having reconfig enabled.
> >> >>
> >> >> Patrick
> >> >>
> >> >> >
> >> >> > -Flavio
> >> >> >
> >> >> >> On 17 Mar 2016, at 17:44, powell molleti
> <powellm79@yahoo.com.INVALID
> >> >
> >> >> wrote:
> >> >> >>
> >> >> >> Hi Flavio,
> >> >> >>
> >> >> >> Generally do config options and command line args come under the
> same
> >> >> SLA as API?. I was assuming as such hence my question. Perhaps if the
> >> >> expectation is that this config option is temporary from get go then
> >> may be
> >> >> it is ok. The default for re-config support will be enabled or
> >> disabled?.
> >> >> >>
> >> >> >> I am just thinking from provisioning point of view when people
> >> generate
> >> >> config options etc.
> >> >> >>
> >> >> >> Thanks
> >> >> >> Powell.
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
> >> fpj@apache.org>
> >> >> wrote:
> >> >> >> Hi Powell,
> >> >> >>
> >> >> >> I was thinking config file and system property server side. What's
> >> your
> >> >> concern with compatibility? The API itself wouldn't change, but the
> >> config
> >> >> option wouldn't exist in previous versions and would not exist
> either in
> >> >> later stable versions of 3.5.
> >> >> >>
> >> >> >> -Flavio
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>> On 16 Mar 2016, at 22:08, powell molleti
> >> <po...@yahoo.com.INVALID>
> >> >> wrote:
> >> >> >>>
> >> >> >>> Will this option be supplied via config file/args/API?. Will this
> >> >> option be a temporary thing i.e what about its compatibility?.
> >> >> >>>
> >> >> >>> thanks
> >> >> >>> Powell.
> >> >> >>>
> >> >> >>>
> >> >> >>>
> >> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
> >> fpj@apache.org>
> >> >> wrote:
> >> >> >>> The main issue to sort out is stability of the API. There is a
> >> >> security concern around the fact that clients can freely reconfigure
> the
> >> >> ensemble. If we follow the plan that Pat proposed some time ago:
> >> >> >>>
> >> >> >>>
> >> >>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >> >> <
> >> >>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >> >> >
> >> >> >>>
> >> >> >>> Locking the API is the main step to move it to beta. Sorting out
> >> bugs
> >> >> is definitely necessary, but it isn't the main thing that is keeping
> >> 3.5 in
> >> >> alpha.
> >> >> >>>
> >> >> >>> About making it experimental, I was raising the option of having
> a
> >> >> switch that disables the API calls, not the code. The reason why that
> >> could
> >> >> work is that anyone using 3.5 who uses the "experimental" API must
> >> explicit
> >> >> turn on the switch and enable the calls. If they do it, they need to
> be
> >> >> aware that the API can change.
> >> >> >>>
> >> >> >>> I must say that I haven't really looked closely into doing it,
> and
> >> I'm
> >> >> not even entirely convinced that this is a good idea, but since Jason
> >> >> raised the point, I'm exploring options.
> >> >> >>>
> >> >> >>> -Flavio
> >> >> >>>
> >> >> >>>
> >> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com>
> >> wrote:
> >> >> >>>>
> >> >> >>>> Looking at the list of ~50 blocker and critical bugs in
> ZooKeeper,
> >> >> only 3-4
> >> >> >>>> are related to reconfig. Given this, and the fact that it is
> run in
> >> >> >>>> production since 2012 in multiple companies, I don't think its
> more
> >> >> >>>> unstable than any other part of ZooKeeper.
> >> >> >>>>
> >> >> >>>> There are multiple reconfig-related bugs that turned out really
> >> >> difficult
> >> >> >>>> to debug without access to the actual system or at least to the
> >> Hudson
> >> >> >>>> machines where some tests are failing. In the past, Michi and I
> >> >> physically
> >> >> >>>> went to Hortonworks to debug one such issue, but this is of
> course
> >> >> not a
> >> >> >>>> good method, and we weren't able to arrange such a visit again.
> >> >> >>>>
> >> >> >>>> Regarding making it optional - the reconfig logic has several
> >> >> different
> >> >> >>>> parts, some would be really difficult to disable using a
> >> configuration
> >> >> >>>> parameter. But the actual dynamic expansion of the cluster is
> >> >> triggered by
> >> >> >>>> the reconfig command, so it should not affect users who don't
> >> invoke
> >> >> it.
> >> >> >>>>
> >> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
> >> fpj@apache.org>
> >> >> wrote:
> >> >> >>>>
> >> >> >>>>> I suppose we could give it a try. How do other folks feel about
> >> it?
> >> >> >>>>>
> >> >> >>>>> -Flavio
> >> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com>
> wrote:
> >> >> >>>>>
> >> >> >>>>>> So, you could enable the dynamic reconfiguration feature
> behind a
> >> >> config
> >> >> >>>>>> option, and document that it should only be enabled
> >> experimentally,
> >> >> use
> >> >> >>>>> at
> >> >> >>>>>> your own risk.  Keep it off by default.  Allow only static
> >> config by
> >> >> >>>>>> default, until it's stable?
> >> >> >>>>>>
> >> >> >>>>>> Jason
> >> >> >>>>>>
> >> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
> >> fpj@apache.org>
> >> >> >>>>> wrote:
> >> >> >>>>>>
> >> >> >>>>>>> Hi Jason,
> >> >> >>>>>>>
> >> >> >>>>>>> The consumer in Kafka is pretty independent from the core
> >> >> (brokers),
> >> >> >>>>>>> that's how that project manages to make such a separation. We
> >> don't
> >> >> >>>>> have
> >> >> >>>>>>> the same with ZooKeeper as the feature we are talking about
> is
> >> >> part of
> >> >> >>>>>> the
> >> >> >>>>>>> server and the only way I see of doing what you say is to
> turn
> >> off
> >> >> >>>>>>> features. More specifically, we'd need to disable the
> reconfig
> >> API
> >> >> and
> >> >> >>>>> do
> >> >> >>>>>>> not allow any change to the configuration, even though the
> code
> >> is
> >> >> >>>>> there.
> >> >> >>>>>>>
> >> >> >>>>>>> Reconfig here refers to the ability of changing the
> >> configuration
> >> >> of an
> >> >> >>>>>>> ensemble (e.g., changing the set of servers).
> >> >> >>>>>>>
> >> >> >>>>>>> -Flavio
> >> >> >>>>>>>
> >> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jbr@squareup.com
> >
> >> >> wrote:
> >> >> >>>>>>>>
> >> >> >>>>>>>> So, it would seem sensible to me to have a release where all
> >> >> features
> >> >> >>>>>> are
> >> >> >>>>>>>> stable, except where noted.  E.g. mark certain features as
> only
> >> >> >>>>> 'alpha
> >> >> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's
> >> possible
> >> >> to
> >> >> >>>>>>> happily
> >> >> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
> >> >> >>>>>>>>
> >> >> >>>>>>>> There's precedent for doing this sort of thing in other
> >> projects,
> >> >> >>>>> e.g.
> >> >> >>>>>> in
> >> >> >>>>>>>> Kafka, they've had several release where a new "Consumer
> API"
> >> is
> >> >> >>>>>> shipped
> >> >> >>>>>>>> that is available for beta-testing, but you can still just
> use
> >> the
> >> >> >>>>>> older
> >> >> >>>>>>>> stable consumer api, etc.
> >> >> >>>>>>>>
> >> >> >>>>>>>> Jason
> >> >> >>>>>>>>
> >> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> >> >> >>>>>>> <powellm79@yahoo.com.invalid
> >> >> >>>>>>>>> wrote:
> >> >> >>>>>>>>
> >> >> >>>>>>>>> Hi Doug,
> >> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using
> it?.
> >> Or
> >> >> have
> >> >> >>>>>> you
> >> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being
> >> >> labeled as
> >> >> >>>>>>> alpha
> >> >> >>>>>>>>> might not be fair, since it is far more stable then what
> most
> >> >> people
> >> >> >>>>>>>>> associate an alpha release to be.
> >> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it will
> >> just
> >> >> work
> >> >> >>>>>> for
> >> >> >>>>>>>>> you?.
> >> >> >>>>>>>>> There are many examples of 3.5.X being used in productions
> >> from
> >> >> my
> >> >> >>>>>>> limited
> >> >> >>>>>>>>> knowledge.
> >> >> >>>>>>>>> ThanksPowell.
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> >> >> >>>>>>> fpj@apache.org>
> >> >> >>>>>>>>> wrote:
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> None of us expected the reconfig changes to take this long
> to
> >> >> >>>>>> stabilize.
> >> >> >>>>>>>>> Until we get there, I don't think we can do anything else
> with
> >> >> 3.5.
> >> >> >>>>>> The
> >> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a
> stable
> >> >> rather
> >> >> >>>>>>> than
> >> >> >>>>>>>>> trying to work around it.
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> There are lots of people interested in seeing 3.5 stable,
> and
> >> if
> >> >> we
> >> >> >>>>>> get
> >> >> >>>>>>>>> everyone to contribute more patches and code reviews, we
> >> should
> >> >> be
> >> >> >>>>>> able
> >> >> >>>>>>> to
> >> >> >>>>>>>>> do it sooner. After all, it is a community based effort, so
> >> the
> >> >> >>>>>>> community
> >> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> -Flavio
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> >> >> cnauroth@hortonworks.com>
> >> >> >>>>>>>>> wrote:
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.
> Since
> >> >> 3.4 is
> >> >> >>>>>> the
> >> >> >>>>>>>>>> stable maintenance line, we are very conservative about
> >> >> >>>>> back-porting
> >> >> >>>>>> to
> >> >> >>>>>>>>>> it.  Our policy is to limit back-ports to critical bug
> fixes
> >> and
> >> >> >>>>> not
> >> >> >>>>>>>>>> introduce any new features in the 3.4 line.  This is a
> >> matter of
> >> >> >>>>>>> managing
> >> >> >>>>>>>>>> risk.
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> Jason, your question about release cadence is a fair
> one.  If
> >> >> it's
> >> >> >>>>>> any
> >> >> >>>>>>>>>> consolation, we are now taking the approach of trying to
> >> limit
> >> >> the
> >> >> >>>>>>> scope
> >> >> >>>>>>>>>> of anything new introduced in 3.5 too.  That would allow
> us
> >> to
> >> >> >>>>> focus
> >> >> >>>>>> on
> >> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public
> >> >> APIs.  I
> >> >> >>>>>>> think
> >> >> >>>>>>>>>> this will help us accelerate the releases into beta and
> >> eventual
> >> >> >>>>> GA.
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to
> hear
> >> >> >>>>>> thoughts
> >> >> >>>>>>>>>> from more experienced committers and PMC members about
> your
> >> >> >>>>> proposal
> >> >> >>>>>> to
> >> >> >>>>>>>>>> try to cut a stable release for a limited subset of what
> is
> >> in
> >> >> >>>>>>> branch-3.5
> >> >> >>>>>>>>>> now.  My instinct is that it would be challenging to
> >> cherry-pick
> >> >> >>>>> out
> >> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would
> >> become
> >> >> >>>>>>> another
> >> >> >>>>>>>>>> release line for an already resource-constrained volunteer
> >> >> staff to
> >> >> >>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to
> >> >> overall
> >> >> >>>>>> 3.5
> >> >> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain
> features
> >> >> >>>>>>> "vanished"
> >> >> >>>>>>>>>> because of not meeting some stability criteria would be
> >> >> >>>>> undesirable.
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> --Chris Nauroth
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jbr@squareup.com
> >
> >> >> wrote:
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>> Chris,
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable
> than
> >> >> >>>>> others
> >> >> >>>>>>>>> (e.g.
> >> >> >>>>>>>>>>> if we don't care about certain new features, is it
> >> relatively
> >> >> >>>>>> stable)?
> >> >> >>>>>>>>>>> Would it be possible to cut out a version that only has
> the
> >> >> bits
> >> >> >>>>> we
> >> >> >>>>>>>>> think
> >> >> >>>>>>>>>>> are stable (and release that)?
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> From that timeline, and the historic release cadence, it
> >> would
> >> >> >>>>> seem
> >> >> >>>>>> to
> >> >> >>>>>>>>> be
> >> >> >>>>>>>>>>> a
> >> >> >>>>>>>>>>> years away before we get to the stable release?
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> Thanks,
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> Jason
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> >> >> >>>>>>>>> cnauroth@hortonworks.com>
> >> >> >>>>>>>>>>> wrote:
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>>> Hello Doug,
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> At this point, I think we're still pretty far away from
> >> >> >>>>> declaring a
> >> >> >>>>>>>>>>>> stable
> >> >> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close
> enough
> >> >> that
> >> >> >>>>>>> anyone
> >> >> >>>>>>>>>>>> can
> >> >> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
> >> >> describes
> >> >> >>>>> the
> >> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5
> line:
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> https://s.apache.org/ADK1
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working
> on
> >> >> >>>>>> resolving a
> >> >> >>>>>>>>>>>> few
> >> >> >>>>>>>>>>>> more blockers before we produce a release candidate.
> >> >> Hopefully
> >> >> >>>>>> that
> >> >> >>>>>>>>>>>> will
> >> >> >>>>>>>>>>>> get done in the next few weeks.
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> --Chris Nauroth
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com>
> wrote:
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>>> I know it's only been a few months, but I was
> wondering if
> >> >> there
> >> >> >>>>>>> was a
> >> >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1.
> Or is
> >> >> there
> >> >> >>>>>> any
> >> >> >>>>>>>>>>>>> chance
> >> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another
> >> person
> >> >> >>>>>> looking
> >> >> >>>>>>>>> to
> >> >> >>>>>>>>>>>>> have
> >> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you
> do!
> >> :)
> >> >> >>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>> --
> >> >> >>>>>>>>>>>>> View this message in context:
> >> >> >>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>
> >> >> >>>>>
> >> >>
> >>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >> >> >>>>>>>>>>>> e
> >> >> >>>>>>>>>>>>> -tp7581744p7582136.html
> >> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
> >> >> Nabble.com.
> >> >> >>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>
> >> >> >>>>>
> >> >> >
> >> >>
> >>
>

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <sh...@gmail.com> wrote:
> I don't think that getConfig should be an admin functionality. It is
> essential for client-side re-balancing
> that we implemented (all clients shoudl be able to detect configuration
> changes via getConfig). It could
> be hidden somewhat by defining higher-level re-balancing
> policies (ZOOKEEPER-2016)
> but there hasn't been enough progress on that. Perhaps instead getConfig
> should be controlled
> by a separate flag ?
>

I believe that the Hadoop community has something we could use:
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html
(whether through annotations or just documenting it in the API javadoc)

e.g. we could list getConfig as public/unstable for example and still
ship it as GA. That would mark it as something that could change re
API policy.

Is the entire config exposed through getConfig? If so then we might
want to enable/disable it with a flag similar to reconfig. Might be
safer to just do that if we're not sure.


Re classification - we could do the same thing with reconfig, but I
think that would be a mistake. If we feel strongly where it should
live long term we should just move it now.

Patrick

>
> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org> wrote:
>
>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <sh...@gmail.com>
>> wrote:
>> > Hi Patrick, Flavio,
>> >
>> > Since there seems to be consensus on this, I can add this flag, unless
>> > someone else wants to. I assume that getConfig should still work
>> regardless
>> > of the flag ? is there a security concern with clients knowing the list
>> of
>> > servers?
>> >
>>
>> We've always hidden that detail from users. We don't even expose which
>> server you're connected to today. I remember Ben (and perhaps Flavio?)
>> highlighting this as important to maintain although I'm not super
>> familiar with the specifics on why. It made sense to me though from
>> the perspective that we don't want clients to make assumptions that
>> probably shouldn't.
>>
>> My thinking is that we should 1) add a config option to enable
>> reconfig (off by default), 2) move reconfig specific functionality
>> from ZooKeeper.java (including getconfig) into an "admin" package,
>> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
>> when folks do want to enable reconfig and are also worried about auth.
>> (e.g. turn on kerb)
>>
>> Again, I don't see any of this as a quality issue personally. As such
>> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
>> were interested in doing such a release. Adjusting the API should be
>> done before we move to "beta" though. Although that seems like a
>> pretty mechanical (eclipse/idea) type refactoring?
>>
>> Patrick
>>
>> > Cheers,
>> > Alex
>> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>> >
>> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fp...@apache.org>
>> wrote:
>> >> > I gotta say that I'm not super excited about this option, but for some
>> >> reason I ended up carrying the flag. To recap, I just raised this option
>> >> because it seems that there are folks interested in features in 3.5 like
>> >> SSL and not necessarily in reconfiguration. SSL is important and to take
>> >> Kafka as an example, it sucks that we can't have a whole set up using
>> SSL.
>> >> For ZK, the real questions are:
>> >> >
>> >> > 1- how fast can we make 3.5 stable?
>> >> > 2- would it be faster if we have a way of disabling reconfiguration?
>> >> > 3- would enough users care about a stable 3.5 that has reconfiguration
>> >> disabled?
>> >> >
>> >> > It is taking a long time to get 3.5 to beta. There has been some good
>> >> activity around 3.5.2 release, which is a great step, but it is unclear
>> >> when 3.5.3 is going to come and if we will be able to make 3.5 beta
>> then.
>> >> Frankly, disabling reconfiguration sounds undesirable because it is an
>> >> important feature, but I currently don't use it in production, so from a
>> >> practical point of view, I can go both ways. Whether we go through the
>> >> trouble of doing 2 depends on users interested in that option and folks
>> >> willing to implement it.
>> >> >
>> >> > To answer your question, Powell, my pseudo-proposal is kind of a funny
>> >> option because once the feature is stable, then we wouldn't need a
>> switch
>> >> any longer, so there is not need of a deprecation path, we just start
>> >> ignoring it from the first beta release. Until it is beta, I'd say that
>> >> default is disabled.
>> >>
>> >> I would argue that we need this even when it does become stable. To me
>> >> this is not a quality issue so much as it is an auth issue. We want to
>> >> make it simple for folks to run a vanilla/old ZK cluster and not worry
>> >> about the security implications of having reconfig enabled.
>> >>
>> >> Patrick
>> >>
>> >> >
>> >> > -Flavio
>> >> >
>> >> >> On 17 Mar 2016, at 17:44, powell molleti <powellm79@yahoo.com.INVALID
>> >
>> >> wrote:
>> >> >>
>> >> >> Hi Flavio,
>> >> >>
>> >> >> Generally do config options and command line args come under the same
>> >> SLA as API?. I was assuming as such hence my question. Perhaps if the
>> >> expectation is that this config option is temporary from get go then
>> may be
>> >> it is ok. The default for re-config support will be enabled or
>> disabled?.
>> >> >>
>> >> >> I am just thinking from provisioning point of view when people
>> generate
>> >> config options etc.
>> >> >>
>> >> >> Thanks
>> >> >> Powell.
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
>> fpj@apache.org>
>> >> wrote:
>> >> >> Hi Powell,
>> >> >>
>> >> >> I was thinking config file and system property server side. What's
>> your
>> >> concern with compatibility? The API itself wouldn't change, but the
>> config
>> >> option wouldn't exist in previous versions and would not exist either in
>> >> later stable versions of 3.5.
>> >> >>
>> >> >> -Flavio
>> >> >>
>> >> >>
>> >> >>
>> >> >>> On 16 Mar 2016, at 22:08, powell molleti
>> <po...@yahoo.com.INVALID>
>> >> wrote:
>> >> >>>
>> >> >>> Will this option be supplied via config file/args/API?. Will this
>> >> option be a temporary thing i.e what about its compatibility?.
>> >> >>>
>> >> >>> thanks
>> >> >>> Powell.
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
>> fpj@apache.org>
>> >> wrote:
>> >> >>> The main issue to sort out is stability of the API. There is a
>> >> security concern around the fact that clients can freely reconfigure the
>> >> ensemble. If we follow the plan that Pat proposed some time ago:
>> >> >>>
>> >> >>>
>> >>
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> >> <
>> >>
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> >> >
>> >> >>>
>> >> >>> Locking the API is the main step to move it to beta. Sorting out
>> bugs
>> >> is definitely necessary, but it isn't the main thing that is keeping
>> 3.5 in
>> >> alpha.
>> >> >>>
>> >> >>> About making it experimental, I was raising the option of having a
>> >> switch that disables the API calls, not the code. The reason why that
>> could
>> >> work is that anyone using 3.5 who uses the "experimental" API must
>> explicit
>> >> turn on the switch and enable the calls. If they do it, they need to be
>> >> aware that the API can change.
>> >> >>>
>> >> >>> I must say that I haven't really looked closely into doing it, and
>> I'm
>> >> not even entirely convinced that this is a good idea, but since Jason
>> >> raised the point, I'm exploring options.
>> >> >>>
>> >> >>> -Flavio
>> >> >>>
>> >> >>>
>> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com>
>> wrote:
>> >> >>>>
>> >> >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
>> >> only 3-4
>> >> >>>> are related to reconfig. Given this, and the fact that it is run in
>> >> >>>> production since 2012 in multiple companies, I don't think its more
>> >> >>>> unstable than any other part of ZooKeeper.
>> >> >>>>
>> >> >>>> There are multiple reconfig-related bugs that turned out really
>> >> difficult
>> >> >>>> to debug without access to the actual system or at least to the
>> Hudson
>> >> >>>> machines where some tests are failing. In the past, Michi and I
>> >> physically
>> >> >>>> went to Hortonworks to debug one such issue, but this is of course
>> >> not a
>> >> >>>> good method, and we weren't able to arrange such a visit again.
>> >> >>>>
>> >> >>>> Regarding making it optional - the reconfig logic has several
>> >> different
>> >> >>>> parts, some would be really difficult to disable using a
>> configuration
>> >> >>>> parameter. But the actual dynamic expansion of the cluster is
>> >> triggered by
>> >> >>>> the reconfig command, so it should not affect users who don't
>> invoke
>> >> it.
>> >> >>>>
>> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
>> fpj@apache.org>
>> >> wrote:
>> >> >>>>
>> >> >>>>> I suppose we could give it a try. How do other folks feel about
>> it?
>> >> >>>>>
>> >> >>>>> -Flavio
>> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>> >> >>>>>
>> >> >>>>>> So, you could enable the dynamic reconfiguration feature behind a
>> >> config
>> >> >>>>>> option, and document that it should only be enabled
>> experimentally,
>> >> use
>> >> >>>>> at
>> >> >>>>>> your own risk.  Keep it off by default.  Allow only static
>> config by
>> >> >>>>>> default, until it's stable?
>> >> >>>>>>
>> >> >>>>>> Jason
>> >> >>>>>>
>> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
>> fpj@apache.org>
>> >> >>>>> wrote:
>> >> >>>>>>
>> >> >>>>>>> Hi Jason,
>> >> >>>>>>>
>> >> >>>>>>> The consumer in Kafka is pretty independent from the core
>> >> (brokers),
>> >> >>>>>>> that's how that project manages to make such a separation. We
>> don't
>> >> >>>>> have
>> >> >>>>>>> the same with ZooKeeper as the feature we are talking about is
>> >> part of
>> >> >>>>>> the
>> >> >>>>>>> server and the only way I see of doing what you say is to turn
>> off
>> >> >>>>>>> features. More specifically, we'd need to disable the reconfig
>> API
>> >> and
>> >> >>>>> do
>> >> >>>>>>> not allow any change to the configuration, even though the code
>> is
>> >> >>>>> there.
>> >> >>>>>>>
>> >> >>>>>>> Reconfig here refers to the ability of changing the
>> configuration
>> >> of an
>> >> >>>>>>> ensemble (e.g., changing the set of servers).
>> >> >>>>>>>
>> >> >>>>>>> -Flavio
>> >> >>>>>>>
>> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com>
>> >> wrote:
>> >> >>>>>>>>
>> >> >>>>>>>> So, it would seem sensible to me to have a release where all
>> >> features
>> >> >>>>>> are
>> >> >>>>>>>> stable, except where noted.  E.g. mark certain features as only
>> >> >>>>> 'alpha
>> >> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's
>> possible
>> >> to
>> >> >>>>>>> happily
>> >> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>> >> >>>>>>>>
>> >> >>>>>>>> There's precedent for doing this sort of thing in other
>> projects,
>> >> >>>>> e.g.
>> >> >>>>>> in
>> >> >>>>>>>> Kafka, they've had several release where a new "Consumer API"
>> is
>> >> >>>>>> shipped
>> >> >>>>>>>> that is available for beta-testing, but you can still just use
>> the
>> >> >>>>>> older
>> >> >>>>>>>> stable consumer api, etc.
>> >> >>>>>>>>
>> >> >>>>>>>> Jason
>> >> >>>>>>>>
>> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>> >> >>>>>>> <powellm79@yahoo.com.invalid
>> >> >>>>>>>>> wrote:
>> >> >>>>>>>>
>> >> >>>>>>>>> Hi Doug,
>> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using it?.
>> Or
>> >> have
>> >> >>>>>> you
>> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being
>> >> labeled as
>> >> >>>>>>> alpha
>> >> >>>>>>>>> might not be fair, since it is far more stable then what most
>> >> people
>> >> >>>>>>>>> associate an alpha release to be.
>> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it will
>> just
>> >> work
>> >> >>>>>> for
>> >> >>>>>>>>> you?.
>> >> >>>>>>>>> There are many examples of 3.5.X being used in productions
>> from
>> >> my
>> >> >>>>>>> limited
>> >> >>>>>>>>> knowledge.
>> >> >>>>>>>>> ThanksPowell.
>> >> >>>>>>>>>
>> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>> >> >>>>>>> fpj@apache.org>
>> >> >>>>>>>>> wrote:
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>> None of us expected the reconfig changes to take this long to
>> >> >>>>>> stabilize.
>> >> >>>>>>>>> Until we get there, I don't think we can do anything else with
>> >> 3.5.
>> >> >>>>>> The
>> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable
>> >> rather
>> >> >>>>>>> than
>> >> >>>>>>>>> trying to work around it.
>> >> >>>>>>>>>
>> >> >>>>>>>>> There are lots of people interested in seeing 3.5 stable, and
>> if
>> >> we
>> >> >>>>>> get
>> >> >>>>>>>>> everyone to contribute more patches and code reviews, we
>> should
>> >> be
>> >> >>>>>> able
>> >> >>>>>>> to
>> >> >>>>>>>>> do it sooner. After all, it is a community based effort, so
>> the
>> >> >>>>>>> community
>> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>> >> >>>>>>>>>
>> >> >>>>>>>>> -Flavio
>> >> >>>>>>>>>
>> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
>> >> cnauroth@hortonworks.com>
>> >> >>>>>>>>> wrote:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since
>> >> 3.4 is
>> >> >>>>>> the
>> >> >>>>>>>>>> stable maintenance line, we are very conservative about
>> >> >>>>> back-porting
>> >> >>>>>> to
>> >> >>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes
>> and
>> >> >>>>> not
>> >> >>>>>>>>>> introduce any new features in the 3.4 line.  This is a
>> matter of
>> >> >>>>>>> managing
>> >> >>>>>>>>>> risk.
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Jason, your question about release cadence is a fair one.  If
>> >> it's
>> >> >>>>>> any
>> >> >>>>>>>>>> consolation, we are now taking the approach of trying to
>> limit
>> >> the
>> >> >>>>>>> scope
>> >> >>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us
>> to
>> >> >>>>> focus
>> >> >>>>>> on
>> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public
>> >> APIs.  I
>> >> >>>>>>> think
>> >> >>>>>>>>>> this will help us accelerate the releases into beta and
>> eventual
>> >> >>>>> GA.
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>> >> >>>>>> thoughts
>> >> >>>>>>>>>> from more experienced committers and PMC members about your
>> >> >>>>> proposal
>> >> >>>>>> to
>> >> >>>>>>>>>> try to cut a stable release for a limited subset of what is
>> in
>> >> >>>>>>> branch-3.5
>> >> >>>>>>>>>> now.  My instinct is that it would be challenging to
>> cherry-pick
>> >> >>>>> out
>> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would
>> become
>> >> >>>>>>> another
>> >> >>>>>>>>>> release line for an already resource-constrained volunteer
>> >> staff to
>> >> >>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to
>> >> overall
>> >> >>>>>> 3.5
>> >> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>> >> >>>>>>> "vanished"
>> >> >>>>>>>>>> because of not meeting some stability criteria would be
>> >> >>>>> undesirable.
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> --Chris Nauroth
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com>
>> >> wrote:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>> Chris,
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>> >> >>>>> others
>> >> >>>>>>>>> (e.g.
>> >> >>>>>>>>>>> if we don't care about certain new features, is it
>> relatively
>> >> >>>>>> stable)?
>> >> >>>>>>>>>>> Would it be possible to cut out a version that only has the
>> >> bits
>> >> >>>>> we
>> >> >>>>>>>>> think
>> >> >>>>>>>>>>> are stable (and release that)?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> From that timeline, and the historic release cadence, it
>> would
>> >> >>>>> seem
>> >> >>>>>> to
>> >> >>>>>>>>> be
>> >> >>>>>>>>>>> a
>> >> >>>>>>>>>>> years away before we get to the stable release?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Thanks,
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Jason
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>> >> >>>>>>>>> cnauroth@hortonworks.com>
>> >> >>>>>>>>>>> wrote:
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>> Hello Doug,
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> At this point, I think we're still pretty far away from
>> >> >>>>> declaring a
>> >> >>>>>>>>>>>> stable
>> >> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough
>> >> that
>> >> >>>>>>> anyone
>> >> >>>>>>>>>>>> can
>> >> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
>> >> describes
>> >> >>>>> the
>> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> https://s.apache.org/ADK1
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>> >> >>>>>> resolving a
>> >> >>>>>>>>>>>> few
>> >> >>>>>>>>>>>> more blockers before we produce a release candidate.
>> >> Hopefully
>> >> >>>>>> that
>> >> >>>>>>>>>>>> will
>> >> >>>>>>>>>>>> get done in the next few weeks.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> --Chris Nauroth
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>> I know it's only been a few months, but I was wondering if
>> >> there
>> >> >>>>>>> was a
>> >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is
>> >> there
>> >> >>>>>> any
>> >> >>>>>>>>>>>>> chance
>> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another
>> person
>> >> >>>>>> looking
>> >> >>>>>>>>> to
>> >> >>>>>>>>>>>>> have
>> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you do!
>> :)
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> --
>> >> >>>>>>>>>>>>> View this message in context:
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>
>> >> >>>>>>
>> >> >>>>>
>> >>
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>> >> >>>>>>>>>>>> e
>> >> >>>>>>>>>>>>> -tp7581744p7582136.html
>> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
>> >> Nabble.com.
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>
>> >> >>>>>
>> >> >
>> >>
>>

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

I don't think that getConfig should be an admin functionality. It is
essential for client-side re-balancing
that we implemented (all clients shoudl be able to detect configuration
changes via getConfig). It could
be hidden somewhat by defining higher-level re-balancing
policies (ZOOKEEPER-2016)
but there hasn't been enough progress on that. Perhaps instead getConfig
should be controlled
by a separate flag ?

Alex

On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <ph...@apache.org> wrote:

> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <sh...@gmail.com>
> wrote:
> > Hi Patrick, Flavio,
> >
> > Since there seems to be consensus on this, I can add this flag, unless
> > someone else wants to. I assume that getConfig should still work
> regardless
> > of the flag ? is there a security concern with clients knowing the list
> of
> > servers?
> >
>
> We've always hidden that detail from users. We don't even expose which
> server you're connected to today. I remember Ben (and perhaps Flavio?)
> highlighting this as important to maintain although I'm not super
> familiar with the specifics on why. It made sense to me though from
> the perspective that we don't want clients to make assumptions that
> probably shouldn't.
>
> My thinking is that we should 1) add a config option to enable
> reconfig (off by default), 2) move reconfig specific functionality
> from ZooKeeper.java (including getconfig) into an "admin" package,
> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
> when folks do want to enable reconfig and are also worried about auth.
> (e.g. turn on kerb)
>
> Again, I don't see any of this as a quality issue personally. As such
> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
> were interested in doing such a release. Adjusting the API should be
> done before we move to "beta" though. Although that seems like a
> pretty mechanical (eclipse/idea) type refactoring?
>
> Patrick
>
> > Cheers,
> > Alex
> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org> wrote:
> >
> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fp...@apache.org>
> wrote:
> >> > I gotta say that I'm not super excited about this option, but for some
> >> reason I ended up carrying the flag. To recap, I just raised this option
> >> because it seems that there are folks interested in features in 3.5 like
> >> SSL and not necessarily in reconfiguration. SSL is important and to take
> >> Kafka as an example, it sucks that we can't have a whole set up using
> SSL.
> >> For ZK, the real questions are:
> >> >
> >> > 1- how fast can we make 3.5 stable?
> >> > 2- would it be faster if we have a way of disabling reconfiguration?
> >> > 3- would enough users care about a stable 3.5 that has reconfiguration
> >> disabled?
> >> >
> >> > It is taking a long time to get 3.5 to beta. There has been some good
> >> activity around 3.5.2 release, which is a great step, but it is unclear
> >> when 3.5.3 is going to come and if we will be able to make 3.5 beta
> then.
> >> Frankly, disabling reconfiguration sounds undesirable because it is an
> >> important feature, but I currently don't use it in production, so from a
> >> practical point of view, I can go both ways. Whether we go through the
> >> trouble of doing 2 depends on users interested in that option and folks
> >> willing to implement it.
> >> >
> >> > To answer your question, Powell, my pseudo-proposal is kind of a funny
> >> option because once the feature is stable, then we wouldn't need a
> switch
> >> any longer, so there is not need of a deprecation path, we just start
> >> ignoring it from the first beta release. Until it is beta, I'd say that
> >> default is disabled.
> >>
> >> I would argue that we need this even when it does become stable. To me
> >> this is not a quality issue so much as it is an auth issue. We want to
> >> make it simple for folks to run a vanilla/old ZK cluster and not worry
> >> about the security implications of having reconfig enabled.
> >>
> >> Patrick
> >>
> >> >
> >> > -Flavio
> >> >
> >> >> On 17 Mar 2016, at 17:44, powell molleti <powellm79@yahoo.com.INVALID
> >
> >> wrote:
> >> >>
> >> >> Hi Flavio,
> >> >>
> >> >> Generally do config options and command line args come under the same
> >> SLA as API?. I was assuming as such hence my question. Perhaps if the
> >> expectation is that this config option is temporary from get go then
> may be
> >> it is ok. The default for re-config support will be enabled or
> disabled?.
> >> >>
> >> >> I am just thinking from provisioning point of view when people
> generate
> >> config options etc.
> >> >>
> >> >> Thanks
> >> >> Powell.
> >> >>
> >> >>
> >> >>
> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
> fpj@apache.org>
> >> wrote:
> >> >> Hi Powell,
> >> >>
> >> >> I was thinking config file and system property server side. What's
> your
> >> concern with compatibility? The API itself wouldn't change, but the
> config
> >> option wouldn't exist in previous versions and would not exist either in
> >> later stable versions of 3.5.
> >> >>
> >> >> -Flavio
> >> >>
> >> >>
> >> >>
> >> >>> On 16 Mar 2016, at 22:08, powell molleti
> <po...@yahoo.com.INVALID>
> >> wrote:
> >> >>>
> >> >>> Will this option be supplied via config file/args/API?. Will this
> >> option be a temporary thing i.e what about its compatibility?.
> >> >>>
> >> >>> thanks
> >> >>> Powell.
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <
> fpj@apache.org>
> >> wrote:
> >> >>> The main issue to sort out is stability of the API. There is a
> >> security concern around the fact that clients can freely reconfigure the
> >> ensemble. If we follow the plan that Pat proposed some time ago:
> >> >>>
> >> >>>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >> <
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >> >
> >> >>>
> >> >>> Locking the API is the main step to move it to beta. Sorting out
> bugs
> >> is definitely necessary, but it isn't the main thing that is keeping
> 3.5 in
> >> alpha.
> >> >>>
> >> >>> About making it experimental, I was raising the option of having a
> >> switch that disables the API calls, not the code. The reason why that
> could
> >> work is that anyone using 3.5 who uses the "experimental" API must
> explicit
> >> turn on the switch and enable the calls. If they do it, they need to be
> >> aware that the API can change.
> >> >>>
> >> >>> I must say that I haven't really looked closely into doing it, and
> I'm
> >> not even entirely convinced that this is a good idea, but since Jason
> >> raised the point, I'm exploring options.
> >> >>>
> >> >>> -Flavio
> >> >>>
> >> >>>
> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com>
> wrote:
> >> >>>>
> >> >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
> >> only 3-4
> >> >>>> are related to reconfig. Given this, and the fact that it is run in
> >> >>>> production since 2012 in multiple companies, I don't think its more
> >> >>>> unstable than any other part of ZooKeeper.
> >> >>>>
> >> >>>> There are multiple reconfig-related bugs that turned out really
> >> difficult
> >> >>>> to debug without access to the actual system or at least to the
> Hudson
> >> >>>> machines where some tests are failing. In the past, Michi and I
> >> physically
> >> >>>> went to Hortonworks to debug one such issue, but this is of course
> >> not a
> >> >>>> good method, and we weren't able to arrange such a visit again.
> >> >>>>
> >> >>>> Regarding making it optional - the reconfig logic has several
> >> different
> >> >>>> parts, some would be really difficult to disable using a
> configuration
> >> >>>> parameter. But the actual dynamic expansion of the cluster is
> >> triggered by
> >> >>>> the reconfig command, so it should not affect users who don't
> invoke
> >> it.
> >> >>>>
> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <
> fpj@apache.org>
> >> wrote:
> >> >>>>
> >> >>>>> I suppose we could give it a try. How do other folks feel about
> it?
> >> >>>>>
> >> >>>>> -Flavio
> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
> >> >>>>>
> >> >>>>>> So, you could enable the dynamic reconfiguration feature behind a
> >> config
> >> >>>>>> option, and document that it should only be enabled
> experimentally,
> >> use
> >> >>>>> at
> >> >>>>>> your own risk.  Keep it off by default.  Allow only static
> config by
> >> >>>>>> default, until it's stable?
> >> >>>>>>
> >> >>>>>> Jason
> >> >>>>>>
> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <
> fpj@apache.org>
> >> >>>>> wrote:
> >> >>>>>>
> >> >>>>>>> Hi Jason,
> >> >>>>>>>
> >> >>>>>>> The consumer in Kafka is pretty independent from the core
> >> (brokers),
> >> >>>>>>> that's how that project manages to make such a separation. We
> don't
> >> >>>>> have
> >> >>>>>>> the same with ZooKeeper as the feature we are talking about is
> >> part of
> >> >>>>>> the
> >> >>>>>>> server and the only way I see of doing what you say is to turn
> off
> >> >>>>>>> features. More specifically, we'd need to disable the reconfig
> API
> >> and
> >> >>>>> do
> >> >>>>>>> not allow any change to the configuration, even though the code
> is
> >> >>>>> there.
> >> >>>>>>>
> >> >>>>>>> Reconfig here refers to the ability of changing the
> configuration
> >> of an
> >> >>>>>>> ensemble (e.g., changing the set of servers).
> >> >>>>>>>
> >> >>>>>>> -Flavio
> >> >>>>>>>
> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com>
> >> wrote:
> >> >>>>>>>>
> >> >>>>>>>> So, it would seem sensible to me to have a release where all
> >> features
> >> >>>>>> are
> >> >>>>>>>> stable, except where noted.  E.g. mark certain features as only
> >> >>>>> 'alpha
> >> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's
> possible
> >> to
> >> >>>>>>> happily
> >> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
> >> >>>>>>>>
> >> >>>>>>>> There's precedent for doing this sort of thing in other
> projects,
> >> >>>>> e.g.
> >> >>>>>> in
> >> >>>>>>>> Kafka, they've had several release where a new "Consumer API"
> is
> >> >>>>>> shipped
> >> >>>>>>>> that is available for beta-testing, but you can still just use
> the
> >> >>>>>> older
> >> >>>>>>>> stable consumer api, etc.
> >> >>>>>>>>
> >> >>>>>>>> Jason
> >> >>>>>>>>
> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> >> >>>>>>> <powellm79@yahoo.com.invalid
> >> >>>>>>>>> wrote:
> >> >>>>>>>>
> >> >>>>>>>>> Hi Doug,
> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using it?.
> Or
> >> have
> >> >>>>>> you
> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being
> >> labeled as
> >> >>>>>>> alpha
> >> >>>>>>>>> might not be fair, since it is far more stable then what most
> >> people
> >> >>>>>>>>> associate an alpha release to be.
> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it will
> just
> >> work
> >> >>>>>> for
> >> >>>>>>>>> you?.
> >> >>>>>>>>> There are many examples of 3.5.X being used in productions
> from
> >> my
> >> >>>>>>> limited
> >> >>>>>>>>> knowledge.
> >> >>>>>>>>> ThanksPowell.
> >> >>>>>>>>>
> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> >> >>>>>>> fpj@apache.org>
> >> >>>>>>>>> wrote:
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>> None of us expected the reconfig changes to take this long to
> >> >>>>>> stabilize.
> >> >>>>>>>>> Until we get there, I don't think we can do anything else with
> >> 3.5.
> >> >>>>>> The
> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable
> >> rather
> >> >>>>>>> than
> >> >>>>>>>>> trying to work around it.
> >> >>>>>>>>>
> >> >>>>>>>>> There are lots of people interested in seeing 3.5 stable, and
> if
> >> we
> >> >>>>>> get
> >> >>>>>>>>> everyone to contribute more patches and code reviews, we
> should
> >> be
> >> >>>>>> able
> >> >>>>>>> to
> >> >>>>>>>>> do it sooner. After all, it is a community based effort, so
> the
> >> >>>>>>> community
> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> >> >>>>>>>>>
> >> >>>>>>>>> -Flavio
> >> >>>>>>>>>
> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> >> cnauroth@hortonworks.com>
> >> >>>>>>>>> wrote:
> >> >>>>>>>>>>
> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since
> >> 3.4 is
> >> >>>>>> the
> >> >>>>>>>>>> stable maintenance line, we are very conservative about
> >> >>>>> back-porting
> >> >>>>>> to
> >> >>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes
> and
> >> >>>>> not
> >> >>>>>>>>>> introduce any new features in the 3.4 line.  This is a
> matter of
> >> >>>>>>> managing
> >> >>>>>>>>>> risk.
> >> >>>>>>>>>>
> >> >>>>>>>>>> Jason, your question about release cadence is a fair one.  If
> >> it's
> >> >>>>>> any
> >> >>>>>>>>>> consolation, we are now taking the approach of trying to
> limit
> >> the
> >> >>>>>>> scope
> >> >>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us
> to
> >> >>>>> focus
> >> >>>>>> on
> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public
> >> APIs.  I
> >> >>>>>>> think
> >> >>>>>>>>>> this will help us accelerate the releases into beta and
> eventual
> >> >>>>> GA.
> >> >>>>>>>>>>
> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
> >> >>>>>> thoughts
> >> >>>>>>>>>> from more experienced committers and PMC members about your
> >> >>>>> proposal
> >> >>>>>> to
> >> >>>>>>>>>> try to cut a stable release for a limited subset of what is
> in
> >> >>>>>>> branch-3.5
> >> >>>>>>>>>> now.  My instinct is that it would be challenging to
> cherry-pick
> >> >>>>> out
> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would
> become
> >> >>>>>>> another
> >> >>>>>>>>>> release line for an already resource-constrained volunteer
> >> staff to
> >> >>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to
> >> overall
> >> >>>>>> 3.5
> >> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
> >> >>>>>>> "vanished"
> >> >>>>>>>>>> because of not meeting some stability criteria would be
> >> >>>>> undesirable.
> >> >>>>>>>>>>
> >> >>>>>>>>>> --Chris Nauroth
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com>
> >> wrote:
> >> >>>>>>>>>>
> >> >>>>>>>>>>> Chris,
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
> >> >>>>> others
> >> >>>>>>>>> (e.g.
> >> >>>>>>>>>>> if we don't care about certain new features, is it
> relatively
> >> >>>>>> stable)?
> >> >>>>>>>>>>> Would it be possible to cut out a version that only has the
> >> bits
> >> >>>>> we
> >> >>>>>>>>> think
> >> >>>>>>>>>>> are stable (and release that)?
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> From that timeline, and the historic release cadence, it
> would
> >> >>>>> seem
> >> >>>>>> to
> >> >>>>>>>>> be
> >> >>>>>>>>>>> a
> >> >>>>>>>>>>> years away before we get to the stable release?
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Thanks,
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Jason
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> >> >>>>>>>>> cnauroth@hortonworks.com>
> >> >>>>>>>>>>> wrote:
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>> Hello Doug,
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> At this point, I think we're still pretty far away from
> >> >>>>> declaring a
> >> >>>>>>>>>>>> stable
> >> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough
> >> that
> >> >>>>>>> anyone
> >> >>>>>>>>>>>> can
> >> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
> >> describes
> >> >>>>> the
> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> https://s.apache.org/ADK1
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
> >> >>>>>> resolving a
> >> >>>>>>>>>>>> few
> >> >>>>>>>>>>>> more blockers before we produce a release candidate.
> >> Hopefully
> >> >>>>>> that
> >> >>>>>>>>>>>> will
> >> >>>>>>>>>>>> get done in the next few weeks.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> --Chris Nauroth
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>> I know it's only been a few months, but I was wondering if
> >> there
> >> >>>>>>> was a
> >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is
> >> there
> >> >>>>>> any
> >> >>>>>>>>>>>>> chance
> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another
> person
> >> >>>>>> looking
> >> >>>>>>>>> to
> >> >>>>>>>>>>>>> have
> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you do!
> :)
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> --
> >> >>>>>>>>>>>>> View this message in context:
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>
> >> >>>>>>
> >> >>>>>
> >>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >> >>>>>>>>>>>> e
> >> >>>>>>>>>>>>> -tp7581744p7582136.html
> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
> >> Nabble.com.
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>
> >> >>>>>
> >> >
> >>
>

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <sh...@gmail.com> wrote:
> Hi Patrick, Flavio,
>
> Since there seems to be consensus on this, I can add this flag, unless
> someone else wants to. I assume that getConfig should still work regardless
> of the flag ? is there a security concern with clients knowing the list of
> servers?
>

We've always hidden that detail from users. We don't even expose which
server you're connected to today. I remember Ben (and perhaps Flavio?)
highlighting this as important to maintain although I'm not super
familiar with the specifics on why. It made sense to me though from
the perspective that we don't want clients to make assumptions that
probably shouldn't.

My thinking is that we should 1) add a config option to enable
reconfig (off by default), 2) move reconfig specific functionality
from ZooKeeper.java (including getconfig) into an "admin" package,
within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
when folks do want to enable reconfig and are also worried about auth.
(e.g. turn on kerb)

Again, I don't see any of this as a quality issue personally. As such
I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
were interested in doing such a release. Adjusting the API should be
done before we move to "beta" though. Although that seems like a
pretty mechanical (eclipse/idea) type refactoring?

Patrick

> Cheers,
> Alex
> On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>
>> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fp...@apache.org> wrote:
>> > I gotta say that I'm not super excited about this option, but for some
>> reason I ended up carrying the flag. To recap, I just raised this option
>> because it seems that there are folks interested in features in 3.5 like
>> SSL and not necessarily in reconfiguration. SSL is important and to take
>> Kafka as an example, it sucks that we can't have a whole set up using SSL.
>> For ZK, the real questions are:
>> >
>> > 1- how fast can we make 3.5 stable?
>> > 2- would it be faster if we have a way of disabling reconfiguration?
>> > 3- would enough users care about a stable 3.5 that has reconfiguration
>> disabled?
>> >
>> > It is taking a long time to get 3.5 to beta. There has been some good
>> activity around 3.5.2 release, which is a great step, but it is unclear
>> when 3.5.3 is going to come and if we will be able to make 3.5 beta then.
>> Frankly, disabling reconfiguration sounds undesirable because it is an
>> important feature, but I currently don't use it in production, so from a
>> practical point of view, I can go both ways. Whether we go through the
>> trouble of doing 2 depends on users interested in that option and folks
>> willing to implement it.
>> >
>> > To answer your question, Powell, my pseudo-proposal is kind of a funny
>> option because once the feature is stable, then we wouldn't need a switch
>> any longer, so there is not need of a deprecation path, we just start
>> ignoring it from the first beta release. Until it is beta, I'd say that
>> default is disabled.
>>
>> I would argue that we need this even when it does become stable. To me
>> this is not a quality issue so much as it is an auth issue. We want to
>> make it simple for folks to run a vanilla/old ZK cluster and not worry
>> about the security implications of having reconfig enabled.
>>
>> Patrick
>>
>> >
>> > -Flavio
>> >
>> >> On 17 Mar 2016, at 17:44, powell molleti <po...@yahoo.com.INVALID>
>> wrote:
>> >>
>> >> Hi Flavio,
>> >>
>> >> Generally do config options and command line args come under the same
>> SLA as API?. I was assuming as such hence my question. Perhaps if the
>> expectation is that this config option is temporary from get go then may be
>> it is ok. The default for re-config support will be enabled or disabled?.
>> >>
>> >> I am just thinking from provisioning point of view when people generate
>> config options etc.
>> >>
>> >> Thanks
>> >> Powell.
>> >>
>> >>
>> >>
>> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <fp...@apache.org>
>> wrote:
>> >> Hi Powell,
>> >>
>> >> I was thinking config file and system property server side. What's your
>> concern with compatibility? The API itself wouldn't change, but the config
>> option wouldn't exist in previous versions and would not exist either in
>> later stable versions of 3.5.
>> >>
>> >> -Flavio
>> >>
>> >>
>> >>
>> >>> On 16 Mar 2016, at 22:08, powell molleti <po...@yahoo.com.INVALID>
>> wrote:
>> >>>
>> >>> Will this option be supplied via config file/args/API?. Will this
>> option be a temporary thing i.e what about its compatibility?.
>> >>>
>> >>> thanks
>> >>> Powell.
>> >>>
>> >>>
>> >>>
>> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <fp...@apache.org>
>> wrote:
>> >>> The main issue to sort out is stability of the API. There is a
>> security concern around the fact that clients can freely reconfigure the
>> ensemble. If we follow the plan that Pat proposed some time ago:
>> >>>
>> >>>
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> <
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> >
>> >>>
>> >>> Locking the API is the main step to move it to beta. Sorting out bugs
>> is definitely necessary, but it isn't the main thing that is keeping 3.5 in
>> alpha.
>> >>>
>> >>> About making it experimental, I was raising the option of having a
>> switch that disables the API calls, not the code. The reason why that could
>> work is that anyone using 3.5 who uses the "experimental" API must explicit
>> turn on the switch and enable the calls. If they do it, they need to be
>> aware that the API can change.
>> >>>
>> >>> I must say that I haven't really looked closely into doing it, and I'm
>> not even entirely convinced that this is a good idea, but since Jason
>> raised the point, I'm exploring options.
>> >>>
>> >>> -Flavio
>> >>>
>> >>>
>> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>> >>>>
>> >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
>> only 3-4
>> >>>> are related to reconfig. Given this, and the fact that it is run in
>> >>>> production since 2012 in multiple companies, I don't think its more
>> >>>> unstable than any other part of ZooKeeper.
>> >>>>
>> >>>> There are multiple reconfig-related bugs that turned out really
>> difficult
>> >>>> to debug without access to the actual system or at least to the Hudson
>> >>>> machines where some tests are failing. In the past, Michi and I
>> physically
>> >>>> went to Hortonworks to debug one such issue, but this is of course
>> not a
>> >>>> good method, and we weren't able to arrange such a visit again.
>> >>>>
>> >>>> Regarding making it optional - the reconfig logic has several
>> different
>> >>>> parts, some would be really difficult to disable using a configuration
>> >>>> parameter. But the actual dynamic expansion of the cluster is
>> triggered by
>> >>>> the reconfig command, so it should not affect users who don't invoke
>> it.
>> >>>>
>> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org>
>> wrote:
>> >>>>
>> >>>>> I suppose we could give it a try. How do other folks feel about it?
>> >>>>>
>> >>>>> -Flavio
>> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>> >>>>>
>> >>>>>> So, you could enable the dynamic reconfiguration feature behind a
>> config
>> >>>>>> option, and document that it should only be enabled experimentally,
>> use
>> >>>>> at
>> >>>>>> your own risk.  Keep it off by default.  Allow only static config by
>> >>>>>> default, until it's stable?
>> >>>>>>
>> >>>>>> Jason
>> >>>>>>
>> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>>> Hi Jason,
>> >>>>>>>
>> >>>>>>> The consumer in Kafka is pretty independent from the core
>> (brokers),
>> >>>>>>> that's how that project manages to make such a separation. We don't
>> >>>>> have
>> >>>>>>> the same with ZooKeeper as the feature we are talking about is
>> part of
>> >>>>>> the
>> >>>>>>> server and the only way I see of doing what you say is to turn off
>> >>>>>>> features. More specifically, we'd need to disable the reconfig API
>> and
>> >>>>> do
>> >>>>>>> not allow any change to the configuration, even though the code is
>> >>>>> there.
>> >>>>>>>
>> >>>>>>> Reconfig here refers to the ability of changing the configuration
>> of an
>> >>>>>>> ensemble (e.g., changing the set of servers).
>> >>>>>>>
>> >>>>>>> -Flavio
>> >>>>>>>
>> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com>
>> wrote:
>> >>>>>>>>
>> >>>>>>>> So, it would seem sensible to me to have a release where all
>> features
>> >>>>>> are
>> >>>>>>>> stable, except where noted.  E.g. mark certain features as only
>> >>>>> 'alpha
>> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible
>> to
>> >>>>>>> happily
>> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>> >>>>>>>>
>> >>>>>>>> There's precedent for doing this sort of thing in other projects,
>> >>>>> e.g.
>> >>>>>> in
>> >>>>>>>> Kafka, they've had several release where a new "Consumer API" is
>> >>>>>> shipped
>> >>>>>>>> that is available for beta-testing, but you can still just use the
>> >>>>>> older
>> >>>>>>>> stable consumer api, etc.
>> >>>>>>>>
>> >>>>>>>> Jason
>> >>>>>>>>
>> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>> >>>>>>> <powellm79@yahoo.com.invalid
>> >>>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi Doug,
>> >>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or
>> have
>> >>>>>> you
>> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being
>> labeled as
>> >>>>>>> alpha
>> >>>>>>>>> might not be fair, since it is far more stable then what most
>> people
>> >>>>>>>>> associate an alpha release to be.
>> >>>>>>>>> Perhaps if you do not use re-config feature may be it will just
>> work
>> >>>>>> for
>> >>>>>>>>> you?.
>> >>>>>>>>> There are many examples of 3.5.X being used in productions from
>> my
>> >>>>>>> limited
>> >>>>>>>>> knowledge.
>> >>>>>>>>> ThanksPowell.
>> >>>>>>>>>
>> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>> >>>>>>> fpj@apache.org>
>> >>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> None of us expected the reconfig changes to take this long to
>> >>>>>> stabilize.
>> >>>>>>>>> Until we get there, I don't think we can do anything else with
>> 3.5.
>> >>>>>> The
>> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable
>> rather
>> >>>>>>> than
>> >>>>>>>>> trying to work around it.
>> >>>>>>>>>
>> >>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if
>> we
>> >>>>>> get
>> >>>>>>>>> everyone to contribute more patches and code reviews, we should
>> be
>> >>>>>> able
>> >>>>>>> to
>> >>>>>>>>> do it sooner. After all, it is a community based effort, so the
>> >>>>>>> community
>> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>> >>>>>>>>>
>> >>>>>>>>> -Flavio
>> >>>>>>>>>
>> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
>> cnauroth@hortonworks.com>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since
>> 3.4 is
>> >>>>>> the
>> >>>>>>>>>> stable maintenance line, we are very conservative about
>> >>>>> back-porting
>> >>>>>> to
>> >>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>> >>>>> not
>> >>>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>> >>>>>>> managing
>> >>>>>>>>>> risk.
>> >>>>>>>>>>
>> >>>>>>>>>> Jason, your question about release cadence is a fair one.  If
>> it's
>> >>>>>> any
>> >>>>>>>>>> consolation, we are now taking the approach of trying to limit
>> the
>> >>>>>>> scope
>> >>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>> >>>>> focus
>> >>>>>> on
>> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public
>> APIs.  I
>> >>>>>>> think
>> >>>>>>>>>> this will help us accelerate the releases into beta and eventual
>> >>>>> GA.
>> >>>>>>>>>>
>> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>> >>>>>> thoughts
>> >>>>>>>>>> from more experienced committers and PMC members about your
>> >>>>> proposal
>> >>>>>> to
>> >>>>>>>>>> try to cut a stable release for a limited subset of what is in
>> >>>>>>> branch-3.5
>> >>>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>> >>>>> out
>> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>> >>>>>>> another
>> >>>>>>>>>> release line for an already resource-constrained volunteer
>> staff to
>> >>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to
>> overall
>> >>>>>> 3.5
>> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>> >>>>>>> "vanished"
>> >>>>>>>>>> because of not meeting some stability criteria would be
>> >>>>> undesirable.
>> >>>>>>>>>>
>> >>>>>>>>>> --Chris Nauroth
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com>
>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Chris,
>> >>>>>>>>>>>
>> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>> >>>>> others
>> >>>>>>>>> (e.g.
>> >>>>>>>>>>> if we don't care about certain new features, is it relatively
>> >>>>>> stable)?
>> >>>>>>>>>>> Would it be possible to cut out a version that only has the
>> bits
>> >>>>> we
>> >>>>>>>>> think
>> >>>>>>>>>>> are stable (and release that)?
>> >>>>>>>>>>>
>> >>>>>>>>>>> From that timeline, and the historic release cadence, it would
>> >>>>> seem
>> >>>>>> to
>> >>>>>>>>> be
>> >>>>>>>>>>> a
>> >>>>>>>>>>> years away before we get to the stable release?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>>
>> >>>>>>>>>>> Jason
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>> >>>>>>>>> cnauroth@hortonworks.com>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hello Doug,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> At this point, I think we're still pretty far away from
>> >>>>> declaring a
>> >>>>>>>>>>>> stable
>> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough
>> that
>> >>>>>>> anyone
>> >>>>>>>>>>>> can
>> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
>> describes
>> >>>>> the
>> >>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> https://s.apache.org/ADK1
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>> >>>>>> resolving a
>> >>>>>>>>>>>> few
>> >>>>>>>>>>>> more blockers before we produce a release candidate.
>> Hopefully
>> >>>>>> that
>> >>>>>>>>>>>> will
>> >>>>>>>>>>>> get done in the next few weeks.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --Chris Nauroth
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> I know it's only been a few months, but I was wondering if
>> there
>> >>>>>>> was a
>> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is
>> there
>> >>>>>> any
>> >>>>>>>>>>>>> chance
>> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>> >>>>>> looking
>> >>>>>>>>> to
>> >>>>>>>>>>>>> have
>> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> View this message in context:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>> >>>>>>>>>>>> e
>> >>>>>>>>>>>>> -tp7581744p7582136.html
>> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
>> Nabble.com.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >
>>

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

Hi Patrick, Flavio,

Since there seems to be consensus on this, I can add this flag, unless
someone else wants to. I assume that getConfig should still work regardless
of the flag ? is there a security concern with clients knowing the list of
servers?

Cheers,
Alex
On Mar 21, 2016 8:34 PM, "Patrick Hunt" <ph...@apache.org> wrote:

> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fp...@apache.org> wrote:
> > I gotta say that I'm not super excited about this option, but for some
> reason I ended up carrying the flag. To recap, I just raised this option
> because it seems that there are folks interested in features in 3.5 like
> SSL and not necessarily in reconfiguration. SSL is important and to take
> Kafka as an example, it sucks that we can't have a whole set up using SSL.
> For ZK, the real questions are:
> >
> > 1- how fast can we make 3.5 stable?
> > 2- would it be faster if we have a way of disabling reconfiguration?
> > 3- would enough users care about a stable 3.5 that has reconfiguration
> disabled?
> >
> > It is taking a long time to get 3.5 to beta. There has been some good
> activity around 3.5.2 release, which is a great step, but it is unclear
> when 3.5.3 is going to come and if we will be able to make 3.5 beta then.
> Frankly, disabling reconfiguration sounds undesirable because it is an
> important feature, but I currently don't use it in production, so from a
> practical point of view, I can go both ways. Whether we go through the
> trouble of doing 2 depends on users interested in that option and folks
> willing to implement it.
> >
> > To answer your question, Powell, my pseudo-proposal is kind of a funny
> option because once the feature is stable, then we wouldn't need a switch
> any longer, so there is not need of a deprecation path, we just start
> ignoring it from the first beta release. Until it is beta, I'd say that
> default is disabled.
>
> I would argue that we need this even when it does become stable. To me
> this is not a quality issue so much as it is an auth issue. We want to
> make it simple for folks to run a vanilla/old ZK cluster and not worry
> about the security implications of having reconfig enabled.
>
> Patrick
>
> >
> > -Flavio
> >
> >> On 17 Mar 2016, at 17:44, powell molleti <po...@yahoo.com.INVALID>
> wrote:
> >>
> >> Hi Flavio,
> >>
> >> Generally do config options and command line args come under the same
> SLA as API?. I was assuming as such hence my question. Perhaps if the
> expectation is that this config option is temporary from get go then may be
> it is ok. The default for re-config support will be enabled or disabled?.
> >>
> >> I am just thinking from provisioning point of view when people generate
> config options etc.
> >>
> >> Thanks
> >> Powell.
> >>
> >>
> >>
> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <fp...@apache.org>
> wrote:
> >> Hi Powell,
> >>
> >> I was thinking config file and system property server side. What's your
> concern with compatibility? The API itself wouldn't change, but the config
> option wouldn't exist in previous versions and would not exist either in
> later stable versions of 3.5.
> >>
> >> -Flavio
> >>
> >>
> >>
> >>> On 16 Mar 2016, at 22:08, powell molleti <po...@yahoo.com.INVALID>
> wrote:
> >>>
> >>> Will this option be supplied via config file/args/API?. Will this
> option be a temporary thing i.e what about its compatibility?.
> >>>
> >>> thanks
> >>> Powell.
> >>>
> >>>
> >>>
> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <fp...@apache.org>
> wrote:
> >>> The main issue to sort out is stability of the API. There is a
> security concern around the fact that clients can freely reconfigure the
> ensemble. If we follow the plan that Pat proposed some time ago:
> >>>
> >>>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> <
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >
> >>>
> >>> Locking the API is the main step to move it to beta. Sorting out bugs
> is definitely necessary, but it isn't the main thing that is keeping 3.5 in
> alpha.
> >>>
> >>> About making it experimental, I was raising the option of having a
> switch that disables the API calls, not the code. The reason why that could
> work is that anyone using 3.5 who uses the "experimental" API must explicit
> turn on the switch and enable the calls. If they do it, they need to be
> aware that the API can change.
> >>>
> >>> I must say that I haven't really looked closely into doing it, and I'm
> not even entirely convinced that this is a good idea, but since Jason
> raised the point, I'm exploring options.
> >>>
> >>> -Flavio
> >>>
> >>>
> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
> >>>>
> >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
> only 3-4
> >>>> are related to reconfig. Given this, and the fact that it is run in
> >>>> production since 2012 in multiple companies, I don't think its more
> >>>> unstable than any other part of ZooKeeper.
> >>>>
> >>>> There are multiple reconfig-related bugs that turned out really
> difficult
> >>>> to debug without access to the actual system or at least to the Hudson
> >>>> machines where some tests are failing. In the past, Michi and I
> physically
> >>>> went to Hortonworks to debug one such issue, but this is of course
> not a
> >>>> good method, and we weren't able to arrange such a visit again.
> >>>>
> >>>> Regarding making it optional - the reconfig logic has several
> different
> >>>> parts, some would be really difficult to disable using a configuration
> >>>> parameter. But the actual dynamic expansion of the cluster is
> triggered by
> >>>> the reconfig command, so it should not affect users who don't invoke
> it.
> >>>>
> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org>
> wrote:
> >>>>
> >>>>> I suppose we could give it a try. How do other folks feel about it?
> >>>>>
> >>>>> -Flavio
> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
> >>>>>
> >>>>>> So, you could enable the dynamic reconfiguration feature behind a
> config
> >>>>>> option, and document that it should only be enabled experimentally,
> use
> >>>>> at
> >>>>>> your own risk.  Keep it off by default.  Allow only static config by
> >>>>>> default, until it's stable?
> >>>>>>
> >>>>>> Jason
> >>>>>>
> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Hi Jason,
> >>>>>>>
> >>>>>>> The consumer in Kafka is pretty independent from the core
> (brokers),
> >>>>>>> that's how that project manages to make such a separation. We don't
> >>>>> have
> >>>>>>> the same with ZooKeeper as the feature we are talking about is
> part of
> >>>>>> the
> >>>>>>> server and the only way I see of doing what you say is to turn off
> >>>>>>> features. More specifically, we'd need to disable the reconfig API
> and
> >>>>> do
> >>>>>>> not allow any change to the configuration, even though the code is
> >>>>> there.
> >>>>>>>
> >>>>>>> Reconfig here refers to the ability of changing the configuration
> of an
> >>>>>>> ensemble (e.g., changing the set of servers).
> >>>>>>>
> >>>>>>> -Flavio
> >>>>>>>
> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com>
> wrote:
> >>>>>>>>
> >>>>>>>> So, it would seem sensible to me to have a release where all
> features
> >>>>>> are
> >>>>>>>> stable, except where noted.  E.g. mark certain features as only
> >>>>> 'alpha
> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible
> to
> >>>>>>> happily
> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
> >>>>>>>>
> >>>>>>>> There's precedent for doing this sort of thing in other projects,
> >>>>> e.g.
> >>>>>> in
> >>>>>>>> Kafka, they've had several release where a new "Consumer API" is
> >>>>>> shipped
> >>>>>>>> that is available for beta-testing, but you can still just use the
> >>>>>> older
> >>>>>>>> stable consumer api, etc.
> >>>>>>>>
> >>>>>>>> Jason
> >>>>>>>>
> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> >>>>>>> <powellm79@yahoo.com.invalid
> >>>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi Doug,
> >>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or
> have
> >>>>>> you
> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being
> labeled as
> >>>>>>> alpha
> >>>>>>>>> might not be fair, since it is far more stable then what most
> people
> >>>>>>>>> associate an alpha release to be.
> >>>>>>>>> Perhaps if you do not use re-config feature may be it will just
> work
> >>>>>> for
> >>>>>>>>> you?.
> >>>>>>>>> There are many examples of 3.5.X being used in productions from
> my
> >>>>>>> limited
> >>>>>>>>> knowledge.
> >>>>>>>>> ThanksPowell.
> >>>>>>>>>
> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> >>>>>>> fpj@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> None of us expected the reconfig changes to take this long to
> >>>>>> stabilize.
> >>>>>>>>> Until we get there, I don't think we can do anything else with
> 3.5.
> >>>>>> The
> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable
> rather
> >>>>>>> than
> >>>>>>>>> trying to work around it.
> >>>>>>>>>
> >>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if
> we
> >>>>>> get
> >>>>>>>>> everyone to contribute more patches and code reviews, we should
> be
> >>>>>> able
> >>>>>>> to
> >>>>>>>>> do it sooner. After all, it is a community based effort, so the
> >>>>>>> community
> >>>>>>>>> shouldn't rely on just 2-3 people doing the work.
> >>>>>>>>>
> >>>>>>>>> -Flavio
> >>>>>>>>>
> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> cnauroth@hortonworks.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since
> 3.4 is
> >>>>>> the
> >>>>>>>>>> stable maintenance line, we are very conservative about
> >>>>> back-porting
> >>>>>> to
> >>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
> >>>>> not
> >>>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
> >>>>>>> managing
> >>>>>>>>>> risk.
> >>>>>>>>>>
> >>>>>>>>>> Jason, your question about release cadence is a fair one.  If
> it's
> >>>>>> any
> >>>>>>>>>> consolation, we are now taking the approach of trying to limit
> the
> >>>>>>> scope
> >>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
> >>>>> focus
> >>>>>> on
> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public
> APIs.  I
> >>>>>>> think
> >>>>>>>>>> this will help us accelerate the releases into beta and eventual
> >>>>> GA.
> >>>>>>>>>>
> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
> >>>>>> thoughts
> >>>>>>>>>> from more experienced committers and PMC members about your
> >>>>> proposal
> >>>>>> to
> >>>>>>>>>> try to cut a stable release for a limited subset of what is in
> >>>>>>> branch-3.5
> >>>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
> >>>>> out
> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
> >>>>>>> another
> >>>>>>>>>> release line for an already resource-constrained volunteer
> staff to
> >>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to
> overall
> >>>>>> 3.5
> >>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
> >>>>>>> "vanished"
> >>>>>>>>>> because of not meeting some stability criteria would be
> >>>>> undesirable.
> >>>>>>>>>>
> >>>>>>>>>> --Chris Nauroth
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com>
> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Chris,
> >>>>>>>>>>>
> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
> >>>>> others
> >>>>>>>>> (e.g.
> >>>>>>>>>>> if we don't care about certain new features, is it relatively
> >>>>>> stable)?
> >>>>>>>>>>> Would it be possible to cut out a version that only has the
> bits
> >>>>> we
> >>>>>>>>> think
> >>>>>>>>>>> are stable (and release that)?
> >>>>>>>>>>>
> >>>>>>>>>>> From that timeline, and the historic release cadence, it would
> >>>>> seem
> >>>>>> to
> >>>>>>>>> be
> >>>>>>>>>>> a
> >>>>>>>>>>> years away before we get to the stable release?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>>
> >>>>>>>>>>> Jason
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> >>>>>>>>> cnauroth@hortonworks.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hello Doug,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks for your interest in the SSL feature!
> >>>>>>>>>>>>
> >>>>>>>>>>>> At this point, I think we're still pretty far away from
> >>>>> declaring a
> >>>>>>>>>>>> stable
> >>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough
> that
> >>>>>>> anyone
> >>>>>>>>>>>> can
> >>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
> describes
> >>>>> the
> >>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
> >>>>>>>>>>>>
> >>>>>>>>>>>> https://s.apache.org/ADK1
> >>>>>>>>>>>>
> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
> >>>>>> resolving a
> >>>>>>>>>>>> few
> >>>>>>>>>>>> more blockers before we produce a release candidate.
> Hopefully
> >>>>>> that
> >>>>>>>>>>>> will
> >>>>>>>>>>>> get done in the next few weeks.
> >>>>>>>>>>>>
> >>>>>>>>>>>> --Chris Nauroth
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> I know it's only been a few months, but I was wondering if
> there
> >>>>>>> was a
> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is
> there
> >>>>>> any
> >>>>>>>>>>>>> chance
> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
> >>>>>> looking
> >>>>>>>>> to
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> View this message in context:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >>>>>>>>>>>> e
> >>>>>>>>>>>>> -tp7581744p7582136.html
> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
> Nabble.com.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >
>

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fp...@apache.org> wrote:
> I gotta say that I'm not super excited about this option, but for some reason I ended up carrying the flag. To recap, I just raised this option because it seems that there are folks interested in features in 3.5 like SSL and not necessarily in reconfiguration. SSL is important and to take Kafka as an example, it sucks that we can't have a whole set up using SSL. For ZK, the real questions are:
>
> 1- how fast can we make 3.5 stable?
> 2- would it be faster if we have a way of disabling reconfiguration?
> 3- would enough users care about a stable 3.5 that has reconfiguration disabled?
>
> It is taking a long time to get 3.5 to beta. There has been some good activity around 3.5.2 release, which is a great step, but it is unclear when 3.5.3 is going to come and if we will be able to make 3.5 beta then. Frankly, disabling reconfiguration sounds undesirable because it is an important feature, but I currently don't use it in production, so from a practical point of view, I can go both ways. Whether we go through the trouble of doing 2 depends on users interested in that option and folks willing to implement it.
>
> To answer your question, Powell, my pseudo-proposal is kind of a funny option because once the feature is stable, then we wouldn't need a switch any longer, so there is not need of a deprecation path, we just start ignoring it from the first beta release. Until it is beta, I'd say that default is disabled.

I would argue that we need this even when it does become stable. To me
this is not a quality issue so much as it is an auth issue. We want to
make it simple for folks to run a vanilla/old ZK cluster and not worry
about the security implications of having reconfig enabled.

Patrick

>
> -Flavio
>
>> On 17 Mar 2016, at 17:44, powell molleti <po...@yahoo.com.INVALID> wrote:
>>
>> Hi Flavio,
>>
>> Generally do config options and command line args come under the same SLA as API?. I was assuming as such hence my question. Perhaps if the expectation is that this config option is temporary from get go then may be it is ok. The default for re-config support will be enabled or disabled?.
>>
>> I am just thinking from provisioning point of view when people generate config options etc.
>>
>> Thanks
>> Powell.
>>
>>
>>
>> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <fp...@apache.org> wrote:
>> Hi Powell,
>>
>> I was thinking config file and system property server side. What's your concern with compatibility? The API itself wouldn't change, but the config option wouldn't exist in previous versions and would not exist either in later stable versions of 3.5.
>>
>> -Flavio
>>
>>
>>
>>> On 16 Mar 2016, at 22:08, powell molleti <po...@yahoo.com.INVALID> wrote:
>>>
>>> Will this option be supplied via config file/args/API?. Will this option be a temporary thing i.e what about its compatibility?.
>>>
>>> thanks
>>> Powell.
>>>
>>>
>>>
>>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
>>> The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:
>>>
>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>
>>>
>>> Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.
>>>
>>> About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.
>>>
>>> I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.
>>>
>>> -Flavio
>>>
>>>
>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>>>>
>>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
>>>> are related to reconfig. Given this, and the fact that it is run in
>>>> production since 2012 in multiple companies, I don't think its more
>>>> unstable than any other part of ZooKeeper.
>>>>
>>>> There are multiple reconfig-related bugs that turned out really difficult
>>>> to debug without access to the actual system or at least to the Hudson
>>>> machines where some tests are failing. In the past, Michi and I physically
>>>> went to Hortonworks to debug one such issue, but this is of course not a
>>>> good method, and we weren't able to arrange such a visit again.
>>>>
>>>> Regarding making it optional - the reconfig logic has several different
>>>> parts, some would be really difficult to disable using a configuration
>>>> parameter. But the actual dynamic expansion of the cluster is triggered by
>>>> the reconfig command, so it should not affect users who don't invoke it.
>>>>
>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
>>>>
>>>>> I suppose we could give it a try. How do other folks feel about it?
>>>>>
>>>>> -Flavio
>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>
>>>>>> So, you could enable the dynamic reconfiguration feature behind a config
>>>>>> option, and document that it should only be enabled experimentally, use
>>>>> at
>>>>>> your own risk.  Keep it off by default.  Allow only static config by
>>>>>> default, until it's stable?
>>>>>>
>>>>>> Jason
>>>>>>
>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>>>> wrote:
>>>>>>
>>>>>>> Hi Jason,
>>>>>>>
>>>>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>>>>> that's how that project manages to make such a separation. We don't
>>>>> have
>>>>>>> the same with ZooKeeper as the feature we are talking about is part of
>>>>>> the
>>>>>>> server and the only way I see of doing what you say is to turn off
>>>>>>> features. More specifically, we'd need to disable the reconfig API and
>>>>> do
>>>>>>> not allow any change to the configuration, even though the code is
>>>>> there.
>>>>>>>
>>>>>>> Reconfig here refers to the ability of changing the configuration of an
>>>>>>> ensemble (e.g., changing the set of servers).
>>>>>>>
>>>>>>> -Flavio
>>>>>>>
>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>>>>>
>>>>>>>> So, it would seem sensible to me to have a release where all features
>>>>>> are
>>>>>>>> stable, except where noted.  E.g. mark certain features as only
>>>>> 'alpha
>>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>>>>> happily
>>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>>>>>
>>>>>>>> There's precedent for doing this sort of thing in other projects,
>>>>> e.g.
>>>>>> in
>>>>>>>> Kafka, they've had several release where a new "Consumer API" is
>>>>>> shipped
>>>>>>>> that is available for beta-testing, but you can still just use the
>>>>>> older
>>>>>>>> stable consumer api, etc.
>>>>>>>>
>>>>>>>> Jason
>>>>>>>>
>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>>>> <powellm79@yahoo.com.invalid
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Doug,
>>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>>>>> you
>>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>>>>> alpha
>>>>>>>>> might not be fair, since it is far more stable then what most people
>>>>>>>>> associate an alpha release to be.
>>>>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>>>>> for
>>>>>>>>> you?.
>>>>>>>>> There are many examples of 3.5.X being used in productions from my
>>>>>>> limited
>>>>>>>>> knowledge.
>>>>>>>>> ThanksPowell.
>>>>>>>>>
>>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>>>>> fpj@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> None of us expected the reconfig changes to take this long to
>>>>>> stabilize.
>>>>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>>>>> The
>>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>>>>> than
>>>>>>>>> trying to work around it.
>>>>>>>>>
>>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>>>>> get
>>>>>>>>> everyone to contribute more patches and code reviews, we should be
>>>>>> able
>>>>>>> to
>>>>>>>>> do it sooner. After all, it is a community based effort, so the
>>>>>>> community
>>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>>>>
>>>>>>>>> -Flavio
>>>>>>>>>
>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>>>>> the
>>>>>>>>>> stable maintenance line, we are very conservative about
>>>>> back-porting
>>>>>> to
>>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>>>> not
>>>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>>>>> managing
>>>>>>>>>> risk.
>>>>>>>>>>
>>>>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>>>>> any
>>>>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>>>>> scope
>>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>>>> focus
>>>>>> on
>>>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>>>>> think
>>>>>>>>>> this will help us accelerate the releases into beta and eventual
>>>>> GA.
>>>>>>>>>>
>>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>>>>> thoughts
>>>>>>>>>> from more experienced committers and PMC members about your
>>>>> proposal
>>>>>> to
>>>>>>>>>> try to cut a stable release for a limited subset of what is in
>>>>>>> branch-3.5
>>>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>>>> out
>>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>>>>> another
>>>>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>>>>> 3.5
>>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>>>>> "vanished"
>>>>>>>>>> because of not meeting some stability criteria would be
>>>>> undesirable.
>>>>>>>>>>
>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Chris,
>>>>>>>>>>>
>>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>>>> others
>>>>>>>>> (e.g.
>>>>>>>>>>> if we don't care about certain new features, is it relatively
>>>>>> stable)?
>>>>>>>>>>> Would it be possible to cut out a version that only has the bits
>>>>> we
>>>>>>>>> think
>>>>>>>>>>> are stable (and release that)?
>>>>>>>>>>>
>>>>>>>>>>> From that timeline, and the historic release cadence, it would
>>>>> seem
>>>>>> to
>>>>>>>>> be
>>>>>>>>>>> a
>>>>>>>>>>> years away before we get to the stable release?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Jason
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello Doug,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>>>>
>>>>>>>>>>>> At this point, I think we're still pretty far away from
>>>>> declaring a
>>>>>>>>>>>> stable
>>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>>>>> anyone
>>>>>>>>>>>> can
>>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>>>>> the
>>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>>>>>
>>>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>>>>
>>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>>>>> resolving a
>>>>>>>>>>>> few
>>>>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>>>>> that
>>>>>>>>>>>> will
>>>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>>>>
>>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>>>>> was a
>>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>>>>> any
>>>>>>>>>>>>> chance
>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>>>>> looking
>>>>>>>>> to
>>>>>>>>>>>>> have
>>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>>>> e
>>>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>

Re: Zookeeper with SSL release date

[Flavio Junqueira <fp...@apache.org>]

I gotta say that I'm not super excited about this option, but for some reason I ended up carrying the flag. To recap, I just raised this option because it seems that there are folks interested in features in 3.5 like SSL and not necessarily in reconfiguration. SSL is important and to take Kafka as an example, it sucks that we can't have a whole set up using SSL. For ZK, the real questions are: 

1- how fast can we make 3.5 stable?
2- would it be faster if we have a way of disabling reconfiguration?
3- would enough users care about a stable 3.5 that has reconfiguration disabled?

It is taking a long time to get 3.5 to beta. There has been some good activity around 3.5.2 release, which is a great step, but it is unclear when 3.5.3 is going to come and if we will be able to make 3.5 beta then. Frankly, disabling reconfiguration sounds undesirable because it is an important feature, but I currently don't use it in production, so from a practical point of view, I can go both ways. Whether we go through the trouble of doing 2 depends on users interested in that option and folks willing to implement it.

To answer your question, Powell, my pseudo-proposal is kind of a funny option because once the feature is stable, then we wouldn't need a switch any longer, so there is not need of a deprecation path, we just start ignoring it from the first beta release. Until it is beta, I'd say that default is disabled.

-Flavio

> On 17 Mar 2016, at 17:44, powell molleti <po...@yahoo.com.INVALID> wrote:
> 
> Hi Flavio,
> 
> Generally do config options and command line args come under the same SLA as API?. I was assuming as such hence my question. Perhaps if the expectation is that this config option is temporary from get go then may be it is ok. The default for re-config support will be enabled or disabled?.
> 
> I am just thinking from provisioning point of view when people generate config options etc.
> 
> Thanks
> Powell.
> 
> 
> 
> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <fp...@apache.org> wrote:
> Hi Powell,
> 
> I was thinking config file and system property server side. What's your concern with compatibility? The API itself wouldn't change, but the config option wouldn't exist in previous versions and would not exist either in later stable versions of 3.5.
> 
> -Flavio
> 
> 
> 
>> On 16 Mar 2016, at 22:08, powell molleti <po...@yahoo.com.INVALID> wrote:
>> 
>> Will this option be supplied via config file/args/API?. Will this option be a temporary thing i.e what about its compatibility?.
>> 
>> thanks
>> Powell.
>> 
>> 
>> 
>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
>> The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:
>> 
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>
>> 
>> Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.
>> 
>> About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.
>> 
>> I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.
>> 
>> -Flavio
>> 
>> 
>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>>> 
>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
>>> are related to reconfig. Given this, and the fact that it is run in
>>> production since 2012 in multiple companies, I don't think its more
>>> unstable than any other part of ZooKeeper.
>>> 
>>> There are multiple reconfig-related bugs that turned out really difficult
>>> to debug without access to the actual system or at least to the Hudson
>>> machines where some tests are failing. In the past, Michi and I physically
>>> went to Hortonworks to debug one such issue, but this is of course not a
>>> good method, and we weren't able to arrange such a visit again.
>>> 
>>> Regarding making it optional - the reconfig logic has several different
>>> parts, some would be really difficult to disable using a configuration
>>> parameter. But the actual dynamic expansion of the cluster is triggered by
>>> the reconfig command, so it should not affect users who don't invoke it.
>>> 
>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
>>> 
>>>> I suppose we could give it a try. How do other folks feel about it?
>>>> 
>>>> -Flavio
>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>> 
>>>>> So, you could enable the dynamic reconfiguration feature behind a config
>>>>> option, and document that it should only be enabled experimentally, use
>>>> at
>>>>> your own risk.  Keep it off by default.  Allow only static config by
>>>>> default, until it's stable?
>>>>> 
>>>>> Jason
>>>>> 
>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>>> wrote:
>>>>> 
>>>>>> Hi Jason,
>>>>>> 
>>>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>>>> that's how that project manages to make such a separation. We don't
>>>> have
>>>>>> the same with ZooKeeper as the feature we are talking about is part of
>>>>> the
>>>>>> server and the only way I see of doing what you say is to turn off
>>>>>> features. More specifically, we'd need to disable the reconfig API and
>>>> do
>>>>>> not allow any change to the configuration, even though the code is
>>>> there.
>>>>>> 
>>>>>> Reconfig here refers to the ability of changing the configuration of an
>>>>>> ensemble (e.g., changing the set of servers).
>>>>>> 
>>>>>> -Flavio
>>>>>> 
>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>>>> 
>>>>>>> So, it would seem sensible to me to have a release where all features
>>>>> are
>>>>>>> stable, except where noted.  E.g. mark certain features as only
>>>> 'alpha
>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>>>> happily
>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>>>> 
>>>>>>> There's precedent for doing this sort of thing in other projects,
>>>> e.g.
>>>>> in
>>>>>>> Kafka, they've had several release where a new "Consumer API" is
>>>>> shipped
>>>>>>> that is available for beta-testing, but you can still just use the
>>>>> older
>>>>>>> stable consumer api, etc.
>>>>>>> 
>>>>>>> Jason
>>>>>>> 
>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>>> <powellm79@yahoo.com.invalid
>>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi Doug,
>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>>>> you
>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>>>> alpha
>>>>>>>> might not be fair, since it is far more stable then what most people
>>>>>>>> associate an alpha release to be.
>>>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>>>> for
>>>>>>>> you?.
>>>>>>>> There are many examples of 3.5.X being used in productions from my
>>>>>> limited
>>>>>>>> knowledge.
>>>>>>>> ThanksPowell.
>>>>>>>> 
>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>>>> fpj@apache.org>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> None of us expected the reconfig changes to take this long to
>>>>> stabilize.
>>>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>>>> The
>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>>>> than
>>>>>>>> trying to work around it.
>>>>>>>> 
>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>>>> get
>>>>>>>> everyone to contribute more patches and code reviews, we should be
>>>>> able
>>>>>> to
>>>>>>>> do it sooner. After all, it is a community based effort, so the
>>>>>> community
>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>>> 
>>>>>>>> -Flavio
>>>>>>>> 
>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>>>> the
>>>>>>>>> stable maintenance line, we are very conservative about
>>>> back-porting
>>>>> to
>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>>> not
>>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>>>> managing
>>>>>>>>> risk.
>>>>>>>>> 
>>>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>>>> any
>>>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>>>> scope
>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>>> focus
>>>>> on
>>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>>>> think
>>>>>>>>> this will help us accelerate the releases into beta and eventual
>>>> GA.
>>>>>>>>> 
>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>>>> thoughts
>>>>>>>>> from more experienced committers and PMC members about your
>>>> proposal
>>>>> to
>>>>>>>>> try to cut a stable release for a limited subset of what is in
>>>>>> branch-3.5
>>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>>> out
>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>>>> another
>>>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>>>> 3.5
>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>>>> "vanished"
>>>>>>>>> because of not meeting some stability criteria would be
>>>> undesirable.
>>>>>>>>> 
>>>>>>>>> --Chris Nauroth
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>>> 
>>>>>>>>>> Chris,
>>>>>>>>>> 
>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>>> others
>>>>>>>> (e.g.
>>>>>>>>>> if we don't care about certain new features, is it relatively
>>>>> stable)?
>>>>>>>>>> Would it be possible to cut out a version that only has the bits
>>>> we
>>>>>>>> think
>>>>>>>>>> are stable (and release that)?
>>>>>>>>>> 
>>>>>>>>>> From that timeline, and the historic release cadence, it would
>>>> seem
>>>>> to
>>>>>>>> be
>>>>>>>>>> a
>>>>>>>>>> years away before we get to the stable release?
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> 
>>>>>>>>>> Jason
>>>>>>>>>> 
>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hello Doug,
>>>>>>>>>>> 
>>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>>> 
>>>>>>>>>>> At this point, I think we're still pretty far away from
>>>> declaring a
>>>>>>>>>>> stable
>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>>>> anyone
>>>>>>>>>>> can
>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>>>> the
>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>>>> 
>>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>>> 
>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>>>> resolving a
>>>>>>>>>>> few
>>>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>>>> that
>>>>>>>>>>> will
>>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>>> 
>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>>>> was a
>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>>>> any
>>>>>>>>>>>> chance
>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>>>> looking
>>>>>>>> to
>>>>>>>>>>>> have
>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>> 
>>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>>> e
>>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 

Re: Zookeeper with SSL release date

[powell molleti <po...@yahoo.com.INVALID>]

Hi Flavio,

Generally do config options and command line args come under the same SLA as API?. I was assuming as such hence my question. Perhaps if the expectation is that this config option is temporary from get go then may be it is ok. The default for re-config support will be enabled or disabled?.

I am just thinking from provisioning point of view when people generate config options etc.

Thanks
Powell.



On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <fp...@apache.org> wrote:
Hi Powell,

I was thinking config file and system property server side. What's your concern with compatibility? The API itself wouldn't change, but the config option wouldn't exist in previous versions and would not exist either in later stable versions of 3.5.

-Flavio

    

> On 16 Mar 2016, at 22:08, powell molleti <po...@yahoo.com.INVALID> wrote:
> 
> Will this option be supplied via config file/args/API?. Will this option be a temporary thing i.e what about its compatibility?.
> 
> thanks
> Powell.
> 
> 
> 
> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
> The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:
> 
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>
> 
> Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.
> 
> About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.
> 
> I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.
> 
> -Flavio
> 
> 
>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>> 
>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
>> are related to reconfig. Given this, and the fact that it is run in
>> production since 2012 in multiple companies, I don't think its more
>> unstable than any other part of ZooKeeper.
>> 
>> There are multiple reconfig-related bugs that turned out really difficult
>> to debug without access to the actual system or at least to the Hudson
>> machines where some tests are failing. In the past, Michi and I physically
>> went to Hortonworks to debug one such issue, but this is of course not a
>> good method, and we weren't able to arrange such a visit again.
>> 
>> Regarding making it optional - the reconfig logic has several different
>> parts, some would be really difficult to disable using a configuration
>> parameter. But the actual dynamic expansion of the cluster is triggered by
>> the reconfig command, so it should not affect users who don't invoke it.
>> 
>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
>> 
>>> I suppose we could give it a try. How do other folks feel about it?
>>> 
>>> -Flavio
>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>> 
>>>> So, you could enable the dynamic reconfiguration feature behind a config
>>>> option, and document that it should only be enabled experimentally, use
>>> at
>>>> your own risk.  Keep it off by default.  Allow only static config by
>>>> default, until it's stable?
>>>> 
>>>> Jason
>>>> 
>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>> wrote:
>>>> 
>>>>> Hi Jason,
>>>>> 
>>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>>> that's how that project manages to make such a separation. We don't
>>> have
>>>>> the same with ZooKeeper as the feature we are talking about is part of
>>>> the
>>>>> server and the only way I see of doing what you say is to turn off
>>>>> features. More specifically, we'd need to disable the reconfig API and
>>> do
>>>>> not allow any change to the configuration, even though the code is
>>> there.
>>>>> 
>>>>> Reconfig here refers to the ability of changing the configuration of an
>>>>> ensemble (e.g., changing the set of servers).
>>>>> 
>>>>> -Flavio
>>>>> 
>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>>> 
>>>>>> So, it would seem sensible to me to have a release where all features
>>>> are
>>>>>> stable, except where noted.  E.g. mark certain features as only
>>> 'alpha
>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>>> happily
>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>>> 
>>>>>> There's precedent for doing this sort of thing in other projects,
>>> e.g.
>>>> in
>>>>>> Kafka, they've had several release where a new "Consumer API" is
>>>> shipped
>>>>>> that is available for beta-testing, but you can still just use the
>>>> older
>>>>>> stable consumer api, etc.
>>>>>> 
>>>>>> Jason
>>>>>> 
>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>> <powellm79@yahoo.com.invalid
>>>>>>> wrote:
>>>>>> 
>>>>>>> Hi Doug,
>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>>> you
>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>>> alpha
>>>>>>> might not be fair, since it is far more stable then what most people
>>>>>>> associate an alpha release to be.
>>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>>> for
>>>>>>> you?.
>>>>>>> There are many examples of 3.5.X being used in productions from my
>>>>> limited
>>>>>>> knowledge.
>>>>>>> ThanksPowell.
>>>>>>> 
>>>>>>>  On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>>> fpj@apache.org>
>>>>>>> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> None of us expected the reconfig changes to take this long to
>>>> stabilize.
>>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>>> The
>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>>> than
>>>>>>> trying to work around it.
>>>>>>> 
>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>>> get
>>>>>>> everyone to contribute more patches and code reviews, we should be
>>>> able
>>>>> to
>>>>>>> do it sooner. After all, it is a community based effort, so the
>>>>> community
>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>> 
>>>>>>> -Flavio
>>>>>>> 
>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>>> the
>>>>>>>> stable maintenance line, we are very conservative about
>>> back-porting
>>>> to
>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>> not
>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>>> managing
>>>>>>>> risk.
>>>>>>>> 
>>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>>> any
>>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>>> scope
>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>> focus
>>>> on
>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>>> think
>>>>>>>> this will help us accelerate the releases into beta and eventual
>>> GA.
>>>>>>>> 
>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>>> thoughts
>>>>>>>> from more experienced committers and PMC members about your
>>> proposal
>>>> to
>>>>>>>> try to cut a stable release for a limited subset of what is in
>>>>> branch-3.5
>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>> out
>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>>> another
>>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>>> 3.5
>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>>> "vanished"
>>>>>>>> because of not meeting some stability criteria would be
>>> undesirable.
>>>>>>>> 
>>>>>>>> --Chris Nauroth
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>> 
>>>>>>>>> Chris,
>>>>>>>>> 
>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>> others
>>>>>>> (e.g.
>>>>>>>>> if we don't care about certain new features, is it relatively
>>>> stable)?
>>>>>>>>> Would it be possible to cut out a version that only has the bits
>>> we
>>>>>>> think
>>>>>>>>> are stable (and release that)?
>>>>>>>>> 
>>>>>>>>> From that timeline, and the historic release cadence, it would
>>> seem
>>>> to
>>>>>>> be
>>>>>>>>> a
>>>>>>>>> years away before we get to the stable release?
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> 
>>>>>>>>> Jason
>>>>>>>>> 
>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hello Doug,
>>>>>>>>>> 
>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>> 
>>>>>>>>>> At this point, I think we're still pretty far away from
>>> declaring a
>>>>>>>>>> stable
>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>>> anyone
>>>>>>>>>> can
>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>>> the
>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>>> 
>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>> 
>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>>> resolving a
>>>>>>>>>> few
>>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>>> that
>>>>>>>>>> will
>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>> 
>>>>>>>>>> --Chris Nauroth
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>>> was a
>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>>> any
>>>>>>>>>>> chance
>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>>> looking
>>>>>>> to
>>>>>>>>>>> have
>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context:
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>> 
>>>>> 
>>>> 
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>> e
>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 

Re: Zookeeper with SSL release date

[Flavio Junqueira <fp...@apache.org>]

Hi Powell,

I was thinking config file and system property server side. What's your concern with compatibility? The API itself wouldn't change, but the config option wouldn't exist in previous versions and would not exist either in later stable versions of 3.5.

-Flavio

    
> On 16 Mar 2016, at 22:08, powell molleti <po...@yahoo.com.INVALID> wrote:
> 
> Will this option be supplied via config file/args/API?. Will this option be a temporary thing i.e what about its compatibility?.
> 
> thanks
> Powell.
> 
> 
> 
> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
> The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:
> 
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>
> 
> Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.
> 
> About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.
> 
> I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.
> 
> -Flavio
> 
> 
>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>> 
>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
>> are related to reconfig. Given this, and the fact that it is run in
>> production since 2012 in multiple companies, I don't think its more
>> unstable than any other part of ZooKeeper.
>> 
>> There are multiple reconfig-related bugs that turned out really difficult
>> to debug without access to the actual system or at least to the Hudson
>> machines where some tests are failing. In the past, Michi and I physically
>> went to Hortonworks to debug one such issue, but this is of course not a
>> good method, and we weren't able to arrange such a visit again.
>> 
>> Regarding making it optional - the reconfig logic has several different
>> parts, some would be really difficult to disable using a configuration
>> parameter. But the actual dynamic expansion of the cluster is triggered by
>> the reconfig command, so it should not affect users who don't invoke it.
>> 
>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
>> 
>>> I suppose we could give it a try. How do other folks feel about it?
>>> 
>>> -Flavio
>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>> 
>>>> So, you could enable the dynamic reconfiguration feature behind a config
>>>> option, and document that it should only be enabled experimentally, use
>>> at
>>>> your own risk.  Keep it off by default.  Allow only static config by
>>>> default, until it's stable?
>>>> 
>>>> Jason
>>>> 
>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>> wrote:
>>>> 
>>>>> Hi Jason,
>>>>> 
>>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>>> that's how that project manages to make such a separation. We don't
>>> have
>>>>> the same with ZooKeeper as the feature we are talking about is part of
>>>> the
>>>>> server and the only way I see of doing what you say is to turn off
>>>>> features. More specifically, we'd need to disable the reconfig API and
>>> do
>>>>> not allow any change to the configuration, even though the code is
>>> there.
>>>>> 
>>>>> Reconfig here refers to the ability of changing the configuration of an
>>>>> ensemble (e.g., changing the set of servers).
>>>>> 
>>>>> -Flavio
>>>>> 
>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>>> 
>>>>>> So, it would seem sensible to me to have a release where all features
>>>> are
>>>>>> stable, except where noted.  E.g. mark certain features as only
>>> 'alpha
>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>>> happily
>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>>> 
>>>>>> There's precedent for doing this sort of thing in other projects,
>>> e.g.
>>>> in
>>>>>> Kafka, they've had several release where a new "Consumer API" is
>>>> shipped
>>>>>> that is available for beta-testing, but you can still just use the
>>>> older
>>>>>> stable consumer api, etc.
>>>>>> 
>>>>>> Jason
>>>>>> 
>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>> <powellm79@yahoo.com.invalid
>>>>>>> wrote:
>>>>>> 
>>>>>>> Hi Doug,
>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>>> you
>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>>> alpha
>>>>>>> might not be fair, since it is far more stable then what most people
>>>>>>> associate an alpha release to be.
>>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>>> for
>>>>>>> you?.
>>>>>>> There are many examples of 3.5.X being used in productions from my
>>>>> limited
>>>>>>> knowledge.
>>>>>>> ThanksPowell.
>>>>>>> 
>>>>>>>  On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>>> fpj@apache.org>
>>>>>>> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> None of us expected the reconfig changes to take this long to
>>>> stabilize.
>>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>>> The
>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>>> than
>>>>>>> trying to work around it.
>>>>>>> 
>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>>> get
>>>>>>> everyone to contribute more patches and code reviews, we should be
>>>> able
>>>>> to
>>>>>>> do it sooner. After all, it is a community based effort, so the
>>>>> community
>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>> 
>>>>>>> -Flavio
>>>>>>> 
>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>>> the
>>>>>>>> stable maintenance line, we are very conservative about
>>> back-porting
>>>> to
>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>> not
>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>>> managing
>>>>>>>> risk.
>>>>>>>> 
>>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>>> any
>>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>>> scope
>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>> focus
>>>> on
>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>>> think
>>>>>>>> this will help us accelerate the releases into beta and eventual
>>> GA.
>>>>>>>> 
>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>>> thoughts
>>>>>>>> from more experienced committers and PMC members about your
>>> proposal
>>>> to
>>>>>>>> try to cut a stable release for a limited subset of what is in
>>>>> branch-3.5
>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>> out
>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>>> another
>>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>>> 3.5
>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>>> "vanished"
>>>>>>>> because of not meeting some stability criteria would be
>>> undesirable.
>>>>>>>> 
>>>>>>>> --Chris Nauroth
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>> 
>>>>>>>>> Chris,
>>>>>>>>> 
>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>> others
>>>>>>> (e.g.
>>>>>>>>> if we don't care about certain new features, is it relatively
>>>> stable)?
>>>>>>>>> Would it be possible to cut out a version that only has the bits
>>> we
>>>>>>> think
>>>>>>>>> are stable (and release that)?
>>>>>>>>> 
>>>>>>>>> From that timeline, and the historic release cadence, it would
>>> seem
>>>> to
>>>>>>> be
>>>>>>>>> a
>>>>>>>>> years away before we get to the stable release?
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> 
>>>>>>>>> Jason
>>>>>>>>> 
>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hello Doug,
>>>>>>>>>> 
>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>> 
>>>>>>>>>> At this point, I think we're still pretty far away from
>>> declaring a
>>>>>>>>>> stable
>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>>> anyone
>>>>>>>>>> can
>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>>> the
>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>>> 
>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>> 
>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>>> resolving a
>>>>>>>>>> few
>>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>>> that
>>>>>>>>>> will
>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>> 
>>>>>>>>>> --Chris Nauroth
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>>> was a
>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>>> any
>>>>>>>>>>> chance
>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>>> looking
>>>>>>> to
>>>>>>>>>>> have
>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context:
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>> 
>>>>> 
>>>> 
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>> e
>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 

Re: Zookeeper with SSL release date

[powell molleti <po...@yahoo.com.INVALID>]

Will this option be supplied via config file/args/API?. Will this option be a temporary thing i.e what about its compatibility?.

thanks
Powell.



On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:

https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>

Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.

About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.

I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.

-Flavio


> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
> 
> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
> are related to reconfig. Given this, and the fact that it is run in
> production since 2012 in multiple companies, I don't think its more
> unstable than any other part of ZooKeeper.
> 
> There are multiple reconfig-related bugs that turned out really difficult
> to debug without access to the actual system or at least to the Hudson
> machines where some tests are failing. In the past, Michi and I physically
> went to Hortonworks to debug one such issue, but this is of course not a
> good method, and we weren't able to arrange such a visit again.
> 
> Regarding making it optional - the reconfig logic has several different
> parts, some would be really difficult to disable using a configuration
> parameter. But the actual dynamic expansion of the cluster is triggered by
> the reconfig command, so it should not affect users who don't invoke it.
> 
> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
> 
>> I suppose we could give it a try. How do other folks feel about it?
>> 
>> -Flavio
>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>> 
>>> So, you could enable the dynamic reconfiguration feature behind a config
>>> option, and document that it should only be enabled experimentally, use
>> at
>>> your own risk.  Keep it off by default.  Allow only static config by
>>> default, until it's stable?
>>> 
>>> Jason
>>> 
>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>> wrote:
>>> 
>>>> Hi Jason,
>>>> 
>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>> that's how that project manages to make such a separation. We don't
>> have
>>>> the same with ZooKeeper as the feature we are talking about is part of
>>> the
>>>> server and the only way I see of doing what you say is to turn off
>>>> features. More specifically, we'd need to disable the reconfig API and
>> do
>>>> not allow any change to the configuration, even though the code is
>> there.
>>>> 
>>>> Reconfig here refers to the ability of changing the configuration of an
>>>> ensemble (e.g., changing the set of servers).
>>>> 
>>>> -Flavio
>>>> 
>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>> 
>>>>> So, it would seem sensible to me to have a release where all features
>>> are
>>>>> stable, except where noted.  E.g. mark certain features as only
>> 'alpha
>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>> happily
>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>> 
>>>>> There's precedent for doing this sort of thing in other projects,
>> e.g.
>>> in
>>>>> Kafka, they've had several release where a new "Consumer API" is
>>> shipped
>>>>> that is available for beta-testing, but you can still just use the
>>> older
>>>>> stable consumer api, etc.
>>>>> 
>>>>> Jason
>>>>> 
>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>> <powellm79@yahoo.com.invalid
>>>>>> wrote:
>>>>> 
>>>>>> Hi Doug,
>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>> you
>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>> alpha
>>>>>> might not be fair, since it is far more stable then what most people
>>>>>> associate an alpha release to be.
>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>> for
>>>>>> you?.
>>>>>> There are many examples of 3.5.X being used in productions from my
>>>> limited
>>>>>> knowledge.
>>>>>> ThanksPowell.
>>>>>> 
>>>>>>   On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>> fpj@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>> 
>>>>>> None of us expected the reconfig changes to take this long to
>>> stabilize.
>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>> The
>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>> than
>>>>>> trying to work around it.
>>>>>> 
>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>> get
>>>>>> everyone to contribute more patches and code reviews, we should be
>>> able
>>>> to
>>>>>> do it sooner. After all, it is a community based effort, so the
>>>> community
>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>> 
>>>>>> -Flavio
>>>>>> 
>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>> the
>>>>>>> stable maintenance line, we are very conservative about
>> back-porting
>>> to
>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>> not
>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>> managing
>>>>>>> risk.
>>>>>>> 
>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>> any
>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>> scope
>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>> focus
>>> on
>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>> think
>>>>>>> this will help us accelerate the releases into beta and eventual
>> GA.
>>>>>>> 
>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>> thoughts
>>>>>>> from more experienced committers and PMC members about your
>> proposal
>>> to
>>>>>>> try to cut a stable release for a limited subset of what is in
>>>> branch-3.5
>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>> out
>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>> another
>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>> 3.5
>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>> "vanished"
>>>>>>> because of not meeting some stability criteria would be
>> undesirable.
>>>>>>> 
>>>>>>> --Chris Nauroth
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>> 
>>>>>>>> Chris,
>>>>>>>> 
>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>> others
>>>>>> (e.g.
>>>>>>>> if we don't care about certain new features, is it relatively
>>> stable)?
>>>>>>>> Would it be possible to cut out a version that only has the bits
>> we
>>>>>> think
>>>>>>>> are stable (and release that)?
>>>>>>>> 
>>>>>>>> From that timeline, and the historic release cadence, it would
>> seem
>>> to
>>>>>> be
>>>>>>>> a
>>>>>>>> years away before we get to the stable release?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> 
>>>>>>>> Jason
>>>>>>>> 
>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>> cnauroth@hortonworks.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hello Doug,
>>>>>>>>> 
>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>> 
>>>>>>>>> At this point, I think we're still pretty far away from
>> declaring a
>>>>>>>>> stable
>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>> anyone
>>>>>>>>> can
>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>> the
>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>> 
>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>> 
>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>> resolving a
>>>>>>>>> few
>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>> that
>>>>>>>>> will
>>>>>>>>> get done in the next few weeks.
>>>>>>>>> 
>>>>>>>>> --Chris Nauroth
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>> was a
>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>> any
>>>>>>>>>> chance
>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>> looking
>>>>>> to
>>>>>>>>>> have
>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>> 
>>>> 
>>> 
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>> e
>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
>> 

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

On Thu, Mar 17, 2016 at 8:10 AM, Flavio Junqueira <fp...@apache.org> wrote:
>
>> On 17 Mar 2016, at 14:12, Patrick Hunt <ph...@apache.org> wrote:
>>
>> On Thu, Mar 17, 2016 at 12:59 AM, Flavio Junqueira <fp...@apache.org> wrote:
>>> I was thinking server side actually. If the switch is off, any attempt to reconfig is a no-op and the server would log a warning. If you we want to propagate the error up, then yeah, we need to start worrying about compatibility and such.
>>>
>>
>> Are you saying someone wants to run with reconfig inaccessible, but
>> doesn't want to have to enable ACL/auth? That actually does make sense
>> to me from a "feature switch" perspective.
>>
>
> I think you meant to disagree here and say "doesn't make sense". It's a good point, but for clarity, are you suggesting we disable it via ACLs?
>

I did mean "does make sense". I've been thinking about it a bit and
I'm coming around to what you are saying, but from a security/auth
perspective rather than quality.

One thing in particular is "principal of least surprise". If you're
running zk w/o security turned on and suddenly folks can do reconfig
operations it's going to potentially be a problem. It's like how we
used to allow "kill" 4lw but then removed it because we didn't want to
expose that ability to clients.

Rather than force people to turn on kerberos (etc...) we could instead
have the feature off by default through a simple configuration. Likely
we'd want to ship with it turned off as the default. Folks that want
to enable reconfig support could do so simply.

Patrick

>>> I also would rather see us address it, although I can fully understand that there are other fixes and improvements in 3.5 that people are interested in.
>>>
>>
>> My thinking is more that reconfig changed a lot of core functionality.
>> It's not a "surface" feature. As such just disabling access doesn't
>> affect quality. I don't see this as a concern for reconfig alone,
>> there were a number of such changes in 3.5 and that's really my
>> concern - overall core quality vs quality of one specific
>> change/feature.
>>
>
> Agreed that disabling doesn't affect quality, but possibly decouples the decision of moving from alpha to beta independent of the security review of reconfig.
>
> -Flavio
>
>>>
>>>> On 17 Mar 2016, at 00:46, Patrick Hunt <ph...@apache.org> wrote:
>>>>
>>>> I'm not a huge fan of turning it off to be honest. Also just turning
>>>> it off at the API level wouldn't be enough, we'd need to turn it off
>>>> at the protocol level (otw it could still be accessed).
>>>>
>>>> I'd rather see us address it than kick it down the road. It's a major
>>>> feature of 3.5.
>>>>
>>>> Patrick
>>>>
>>>> On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
>>>>> The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:
>>>>>
>>>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>
>>>>>
>>>>> Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.
>>>>>
>>>>> About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.
>>>>>
>>>>> I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.
>>>>>
>>>>> -Flavio
>>>>>
>>>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>>>>>>
>>>>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
>>>>>> are related to reconfig. Given this, and the fact that it is run in
>>>>>> production since 2012 in multiple companies, I don't think its more
>>>>>> unstable than any other part of ZooKeeper.
>>>>>>
>>>>>> There are multiple reconfig-related bugs that turned out really difficult
>>>>>> to debug without access to the actual system or at least to the Hudson
>>>>>> machines where some tests are failing. In the past, Michi and I physically
>>>>>> went to Hortonworks to debug one such issue, but this is of course not a
>>>>>> good method, and we weren't able to arrange such a visit again.
>>>>>>
>>>>>> Regarding making it optional - the reconfig logic has several different
>>>>>> parts, some would be really difficult to disable using a configuration
>>>>>> parameter. But the actual dynamic expansion of the cluster is triggered by
>>>>>> the reconfig command, so it should not affect users who don't invoke it.
>>>>>>
>>>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
>>>>>>
>>>>>>> I suppose we could give it a try. How do other folks feel about it?
>>>>>>>
>>>>>>> -Flavio
>>>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>
>>>>>>>> So, you could enable the dynamic reconfiguration feature behind a config
>>>>>>>> option, and document that it should only be enabled experimentally, use
>>>>>>> at
>>>>>>>> your own risk.  Keep it off by default.  Allow only static config by
>>>>>>>> default, until it's stable?
>>>>>>>>
>>>>>>>> Jason
>>>>>>>>
>>>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Jason,
>>>>>>>>>
>>>>>>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>>>>>>> that's how that project manages to make such a separation. We don't
>>>>>>> have
>>>>>>>>> the same with ZooKeeper as the feature we are talking about is part of
>>>>>>>> the
>>>>>>>>> server and the only way I see of doing what you say is to turn off
>>>>>>>>> features. More specifically, we'd need to disable the reconfig API and
>>>>>>> do
>>>>>>>>> not allow any change to the configuration, even though the code is
>>>>>>> there.
>>>>>>>>>
>>>>>>>>> Reconfig here refers to the ability of changing the configuration of an
>>>>>>>>> ensemble (e.g., changing the set of servers).
>>>>>>>>>
>>>>>>>>> -Flavio
>>>>>>>>>
>>>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>>>>>>>
>>>>>>>>>> So, it would seem sensible to me to have a release where all features
>>>>>>>> are
>>>>>>>>>> stable, except where noted.  E.g. mark certain features as only
>>>>>>> 'alpha
>>>>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>>>>>>> happily
>>>>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>>>>>>>
>>>>>>>>>> There's precedent for doing this sort of thing in other projects,
>>>>>>> e.g.
>>>>>>>> in
>>>>>>>>>> Kafka, they've had several release where a new "Consumer API" is
>>>>>>>> shipped
>>>>>>>>>> that is available for beta-testing, but you can still just use the
>>>>>>>> older
>>>>>>>>>> stable consumer api, etc.
>>>>>>>>>>
>>>>>>>>>> Jason
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>>>>>> <powellm79@yahoo.com.invalid
>>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Doug,
>>>>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>>>>>>> you
>>>>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>>>>>>> alpha
>>>>>>>>>>> might not be fair, since it is far more stable then what most people
>>>>>>>>>>> associate an alpha release to be.
>>>>>>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>>>>>>> for
>>>>>>>>>>> you?.
>>>>>>>>>>> There are many examples of 3.5.X being used in productions from my
>>>>>>>>> limited
>>>>>>>>>>> knowledge.
>>>>>>>>>>> ThanksPowell.
>>>>>>>>>>>
>>>>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>>>>>>> fpj@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> None of us expected the reconfig changes to take this long to
>>>>>>>> stabilize.
>>>>>>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>>>>>>> The
>>>>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>>>>>>> than
>>>>>>>>>>> trying to work around it.
>>>>>>>>>>>
>>>>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>>>>>>> get
>>>>>>>>>>> everyone to contribute more patches and code reviews, we should be
>>>>>>>> able
>>>>>>>>> to
>>>>>>>>>>> do it sooner. After all, it is a community based effort, so the
>>>>>>>>> community
>>>>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>>>>>>
>>>>>>>>>>> -Flavio
>>>>>>>>>>>
>>>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>>>>>>> the
>>>>>>>>>>>> stable maintenance line, we are very conservative about
>>>>>>> back-porting
>>>>>>>> to
>>>>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>>>>>> not
>>>>>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>>>>>>> managing
>>>>>>>>>>>> risk.
>>>>>>>>>>>>
>>>>>>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>>>>>>> any
>>>>>>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>>>>>>> scope
>>>>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>>>>>> focus
>>>>>>>> on
>>>>>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>>>>>>> think
>>>>>>>>>>>> this will help us accelerate the releases into beta and eventual
>>>>>>> GA.
>>>>>>>>>>>>
>>>>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>>>>>>> thoughts
>>>>>>>>>>>> from more experienced committers and PMC members about your
>>>>>>> proposal
>>>>>>>> to
>>>>>>>>>>>> try to cut a stable release for a limited subset of what is in
>>>>>>>>> branch-3.5
>>>>>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>>>>>> out
>>>>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>>>>>>> another
>>>>>>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>>>>>>> 3.5
>>>>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>>>>>>> "vanished"
>>>>>>>>>>>> because of not meeting some stability criteria would be
>>>>>>> undesirable.
>>>>>>>>>>>>
>>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Chris,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>>>>>> others
>>>>>>>>>>> (e.g.
>>>>>>>>>>>>> if we don't care about certain new features, is it relatively
>>>>>>>> stable)?
>>>>>>>>>>>>> Would it be possible to cut out a version that only has the bits
>>>>>>> we
>>>>>>>>>>> think
>>>>>>>>>>>>> are stable (and release that)?
>>>>>>>>>>>>>
>>>>>>>>>>>>> From that timeline, and the historic release cadence, it would
>>>>>>> seem
>>>>>>>> to
>>>>>>>>>>> be
>>>>>>>>>>>>> a
>>>>>>>>>>>>> years away before we get to the stable release?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jason
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello Doug,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> At this point, I think we're still pretty far away from
>>>>>>> declaring a
>>>>>>>>>>>>>> stable
>>>>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>>>>>>> anyone
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>>>>>>> the
>>>>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>>>>>>> resolving a
>>>>>>>>>>>>>> few
>>>>>>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>>>>>>> that
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>>>>>>> was a
>>>>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>>>>>>> any
>>>>>>>>>>>>>>> chance
>>>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>>>>>>> looking
>>>>>>>>>>> to
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>>>>>> e
>>>>>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>

Re: Zookeeper with SSL release date

[Flavio Junqueira <fp...@apache.org>]

> On 17 Mar 2016, at 14:12, Patrick Hunt <ph...@apache.org> wrote:
> 
> On Thu, Mar 17, 2016 at 12:59 AM, Flavio Junqueira <fp...@apache.org> wrote:
>> I was thinking server side actually. If the switch is off, any attempt to reconfig is a no-op and the server would log a warning. If you we want to propagate the error up, then yeah, we need to start worrying about compatibility and such.
>> 
> 
> Are you saying someone wants to run with reconfig inaccessible, but
> doesn't want to have to enable ACL/auth? That actually does make sense
> to me from a "feature switch" perspective.
> 

I think you meant to disagree here and say "doesn't make sense". It's a good point, but for clarity, are you suggesting we disable it via ACLs? 

>> I also would rather see us address it, although I can fully understand that there are other fixes and improvements in 3.5 that people are interested in.
>> 
> 
> My thinking is more that reconfig changed a lot of core functionality.
> It's not a "surface" feature. As such just disabling access doesn't
> affect quality. I don't see this as a concern for reconfig alone,
> there were a number of such changes in 3.5 and that's really my
> concern - overall core quality vs quality of one specific
> change/feature.
> 

Agreed that disabling doesn't affect quality, but possibly decouples the decision of moving from alpha to beta independent of the security review of reconfig.

-Flavio

>> 
>>> On 17 Mar 2016, at 00:46, Patrick Hunt <ph...@apache.org> wrote:
>>> 
>>> I'm not a huge fan of turning it off to be honest. Also just turning
>>> it off at the API level wouldn't be enough, we'd need to turn it off
>>> at the protocol level (otw it could still be accessed).
>>> 
>>> I'd rather see us address it than kick it down the road. It's a major
>>> feature of 3.5.
>>> 
>>> Patrick
>>> 
>>> On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
>>>> The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:
>>>> 
>>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>
>>>> 
>>>> Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.
>>>> 
>>>> About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.
>>>> 
>>>> I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.
>>>> 
>>>> -Flavio
>>>> 
>>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>>>>> 
>>>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
>>>>> are related to reconfig. Given this, and the fact that it is run in
>>>>> production since 2012 in multiple companies, I don't think its more
>>>>> unstable than any other part of ZooKeeper.
>>>>> 
>>>>> There are multiple reconfig-related bugs that turned out really difficult
>>>>> to debug without access to the actual system or at least to the Hudson
>>>>> machines where some tests are failing. In the past, Michi and I physically
>>>>> went to Hortonworks to debug one such issue, but this is of course not a
>>>>> good method, and we weren't able to arrange such a visit again.
>>>>> 
>>>>> Regarding making it optional - the reconfig logic has several different
>>>>> parts, some would be really difficult to disable using a configuration
>>>>> parameter. But the actual dynamic expansion of the cluster is triggered by
>>>>> the reconfig command, so it should not affect users who don't invoke it.
>>>>> 
>>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
>>>>> 
>>>>>> I suppose we could give it a try. How do other folks feel about it?
>>>>>> 
>>>>>> -Flavio
>>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>> 
>>>>>>> So, you could enable the dynamic reconfiguration feature behind a config
>>>>>>> option, and document that it should only be enabled experimentally, use
>>>>>> at
>>>>>>> your own risk.  Keep it off by default.  Allow only static config by
>>>>>>> default, until it's stable?
>>>>>>> 
>>>>>>> Jason
>>>>>>> 
>>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi Jason,
>>>>>>>> 
>>>>>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>>>>>> that's how that project manages to make such a separation. We don't
>>>>>> have
>>>>>>>> the same with ZooKeeper as the feature we are talking about is part of
>>>>>>> the
>>>>>>>> server and the only way I see of doing what you say is to turn off
>>>>>>>> features. More specifically, we'd need to disable the reconfig API and
>>>>>> do
>>>>>>>> not allow any change to the configuration, even though the code is
>>>>>> there.
>>>>>>>> 
>>>>>>>> Reconfig here refers to the ability of changing the configuration of an
>>>>>>>> ensemble (e.g., changing the set of servers).
>>>>>>>> 
>>>>>>>> -Flavio
>>>>>>>> 
>>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>>>>>> 
>>>>>>>>> So, it would seem sensible to me to have a release where all features
>>>>>>> are
>>>>>>>>> stable, except where noted.  E.g. mark certain features as only
>>>>>> 'alpha
>>>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>>>>>> happily
>>>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>>>>>> 
>>>>>>>>> There's precedent for doing this sort of thing in other projects,
>>>>>> e.g.
>>>>>>> in
>>>>>>>>> Kafka, they've had several release where a new "Consumer API" is
>>>>>>> shipped
>>>>>>>>> that is available for beta-testing, but you can still just use the
>>>>>>> older
>>>>>>>>> stable consumer api, etc.
>>>>>>>>> 
>>>>>>>>> Jason
>>>>>>>>> 
>>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>>>>> <powellm79@yahoo.com.invalid
>>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hi Doug,
>>>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>>>>>> you
>>>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>>>>>> alpha
>>>>>>>>>> might not be fair, since it is far more stable then what most people
>>>>>>>>>> associate an alpha release to be.
>>>>>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>>>>>> for
>>>>>>>>>> you?.
>>>>>>>>>> There are many examples of 3.5.X being used in productions from my
>>>>>>>> limited
>>>>>>>>>> knowledge.
>>>>>>>>>> ThanksPowell.
>>>>>>>>>> 
>>>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>>>>>> fpj@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> None of us expected the reconfig changes to take this long to
>>>>>>> stabilize.
>>>>>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>>>>>> The
>>>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>>>>>> than
>>>>>>>>>> trying to work around it.
>>>>>>>>>> 
>>>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>>>>>> get
>>>>>>>>>> everyone to contribute more patches and code reviews, we should be
>>>>>>> able
>>>>>>>> to
>>>>>>>>>> do it sooner. After all, it is a community based effort, so the
>>>>>>>> community
>>>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>>>>> 
>>>>>>>>>> -Flavio
>>>>>>>>>> 
>>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>>>>>> the
>>>>>>>>>>> stable maintenance line, we are very conservative about
>>>>>> back-porting
>>>>>>> to
>>>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>>>>> not
>>>>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>>>>>> managing
>>>>>>>>>>> risk.
>>>>>>>>>>> 
>>>>>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>>>>>> any
>>>>>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>>>>>> scope
>>>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>>>>> focus
>>>>>>> on
>>>>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>>>>>> think
>>>>>>>>>>> this will help us accelerate the releases into beta and eventual
>>>>>> GA.
>>>>>>>>>>> 
>>>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>>>>>> thoughts
>>>>>>>>>>> from more experienced committers and PMC members about your
>>>>>> proposal
>>>>>>> to
>>>>>>>>>>> try to cut a stable release for a limited subset of what is in
>>>>>>>> branch-3.5
>>>>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>>>>> out
>>>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>>>>>> another
>>>>>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>>>>>> 3.5
>>>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>>>>>> "vanished"
>>>>>>>>>>> because of not meeting some stability criteria would be
>>>>>> undesirable.
>>>>>>>>>>> 
>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Chris,
>>>>>>>>>>>> 
>>>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>>>>> others
>>>>>>>>>> (e.g.
>>>>>>>>>>>> if we don't care about certain new features, is it relatively
>>>>>>> stable)?
>>>>>>>>>>>> Would it be possible to cut out a version that only has the bits
>>>>>> we
>>>>>>>>>> think
>>>>>>>>>>>> are stable (and release that)?
>>>>>>>>>>>> 
>>>>>>>>>>>> From that timeline, and the historic release cadence, it would
>>>>>> seem
>>>>>>> to
>>>>>>>>>> be
>>>>>>>>>>>> a
>>>>>>>>>>>> years away before we get to the stable release?
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> 
>>>>>>>>>>>> Jason
>>>>>>>>>>>> 
>>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hello Doug,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> At this point, I think we're still pretty far away from
>>>>>> declaring a
>>>>>>>>>>>>> stable
>>>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>>>>>> anyone
>>>>>>>>>>>>> can
>>>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>>>>>> the
>>>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>>>>> 
>>>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>>>>>> resolving a
>>>>>>>>>>>>> few
>>>>>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>>>>>> that
>>>>>>>>>>>>> will
>>>>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>>>>>> was a
>>>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>>>>>> any
>>>>>>>>>>>>>> chance
>>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>>>>>> looking
>>>>>>>>>> to
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>>>>> e
>>>>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>> 

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

On Thu, Mar 17, 2016 at 12:59 AM, Flavio Junqueira <fp...@apache.org> wrote:
> I was thinking server side actually. If the switch is off, any attempt to reconfig is a no-op and the server would log a warning. If you we want to propagate the error up, then yeah, we need to start worrying about compatibility and such.
>

Are you saying someone wants to run with reconfig inaccessible, but
doesn't want to have to enable ACL/auth? That actually does make sense
to me from a "feature switch" perspective.

> I also would rather see us address it, although I can fully understand that there are other fixes and improvements in 3.5 that people are interested in.
>

My thinking is more that reconfig changed a lot of core functionality.
It's not a "surface" feature. As such just disabling access doesn't
affect quality. I don't see this as a concern for reconfig alone,
there were a number of such changes in 3.5 and that's really my
concern - overall core quality vs quality of one specific
change/feature.

> -Flavio
>
>> On 17 Mar 2016, at 00:46, Patrick Hunt <ph...@apache.org> wrote:
>>
>> I'm not a huge fan of turning it off to be honest. Also just turning
>> it off at the API level wouldn't be enough, we'd need to turn it off
>> at the protocol level (otw it could still be accessed).
>>
>> I'd rather see us address it than kick it down the road. It's a major
>> feature of 3.5.
>>
>> Patrick
>>
>> On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
>>> The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:
>>>
>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>
>>>
>>> Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.
>>>
>>> About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.
>>>
>>> I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.
>>>
>>> -Flavio
>>>
>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>>>>
>>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
>>>> are related to reconfig. Given this, and the fact that it is run in
>>>> production since 2012 in multiple companies, I don't think its more
>>>> unstable than any other part of ZooKeeper.
>>>>
>>>> There are multiple reconfig-related bugs that turned out really difficult
>>>> to debug without access to the actual system or at least to the Hudson
>>>> machines where some tests are failing. In the past, Michi and I physically
>>>> went to Hortonworks to debug one such issue, but this is of course not a
>>>> good method, and we weren't able to arrange such a visit again.
>>>>
>>>> Regarding making it optional - the reconfig logic has several different
>>>> parts, some would be really difficult to disable using a configuration
>>>> parameter. But the actual dynamic expansion of the cluster is triggered by
>>>> the reconfig command, so it should not affect users who don't invoke it.
>>>>
>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
>>>>
>>>>> I suppose we could give it a try. How do other folks feel about it?
>>>>>
>>>>> -Flavio
>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>
>>>>>> So, you could enable the dynamic reconfiguration feature behind a config
>>>>>> option, and document that it should only be enabled experimentally, use
>>>>> at
>>>>>> your own risk.  Keep it off by default.  Allow only static config by
>>>>>> default, until it's stable?
>>>>>>
>>>>>> Jason
>>>>>>
>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>>>> wrote:
>>>>>>
>>>>>>> Hi Jason,
>>>>>>>
>>>>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>>>>> that's how that project manages to make such a separation. We don't
>>>>> have
>>>>>>> the same with ZooKeeper as the feature we are talking about is part of
>>>>>> the
>>>>>>> server and the only way I see of doing what you say is to turn off
>>>>>>> features. More specifically, we'd need to disable the reconfig API and
>>>>> do
>>>>>>> not allow any change to the configuration, even though the code is
>>>>> there.
>>>>>>>
>>>>>>> Reconfig here refers to the ability of changing the configuration of an
>>>>>>> ensemble (e.g., changing the set of servers).
>>>>>>>
>>>>>>> -Flavio
>>>>>>>
>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>>>>>
>>>>>>>> So, it would seem sensible to me to have a release where all features
>>>>>> are
>>>>>>>> stable, except where noted.  E.g. mark certain features as only
>>>>> 'alpha
>>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>>>>> happily
>>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>>>>>
>>>>>>>> There's precedent for doing this sort of thing in other projects,
>>>>> e.g.
>>>>>> in
>>>>>>>> Kafka, they've had several release where a new "Consumer API" is
>>>>>> shipped
>>>>>>>> that is available for beta-testing, but you can still just use the
>>>>>> older
>>>>>>>> stable consumer api, etc.
>>>>>>>>
>>>>>>>> Jason
>>>>>>>>
>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>>>> <powellm79@yahoo.com.invalid
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Doug,
>>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>>>>> you
>>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>>>>> alpha
>>>>>>>>> might not be fair, since it is far more stable then what most people
>>>>>>>>> associate an alpha release to be.
>>>>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>>>>> for
>>>>>>>>> you?.
>>>>>>>>> There are many examples of 3.5.X being used in productions from my
>>>>>>> limited
>>>>>>>>> knowledge.
>>>>>>>>> ThanksPowell.
>>>>>>>>>
>>>>>>>>>  On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>>>>> fpj@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> None of us expected the reconfig changes to take this long to
>>>>>> stabilize.
>>>>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>>>>> The
>>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>>>>> than
>>>>>>>>> trying to work around it.
>>>>>>>>>
>>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>>>>> get
>>>>>>>>> everyone to contribute more patches and code reviews, we should be
>>>>>> able
>>>>>>> to
>>>>>>>>> do it sooner. After all, it is a community based effort, so the
>>>>>>> community
>>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>>>>
>>>>>>>>> -Flavio
>>>>>>>>>
>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>>>>> the
>>>>>>>>>> stable maintenance line, we are very conservative about
>>>>> back-porting
>>>>>> to
>>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>>>> not
>>>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>>>>> managing
>>>>>>>>>> risk.
>>>>>>>>>>
>>>>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>>>>> any
>>>>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>>>>> scope
>>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>>>> focus
>>>>>> on
>>>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>>>>> think
>>>>>>>>>> this will help us accelerate the releases into beta and eventual
>>>>> GA.
>>>>>>>>>>
>>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>>>>> thoughts
>>>>>>>>>> from more experienced committers and PMC members about your
>>>>> proposal
>>>>>> to
>>>>>>>>>> try to cut a stable release for a limited subset of what is in
>>>>>>> branch-3.5
>>>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>>>> out
>>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>>>>> another
>>>>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>>>>> 3.5
>>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>>>>> "vanished"
>>>>>>>>>> because of not meeting some stability criteria would be
>>>>> undesirable.
>>>>>>>>>>
>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Chris,
>>>>>>>>>>>
>>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>>>> others
>>>>>>>>> (e.g.
>>>>>>>>>>> if we don't care about certain new features, is it relatively
>>>>>> stable)?
>>>>>>>>>>> Would it be possible to cut out a version that only has the bits
>>>>> we
>>>>>>>>> think
>>>>>>>>>>> are stable (and release that)?
>>>>>>>>>>>
>>>>>>>>>>> From that timeline, and the historic release cadence, it would
>>>>> seem
>>>>>> to
>>>>>>>>> be
>>>>>>>>>>> a
>>>>>>>>>>> years away before we get to the stable release?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Jason
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello Doug,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>>>>
>>>>>>>>>>>> At this point, I think we're still pretty far away from
>>>>> declaring a
>>>>>>>>>>>> stable
>>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>>>>> anyone
>>>>>>>>>>>> can
>>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>>>>> the
>>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>>>>>
>>>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>>>>
>>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>>>>> resolving a
>>>>>>>>>>>> few
>>>>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>>>>> that
>>>>>>>>>>>> will
>>>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>>>>
>>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>>>>> was a
>>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>>>>> any
>>>>>>>>>>>>> chance
>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>>>>> looking
>>>>>>>>> to
>>>>>>>>>>>>> have
>>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>>>> e
>>>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>

Re: Zookeeper with SSL release date

[Flavio Junqueira <fp...@apache.org>]

I was thinking server side actually. If the switch is off, any attempt to reconfig is a no-op and the server would log a warning. If you we want to propagate the error up, then yeah, we need to start worrying about compatibility and such.

I also would rather see us address it, although I can fully understand that there are other fixes and improvements in 3.5 that people are interested in.

-Flavio

> On 17 Mar 2016, at 00:46, Patrick Hunt <ph...@apache.org> wrote:
> 
> I'm not a huge fan of turning it off to be honest. Also just turning
> it off at the API level wouldn't be enough, we'd need to turn it off
> at the protocol level (otw it could still be accessed).
> 
> I'd rather see us address it than kick it down the road. It's a major
> feature of 3.5.
> 
> Patrick
> 
> On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
>> The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:
>> 
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>
>> 
>> Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.
>> 
>> About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.
>> 
>> I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.
>> 
>> -Flavio
>> 
>>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>>> 
>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
>>> are related to reconfig. Given this, and the fact that it is run in
>>> production since 2012 in multiple companies, I don't think its more
>>> unstable than any other part of ZooKeeper.
>>> 
>>> There are multiple reconfig-related bugs that turned out really difficult
>>> to debug without access to the actual system or at least to the Hudson
>>> machines where some tests are failing. In the past, Michi and I physically
>>> went to Hortonworks to debug one such issue, but this is of course not a
>>> good method, and we weren't able to arrange such a visit again.
>>> 
>>> Regarding making it optional - the reconfig logic has several different
>>> parts, some would be really difficult to disable using a configuration
>>> parameter. But the actual dynamic expansion of the cluster is triggered by
>>> the reconfig command, so it should not affect users who don't invoke it.
>>> 
>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
>>> 
>>>> I suppose we could give it a try. How do other folks feel about it?
>>>> 
>>>> -Flavio
>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>> 
>>>>> So, you could enable the dynamic reconfiguration feature behind a config
>>>>> option, and document that it should only be enabled experimentally, use
>>>> at
>>>>> your own risk.  Keep it off by default.  Allow only static config by
>>>>> default, until it's stable?
>>>>> 
>>>>> Jason
>>>>> 
>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>>> wrote:
>>>>> 
>>>>>> Hi Jason,
>>>>>> 
>>>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>>>> that's how that project manages to make such a separation. We don't
>>>> have
>>>>>> the same with ZooKeeper as the feature we are talking about is part of
>>>>> the
>>>>>> server and the only way I see of doing what you say is to turn off
>>>>>> features. More specifically, we'd need to disable the reconfig API and
>>>> do
>>>>>> not allow any change to the configuration, even though the code is
>>>> there.
>>>>>> 
>>>>>> Reconfig here refers to the ability of changing the configuration of an
>>>>>> ensemble (e.g., changing the set of servers).
>>>>>> 
>>>>>> -Flavio
>>>>>> 
>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>>>> 
>>>>>>> So, it would seem sensible to me to have a release where all features
>>>>> are
>>>>>>> stable, except where noted.  E.g. mark certain features as only
>>>> 'alpha
>>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>>>> happily
>>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>>>> 
>>>>>>> There's precedent for doing this sort of thing in other projects,
>>>> e.g.
>>>>> in
>>>>>>> Kafka, they've had several release where a new "Consumer API" is
>>>>> shipped
>>>>>>> that is available for beta-testing, but you can still just use the
>>>>> older
>>>>>>> stable consumer api, etc.
>>>>>>> 
>>>>>>> Jason
>>>>>>> 
>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>>> <powellm79@yahoo.com.invalid
>>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi Doug,
>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>>>> you
>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>>>> alpha
>>>>>>>> might not be fair, since it is far more stable then what most people
>>>>>>>> associate an alpha release to be.
>>>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>>>> for
>>>>>>>> you?.
>>>>>>>> There are many examples of 3.5.X being used in productions from my
>>>>>> limited
>>>>>>>> knowledge.
>>>>>>>> ThanksPowell.
>>>>>>>> 
>>>>>>>>  On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>>>> fpj@apache.org>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> None of us expected the reconfig changes to take this long to
>>>>> stabilize.
>>>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>>>> The
>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>>>> than
>>>>>>>> trying to work around it.
>>>>>>>> 
>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>>>> get
>>>>>>>> everyone to contribute more patches and code reviews, we should be
>>>>> able
>>>>>> to
>>>>>>>> do it sooner. After all, it is a community based effort, so the
>>>>>> community
>>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>>> 
>>>>>>>> -Flavio
>>>>>>>> 
>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>>>> the
>>>>>>>>> stable maintenance line, we are very conservative about
>>>> back-porting
>>>>> to
>>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>>> not
>>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>>>> managing
>>>>>>>>> risk.
>>>>>>>>> 
>>>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>>>> any
>>>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>>>> scope
>>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>>> focus
>>>>> on
>>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>>>> think
>>>>>>>>> this will help us accelerate the releases into beta and eventual
>>>> GA.
>>>>>>>>> 
>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>>>> thoughts
>>>>>>>>> from more experienced committers and PMC members about your
>>>> proposal
>>>>> to
>>>>>>>>> try to cut a stable release for a limited subset of what is in
>>>>>> branch-3.5
>>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>>> out
>>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>>>> another
>>>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>>>> 3.5
>>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>>>> "vanished"
>>>>>>>>> because of not meeting some stability criteria would be
>>>> undesirable.
>>>>>>>>> 
>>>>>>>>> --Chris Nauroth
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>>> 
>>>>>>>>>> Chris,
>>>>>>>>>> 
>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>>> others
>>>>>>>> (e.g.
>>>>>>>>>> if we don't care about certain new features, is it relatively
>>>>> stable)?
>>>>>>>>>> Would it be possible to cut out a version that only has the bits
>>>> we
>>>>>>>> think
>>>>>>>>>> are stable (and release that)?
>>>>>>>>>> 
>>>>>>>>>> From that timeline, and the historic release cadence, it would
>>>> seem
>>>>> to
>>>>>>>> be
>>>>>>>>>> a
>>>>>>>>>> years away before we get to the stable release?
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> 
>>>>>>>>>> Jason
>>>>>>>>>> 
>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hello Doug,
>>>>>>>>>>> 
>>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>>> 
>>>>>>>>>>> At this point, I think we're still pretty far away from
>>>> declaring a
>>>>>>>>>>> stable
>>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>>>> anyone
>>>>>>>>>>> can
>>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>>>> the
>>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>>>> 
>>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>>> 
>>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>>>> resolving a
>>>>>>>>>>> few
>>>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>>>> that
>>>>>>>>>>> will
>>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>>> 
>>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>>>> was a
>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>>>> any
>>>>>>>>>>>> chance
>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>>>> looking
>>>>>>>> to
>>>>>>>>>>>> have
>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>> 
>>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>>> e
>>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> 

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

ps. here's my jira dashboard, it tracks both 3.4 and 3.5
activity/status and gives some insight on progress on each:
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12327688
granted it only shows the current "in progress" releases (3.4.9/3.5.2)

Patrick

On Wed, Mar 16, 2016 at 10:56 PM, Patrick Hunt <ph...@apache.org> wrote:
> On Wed, Mar 16, 2016 at 10:24 PM, Alexander Shraer <sh...@gmail.com> wrote:
>> Here is a link for bugs marked as 3.5.2:
>> https://issues.apache.org/jira/browse/ZOOKEEPER/fixforversion/12331981/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-issues-panel
>>
>> The API issue Flavio mentioned is
>> https://issues.apache.org/jira/browse/ZOOKEEPER-2014 personally I don't
>> think this issue
>> is significant enough to block the release, but I may be wrong. ZooKeeper
>> supports ACLs and these can be used to solve the
>> issue described in the JIRA, at least until a better solution is in place.
>>
>
> I think the linked jira does a good job of covering the concerns
> expressed by multiple folks. Granted I may be more sensitive to the
> issues as I deal with enterprise users generally.
>
> Patrick
>
>>
>>
>> On Wed, Mar 16, 2016 at 9:33 PM, Jason Rosenberg <jb...@squareup.com> wrote:
>>
>>> Forgive me, as I have not long been an active member of the zookeeper
>>> community (other than as a grateful user over the last 3 years or so).
>>>
>>> If I understand correclty, 3.5.X has been alpha for several years or so
>>> now?  I think if there isn't a plan to have a stable release soon (say
>>> within another year), it's a problem.  It should be about having a regular
>>> release cycle, with the hope that new features and bug fixes become
>>> available in a reasonable time.  If one feature is just not stable, then it
>>> shouldn't block other features, etc.  Saying a feature is a major part of
>>> 3.5, doesn't quite make sense in this formulation.  Instead releases
>>> incorporate features, and if features get delayed, they can be postponed to
>>> a subsequent release.
>>>
>>> The issue is that we have people saying they don't want to fix things in
>>> 3.4.X (or back port fixes from 3.5.X to 3.4.X).  But if 3.5.X is still
>>> literally still years away (after having been under development for years),
>>> we should re-evaluate, no?
>>>
>>> Jason
>>>
>>> On Wed, Mar 16, 2016 at 8:46 PM, Patrick Hunt <ph...@apache.org> wrote:
>>>
>>> > I'm not a huge fan of turning it off to be honest. Also just turning
>>> > it off at the API level wouldn't be enough, we'd need to turn it off
>>> > at the protocol level (otw it could still be accessed).
>>> >
>>> > I'd rather see us address it than kick it down the road. It's a major
>>> > feature of 3.5.
>>> >
>>> > Patrick
>>> >
>>> > On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org>
>>> wrote:
>>> > > The main issue to sort out is stability of the API. There is a security
>>> > concern around the fact that clients can freely reconfigure the ensemble.
>>> > If we follow the plan that Pat proposed some time ago:
>>> > >
>>> > >
>>> >
>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>>> > <
>>> >
>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>>> > >
>>> > >
>>> > > Locking the API is the main step to move it to beta. Sorting out bugs
>>> is
>>> > definitely necessary, but it isn't the main thing that is keeping 3.5 in
>>> > alpha.
>>> > >
>>> > > About making it experimental, I was raising the option of having a
>>> > switch that disables the API calls, not the code. The reason why that
>>> could
>>> > work is that anyone using 3.5 who uses the "experimental" API must
>>> explicit
>>> > turn on the switch and enable the calls. If they do it, they need to be
>>> > aware that the API can change.
>>> > >
>>> > >  I must say that I haven't really looked closely into doing it, and I'm
>>> > not even entirely convinced that this is a good idea, but since Jason
>>> > raised the point, I'm exploring options.
>>> > >
>>> > > -Flavio
>>> > >
>>> > >> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>>> > >>
>>> > >> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
>>> only
>>> > 3-4
>>> > >> are related to reconfig. Given this, and the fact that it is run in
>>> > >> production since 2012 in multiple companies, I don't think its more
>>> > >> unstable than any other part of ZooKeeper.
>>> > >>
>>> > >> There are multiple reconfig-related bugs that turned out really
>>> > difficult
>>> > >> to debug without access to the actual system or at least to the Hudson
>>> > >> machines where some tests are failing. In the past, Michi and I
>>> > physically
>>> > >> went to Hortonworks to debug one such issue, but this is of course
>>> not a
>>> > >> good method, and we weren't able to arrange such a visit again.
>>> > >>
>>> > >> Regarding making it optional - the reconfig logic has several
>>> different
>>> > >> parts, some would be really difficult to disable using a configuration
>>> > >> parameter. But the actual dynamic expansion of the cluster is
>>> triggered
>>> > by
>>> > >> the reconfig command, so it should not affect users who don't invoke
>>> it.
>>> > >>
>>> > >> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org>
>>> > wrote:
>>> > >>
>>> > >>> I suppose we could give it a try. How do other folks feel about it?
>>> > >>>
>>> > >>> -Flavio
>>> > >>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>> > >>>
>>> > >>>> So, you could enable the dynamic reconfiguration feature behind a
>>> > config
>>> > >>>> option, and document that it should only be enabled experimentally,
>>> > use
>>> > >>> at
>>> > >>>> your own risk.  Keep it off by default.  Allow only static config by
>>> > >>>> default, until it's stable?
>>> > >>>>
>>> > >>>> Jason
>>> > >>>>
>>> > >>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>> > >>> wrote:
>>> > >>>>
>>> > >>>>> Hi Jason,
>>> > >>>>>
>>> > >>>>> The consumer in Kafka is pretty independent from the core
>>> (brokers),
>>> > >>>>> that's how that project manages to make such a separation. We don't
>>> > >>> have
>>> > >>>>> the same with ZooKeeper as the feature we are talking about is part
>>> > of
>>> > >>>> the
>>> > >>>>> server and the only way I see of doing what you say is to turn off
>>> > >>>>> features. More specifically, we'd need to disable the reconfig API
>>> > and
>>> > >>> do
>>> > >>>>> not allow any change to the configuration, even though the code is
>>> > >>> there.
>>> > >>>>>
>>> > >>>>> Reconfig here refers to the ability of changing the configuration
>>> of
>>> > an
>>> > >>>>> ensemble (e.g., changing the set of servers).
>>> > >>>>>
>>> > >>>>> -Flavio
>>> > >>>>>
>>> > >>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com>
>>> wrote:
>>> > >>>>>>
>>> > >>>>>> So, it would seem sensible to me to have a release where all
>>> > features
>>> > >>>> are
>>> > >>>>>> stable, except where noted.  E.g. mark certain features as only
>>> > >>> 'alpha
>>> > >>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible
>>> to
>>> > >>>>> happily
>>> > >>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>> > >>>>>>
>>> > >>>>>> There's precedent for doing this sort of thing in other projects,
>>> > >>> e.g.
>>> > >>>> in
>>> > >>>>>> Kafka, they've had several release where a new "Consumer API" is
>>> > >>>> shipped
>>> > >>>>>> that is available for beta-testing, but you can still just use the
>>> > >>>> older
>>> > >>>>>> stable consumer api, etc.
>>> > >>>>>>
>>> > >>>>>> Jason
>>> > >>>>>>
>>> > >>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>> > >>>>> <powellm79@yahoo.com.invalid
>>> > >>>>>>> wrote:
>>> > >>>>>>
>>> > >>>>>>> Hi Doug,
>>> > >>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or
>>> > have
>>> > >>>> you
>>> > >>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled
>>> > as
>>> > >>>>> alpha
>>> > >>>>>>> might not be fair, since it is far more stable then what most
>>> > people
>>> > >>>>>>> associate an alpha release to be.
>>> > >>>>>>> Perhaps if you do not use re-config feature may be it will just
>>> > work
>>> > >>>> for
>>> > >>>>>>> you?.
>>> > >>>>>>> There are many examples of 3.5.X being used in productions from
>>> my
>>> > >>>>> limited
>>> > >>>>>>> knowledge.
>>> > >>>>>>> ThanksPowell.
>>> > >>>>>>>
>>> > >>>>>>>   On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>> > >>>>> fpj@apache.org>
>>> > >>>>>>> wrote:
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>> None of us expected the reconfig changes to take this long to
>>> > >>>> stabilize.
>>> > >>>>>>> Until we get there, I don't think we can do anything else with
>>> 3.5.
>>> > >>>> The
>>> > >>>>>>> best bet we have is to work harder to bring 3.5 into a stable
>>> > rather
>>> > >>>>> than
>>> > >>>>>>> trying to work around it.
>>> > >>>>>>>
>>> > >>>>>>> There are lots of people interested in seeing 3.5 stable, and if
>>> we
>>> > >>>> get
>>> > >>>>>>> everyone to contribute more patches and code reviews, we should
>>> be
>>> > >>>> able
>>> > >>>>> to
>>> > >>>>>>> do it sooner. After all, it is a community based effort, so the
>>> > >>>>> community
>>> > >>>>>>> shouldn't rely on just 2-3 people doing the work.
>>> > >>>>>>>
>>> > >>>>>>> -Flavio
>>> > >>>>>>>
>>> > >>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
>>> cnauroth@hortonworks.com
>>> > >
>>> > >>>>>>> wrote:
>>> > >>>>>>>>
>>> > >>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4
>>> > is
>>> > >>>> the
>>> > >>>>>>>> stable maintenance line, we are very conservative about
>>> > >>> back-porting
>>> > >>>> to
>>> > >>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>> > >>> not
>>> > >>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>> > >>>>> managing
>>> > >>>>>>>> risk.
>>> > >>>>>>>>
>>> > >>>>>>>> Jason, your question about release cadence is a fair one.  If
>>> it's
>>> > >>>> any
>>> > >>>>>>>> consolation, we are now taking the approach of trying to limit
>>> the
>>> > >>>>> scope
>>> > >>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>> > >>> focus
>>> > >>>> on
>>> > >>>>>>>> stabilization: resolving blocker bugs and freezing public
>>> APIs.  I
>>> > >>>>> think
>>> > >>>>>>>> this will help us accelerate the releases into beta and eventual
>>> > >>> GA.
>>> > >>>>>>>>
>>> > >>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>> > >>>> thoughts
>>> > >>>>>>>> from more experienced committers and PMC members about your
>>> > >>> proposal
>>> > >>>> to
>>> > >>>>>>>> try to cut a stable release for a limited subset of what is in
>>> > >>>>> branch-3.5
>>> > >>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>> > >>> out
>>> > >>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>> > >>>>> another
>>> > >>>>>>>> release line for an already resource-constrained volunteer staff
>>> > to
>>> > >>>>>>>> manage.  I'd prefer to dedicate those limited resources to
>>> overall
>>> > >>>> 3.5
>>> > >>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>> > >>>>> "vanished"
>>> > >>>>>>>> because of not meeting some stability criteria would be
>>> > >>> undesirable.
>>> > >>>>>>>>
>>> > >>>>>>>> --Chris Nauroth
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com>
>>> wrote:
>>> > >>>>>>>>
>>> > >>>>>>>>> Chris,
>>> > >>>>>>>>>
>>> > >>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>> > >>> others
>>> > >>>>>>> (e.g.
>>> > >>>>>>>>> if we don't care about certain new features, is it relatively
>>> > >>>> stable)?
>>> > >>>>>>>>> Would it be possible to cut out a version that only has the
>>> bits
>>> > >>> we
>>> > >>>>>>> think
>>> > >>>>>>>>> are stable (and release that)?
>>> > >>>>>>>>>
>>> > >>>>>>>>> From that timeline, and the historic release cadence, it would
>>> > >>> seem
>>> > >>>> to
>>> > >>>>>>> be
>>> > >>>>>>>>> a
>>> > >>>>>>>>> years away before we get to the stable release?
>>> > >>>>>>>>>
>>> > >>>>>>>>> Thanks,
>>> > >>>>>>>>>
>>> > >>>>>>>>> Jason
>>> > >>>>>>>>>
>>> > >>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>> > >>>>>>> cnauroth@hortonworks.com>
>>> > >>>>>>>>> wrote:
>>> > >>>>>>>>>
>>> > >>>>>>>>>> Hello Doug,
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> Thanks for your interest in the SSL feature!
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> At this point, I think we're still pretty far away from
>>> > >>> declaring a
>>> > >>>>>>>>>> stable
>>> > >>>>>>>>>> release in the 3.5 line.  I don't think we're close enough
>>> that
>>> > >>>>> anyone
>>> > >>>>>>>>>> can
>>> > >>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
>>> describes
>>> > >>> the
>>> > >>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> https://s.apache.org/ADK1
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>> > >>>> resolving a
>>> > >>>>>>>>>> few
>>> > >>>>>>>>>> more blockers before we produce a release candidate.
>>> Hopefully
>>> > >>>> that
>>> > >>>>>>>>>> will
>>> > >>>>>>>>>> get done in the next few weeks.
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> --Chris Nauroth
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>> I know it's only been a few months, but I was wondering if
>>> > there
>>> > >>>>> was a
>>> > >>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is
>>> > there
>>> > >>>> any
>>> > >>>>>>>>>>> chance
>>> > >>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>> > >>>> looking
>>> > >>>>>>> to
>>> > >>>>>>>>>>> have
>>> > >>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> --
>>> > >>>>>>>>>>> View this message in context:
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>
>>> > >>>>>
>>> > >>>>
>>> > >>>
>>> >
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>> > >>>>>>>>>> e
>>> > >>>>>>>>>>> -tp7581744p7582136.html
>>> > >>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
>>> > Nabble.com.
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>
>>> > >>>>>
>>> > >>>>
>>> > >>>
>>> > >
>>> >
>>>

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

On Wed, Mar 16, 2016 at 10:24 PM, Alexander Shraer <sh...@gmail.com> wrote:
> Here is a link for bugs marked as 3.5.2:
> https://issues.apache.org/jira/browse/ZOOKEEPER/fixforversion/12331981/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-issues-panel
>
> The API issue Flavio mentioned is
> https://issues.apache.org/jira/browse/ZOOKEEPER-2014 personally I don't
> think this issue
> is significant enough to block the release, but I may be wrong. ZooKeeper
> supports ACLs and these can be used to solve the
> issue described in the JIRA, at least until a better solution is in place.
>

I think the linked jira does a good job of covering the concerns
expressed by multiple folks. Granted I may be more sensitive to the
issues as I deal with enterprise users generally.

Patrick

>
>
> On Wed, Mar 16, 2016 at 9:33 PM, Jason Rosenberg <jb...@squareup.com> wrote:
>
>> Forgive me, as I have not long been an active member of the zookeeper
>> community (other than as a grateful user over the last 3 years or so).
>>
>> If I understand correclty, 3.5.X has been alpha for several years or so
>> now?  I think if there isn't a plan to have a stable release soon (say
>> within another year), it's a problem.  It should be about having a regular
>> release cycle, with the hope that new features and bug fixes become
>> available in a reasonable time.  If one feature is just not stable, then it
>> shouldn't block other features, etc.  Saying a feature is a major part of
>> 3.5, doesn't quite make sense in this formulation.  Instead releases
>> incorporate features, and if features get delayed, they can be postponed to
>> a subsequent release.
>>
>> The issue is that we have people saying they don't want to fix things in
>> 3.4.X (or back port fixes from 3.5.X to 3.4.X).  But if 3.5.X is still
>> literally still years away (after having been under development for years),
>> we should re-evaluate, no?
>>
>> Jason
>>
>> On Wed, Mar 16, 2016 at 8:46 PM, Patrick Hunt <ph...@apache.org> wrote:
>>
>> > I'm not a huge fan of turning it off to be honest. Also just turning
>> > it off at the API level wouldn't be enough, we'd need to turn it off
>> > at the protocol level (otw it could still be accessed).
>> >
>> > I'd rather see us address it than kick it down the road. It's a major
>> > feature of 3.5.
>> >
>> > Patrick
>> >
>> > On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org>
>> wrote:
>> > > The main issue to sort out is stability of the API. There is a security
>> > concern around the fact that clients can freely reconfigure the ensemble.
>> > If we follow the plan that Pat proposed some time ago:
>> > >
>> > >
>> >
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> > <
>> >
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> > >
>> > >
>> > > Locking the API is the main step to move it to beta. Sorting out bugs
>> is
>> > definitely necessary, but it isn't the main thing that is keeping 3.5 in
>> > alpha.
>> > >
>> > > About making it experimental, I was raising the option of having a
>> > switch that disables the API calls, not the code. The reason why that
>> could
>> > work is that anyone using 3.5 who uses the "experimental" API must
>> explicit
>> > turn on the switch and enable the calls. If they do it, they need to be
>> > aware that the API can change.
>> > >
>> > >  I must say that I haven't really looked closely into doing it, and I'm
>> > not even entirely convinced that this is a good idea, but since Jason
>> > raised the point, I'm exploring options.
>> > >
>> > > -Flavio
>> > >
>> > >> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>> > >>
>> > >> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
>> only
>> > 3-4
>> > >> are related to reconfig. Given this, and the fact that it is run in
>> > >> production since 2012 in multiple companies, I don't think its more
>> > >> unstable than any other part of ZooKeeper.
>> > >>
>> > >> There are multiple reconfig-related bugs that turned out really
>> > difficult
>> > >> to debug without access to the actual system or at least to the Hudson
>> > >> machines where some tests are failing. In the past, Michi and I
>> > physically
>> > >> went to Hortonworks to debug one such issue, but this is of course
>> not a
>> > >> good method, and we weren't able to arrange such a visit again.
>> > >>
>> > >> Regarding making it optional - the reconfig logic has several
>> different
>> > >> parts, some would be really difficult to disable using a configuration
>> > >> parameter. But the actual dynamic expansion of the cluster is
>> triggered
>> > by
>> > >> the reconfig command, so it should not affect users who don't invoke
>> it.
>> > >>
>> > >> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org>
>> > wrote:
>> > >>
>> > >>> I suppose we could give it a try. How do other folks feel about it?
>> > >>>
>> > >>> -Flavio
>> > >>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>> > >>>
>> > >>>> So, you could enable the dynamic reconfiguration feature behind a
>> > config
>> > >>>> option, and document that it should only be enabled experimentally,
>> > use
>> > >>> at
>> > >>>> your own risk.  Keep it off by default.  Allow only static config by
>> > >>>> default, until it's stable?
>> > >>>>
>> > >>>> Jason
>> > >>>>
>> > >>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>> > >>> wrote:
>> > >>>>
>> > >>>>> Hi Jason,
>> > >>>>>
>> > >>>>> The consumer in Kafka is pretty independent from the core
>> (brokers),
>> > >>>>> that's how that project manages to make such a separation. We don't
>> > >>> have
>> > >>>>> the same with ZooKeeper as the feature we are talking about is part
>> > of
>> > >>>> the
>> > >>>>> server and the only way I see of doing what you say is to turn off
>> > >>>>> features. More specifically, we'd need to disable the reconfig API
>> > and
>> > >>> do
>> > >>>>> not allow any change to the configuration, even though the code is
>> > >>> there.
>> > >>>>>
>> > >>>>> Reconfig here refers to the ability of changing the configuration
>> of
>> > an
>> > >>>>> ensemble (e.g., changing the set of servers).
>> > >>>>>
>> > >>>>> -Flavio
>> > >>>>>
>> > >>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com>
>> wrote:
>> > >>>>>>
>> > >>>>>> So, it would seem sensible to me to have a release where all
>> > features
>> > >>>> are
>> > >>>>>> stable, except where noted.  E.g. mark certain features as only
>> > >>> 'alpha
>> > >>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible
>> to
>> > >>>>> happily
>> > >>>>>> use 3.5.X without exercising the unstable re-config bits?).
>> > >>>>>>
>> > >>>>>> There's precedent for doing this sort of thing in other projects,
>> > >>> e.g.
>> > >>>> in
>> > >>>>>> Kafka, they've had several release where a new "Consumer API" is
>> > >>>> shipped
>> > >>>>>> that is available for beta-testing, but you can still just use the
>> > >>>> older
>> > >>>>>> stable consumer api, etc.
>> > >>>>>>
>> > >>>>>> Jason
>> > >>>>>>
>> > >>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>> > >>>>> <powellm79@yahoo.com.invalid
>> > >>>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> Hi Doug,
>> > >>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or
>> > have
>> > >>>> you
>> > >>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled
>> > as
>> > >>>>> alpha
>> > >>>>>>> might not be fair, since it is far more stable then what most
>> > people
>> > >>>>>>> associate an alpha release to be.
>> > >>>>>>> Perhaps if you do not use re-config feature may be it will just
>> > work
>> > >>>> for
>> > >>>>>>> you?.
>> > >>>>>>> There are many examples of 3.5.X being used in productions from
>> my
>> > >>>>> limited
>> > >>>>>>> knowledge.
>> > >>>>>>> ThanksPowell.
>> > >>>>>>>
>> > >>>>>>>   On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>> > >>>>> fpj@apache.org>
>> > >>>>>>> wrote:
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> None of us expected the reconfig changes to take this long to
>> > >>>> stabilize.
>> > >>>>>>> Until we get there, I don't think we can do anything else with
>> 3.5.
>> > >>>> The
>> > >>>>>>> best bet we have is to work harder to bring 3.5 into a stable
>> > rather
>> > >>>>> than
>> > >>>>>>> trying to work around it.
>> > >>>>>>>
>> > >>>>>>> There are lots of people interested in seeing 3.5 stable, and if
>> we
>> > >>>> get
>> > >>>>>>> everyone to contribute more patches and code reviews, we should
>> be
>> > >>>> able
>> > >>>>> to
>> > >>>>>>> do it sooner. After all, it is a community based effort, so the
>> > >>>>> community
>> > >>>>>>> shouldn't rely on just 2-3 people doing the work.
>> > >>>>>>>
>> > >>>>>>> -Flavio
>> > >>>>>>>
>> > >>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
>> cnauroth@hortonworks.com
>> > >
>> > >>>>>>> wrote:
>> > >>>>>>>>
>> > >>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4
>> > is
>> > >>>> the
>> > >>>>>>>> stable maintenance line, we are very conservative about
>> > >>> back-porting
>> > >>>> to
>> > >>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>> > >>> not
>> > >>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>> > >>>>> managing
>> > >>>>>>>> risk.
>> > >>>>>>>>
>> > >>>>>>>> Jason, your question about release cadence is a fair one.  If
>> it's
>> > >>>> any
>> > >>>>>>>> consolation, we are now taking the approach of trying to limit
>> the
>> > >>>>> scope
>> > >>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>> > >>> focus
>> > >>>> on
>> > >>>>>>>> stabilization: resolving blocker bugs and freezing public
>> APIs.  I
>> > >>>>> think
>> > >>>>>>>> this will help us accelerate the releases into beta and eventual
>> > >>> GA.
>> > >>>>>>>>
>> > >>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>> > >>>> thoughts
>> > >>>>>>>> from more experienced committers and PMC members about your
>> > >>> proposal
>> > >>>> to
>> > >>>>>>>> try to cut a stable release for a limited subset of what is in
>> > >>>>> branch-3.5
>> > >>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>> > >>> out
>> > >>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>> > >>>>> another
>> > >>>>>>>> release line for an already resource-constrained volunteer staff
>> > to
>> > >>>>>>>> manage.  I'd prefer to dedicate those limited resources to
>> overall
>> > >>>> 3.5
>> > >>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>> > >>>>> "vanished"
>> > >>>>>>>> because of not meeting some stability criteria would be
>> > >>> undesirable.
>> > >>>>>>>>
>> > >>>>>>>> --Chris Nauroth
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com>
>> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> Chris,
>> > >>>>>>>>>
>> > >>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>> > >>> others
>> > >>>>>>> (e.g.
>> > >>>>>>>>> if we don't care about certain new features, is it relatively
>> > >>>> stable)?
>> > >>>>>>>>> Would it be possible to cut out a version that only has the
>> bits
>> > >>> we
>> > >>>>>>> think
>> > >>>>>>>>> are stable (and release that)?
>> > >>>>>>>>>
>> > >>>>>>>>> From that timeline, and the historic release cadence, it would
>> > >>> seem
>> > >>>> to
>> > >>>>>>> be
>> > >>>>>>>>> a
>> > >>>>>>>>> years away before we get to the stable release?
>> > >>>>>>>>>
>> > >>>>>>>>> Thanks,
>> > >>>>>>>>>
>> > >>>>>>>>> Jason
>> > >>>>>>>>>
>> > >>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>> > >>>>>>> cnauroth@hortonworks.com>
>> > >>>>>>>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> Hello Doug,
>> > >>>>>>>>>>
>> > >>>>>>>>>> Thanks for your interest in the SSL feature!
>> > >>>>>>>>>>
>> > >>>>>>>>>> At this point, I think we're still pretty far away from
>> > >>> declaring a
>> > >>>>>>>>>> stable
>> > >>>>>>>>>> release in the 3.5 line.  I don't think we're close enough
>> that
>> > >>>>> anyone
>> > >>>>>>>>>> can
>> > >>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
>> describes
>> > >>> the
>> > >>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>> > >>>>>>>>>>
>> > >>>>>>>>>> https://s.apache.org/ADK1
>> > >>>>>>>>>>
>> > >>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>> > >>>> resolving a
>> > >>>>>>>>>> few
>> > >>>>>>>>>> more blockers before we produce a release candidate.
>> Hopefully
>> > >>>> that
>> > >>>>>>>>>> will
>> > >>>>>>>>>> get done in the next few weeks.
>> > >>>>>>>>>>
>> > >>>>>>>>>> --Chris Nauroth
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>>> I know it's only been a few months, but I was wondering if
>> > there
>> > >>>>> was a
>> > >>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is
>> > there
>> > >>>> any
>> > >>>>>>>>>>> chance
>> > >>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>> > >>>> looking
>> > >>>>>>> to
>> > >>>>>>>>>>> have
>> > >>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> --
>> > >>>>>>>>>>> View this message in context:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>
>> > >>>>>
>> > >>>>
>> > >>>
>> >
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>> > >>>>>>>>>> e
>> > >>>>>>>>>>> -tp7581744p7582136.html
>> > >>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
>> > Nabble.com.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>
>> > >>>
>> > >
>> >
>>

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

Here is a link for bugs marked as 3.5.2:
https://issues.apache.org/jira/browse/ZOOKEEPER/fixforversion/12331981/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-issues-panel

The API issue Flavio mentioned is
https://issues.apache.org/jira/browse/ZOOKEEPER-2014 personally I don't
think this issue
is significant enough to block the release, but I may be wrong. ZooKeeper
supports ACLs and these can be used to solve the
issue described in the JIRA, at least until a better solution is in place.


Alex


On Wed, Mar 16, 2016 at 9:33 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> Forgive me, as I have not long been an active member of the zookeeper
> community (other than as a grateful user over the last 3 years or so).
>
> If I understand correclty, 3.5.X has been alpha for several years or so
> now?  I think if there isn't a plan to have a stable release soon (say
> within another year), it's a problem.  It should be about having a regular
> release cycle, with the hope that new features and bug fixes become
> available in a reasonable time.  If one feature is just not stable, then it
> shouldn't block other features, etc.  Saying a feature is a major part of
> 3.5, doesn't quite make sense in this formulation.  Instead releases
> incorporate features, and if features get delayed, they can be postponed to
> a subsequent release.
>
> The issue is that we have people saying they don't want to fix things in
> 3.4.X (or back port fixes from 3.5.X to 3.4.X).  But if 3.5.X is still
> literally still years away (after having been under development for years),
> we should re-evaluate, no?
>
> Jason
>
> On Wed, Mar 16, 2016 at 8:46 PM, Patrick Hunt <ph...@apache.org> wrote:
>
> > I'm not a huge fan of turning it off to be honest. Also just turning
> > it off at the API level wouldn't be enough, we'd need to turn it off
> > at the protocol level (otw it could still be accessed).
> >
> > I'd rather see us address it than kick it down the road. It's a major
> > feature of 3.5.
> >
> > Patrick
> >
> > On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org>
> wrote:
> > > The main issue to sort out is stability of the API. There is a security
> > concern around the fact that clients can freely reconfigure the ensemble.
> > If we follow the plan that Pat proposed some time ago:
> > >
> > >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > <
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> > >
> > >
> > > Locking the API is the main step to move it to beta. Sorting out bugs
> is
> > definitely necessary, but it isn't the main thing that is keeping 3.5 in
> > alpha.
> > >
> > > About making it experimental, I was raising the option of having a
> > switch that disables the API calls, not the code. The reason why that
> could
> > work is that anyone using 3.5 who uses the "experimental" API must
> explicit
> > turn on the switch and enable the calls. If they do it, they need to be
> > aware that the API can change.
> > >
> > >  I must say that I haven't really looked closely into doing it, and I'm
> > not even entirely convinced that this is a good idea, but since Jason
> > raised the point, I'm exploring options.
> > >
> > > -Flavio
> > >
> > >> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
> > >>
> > >> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
> only
> > 3-4
> > >> are related to reconfig. Given this, and the fact that it is run in
> > >> production since 2012 in multiple companies, I don't think its more
> > >> unstable than any other part of ZooKeeper.
> > >>
> > >> There are multiple reconfig-related bugs that turned out really
> > difficult
> > >> to debug without access to the actual system or at least to the Hudson
> > >> machines where some tests are failing. In the past, Michi and I
> > physically
> > >> went to Hortonworks to debug one such issue, but this is of course
> not a
> > >> good method, and we weren't able to arrange such a visit again.
> > >>
> > >> Regarding making it optional - the reconfig logic has several
> different
> > >> parts, some would be really difficult to disable using a configuration
> > >> parameter. But the actual dynamic expansion of the cluster is
> triggered
> > by
> > >> the reconfig command, so it should not affect users who don't invoke
> it.
> > >>
> > >> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org>
> > wrote:
> > >>
> > >>> I suppose we could give it a try. How do other folks feel about it?
> > >>>
> > >>> -Flavio
> > >>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
> > >>>
> > >>>> So, you could enable the dynamic reconfiguration feature behind a
> > config
> > >>>> option, and document that it should only be enabled experimentally,
> > use
> > >>> at
> > >>>> your own risk.  Keep it off by default.  Allow only static config by
> > >>>> default, until it's stable?
> > >>>>
> > >>>> Jason
> > >>>>
> > >>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
> > >>> wrote:
> > >>>>
> > >>>>> Hi Jason,
> > >>>>>
> > >>>>> The consumer in Kafka is pretty independent from the core
> (brokers),
> > >>>>> that's how that project manages to make such a separation. We don't
> > >>> have
> > >>>>> the same with ZooKeeper as the feature we are talking about is part
> > of
> > >>>> the
> > >>>>> server and the only way I see of doing what you say is to turn off
> > >>>>> features. More specifically, we'd need to disable the reconfig API
> > and
> > >>> do
> > >>>>> not allow any change to the configuration, even though the code is
> > >>> there.
> > >>>>>
> > >>>>> Reconfig here refers to the ability of changing the configuration
> of
> > an
> > >>>>> ensemble (e.g., changing the set of servers).
> > >>>>>
> > >>>>> -Flavio
> > >>>>>
> > >>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com>
> wrote:
> > >>>>>>
> > >>>>>> So, it would seem sensible to me to have a release where all
> > features
> > >>>> are
> > >>>>>> stable, except where noted.  E.g. mark certain features as only
> > >>> 'alpha
> > >>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible
> to
> > >>>>> happily
> > >>>>>> use 3.5.X without exercising the unstable re-config bits?).
> > >>>>>>
> > >>>>>> There's precedent for doing this sort of thing in other projects,
> > >>> e.g.
> > >>>> in
> > >>>>>> Kafka, they've had several release where a new "Consumer API" is
> > >>>> shipped
> > >>>>>> that is available for beta-testing, but you can still just use the
> > >>>> older
> > >>>>>> stable consumer api, etc.
> > >>>>>>
> > >>>>>> Jason
> > >>>>>>
> > >>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > >>>>> <powellm79@yahoo.com.invalid
> > >>>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi Doug,
> > >>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or
> > have
> > >>>> you
> > >>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled
> > as
> > >>>>> alpha
> > >>>>>>> might not be fair, since it is far more stable then what most
> > people
> > >>>>>>> associate an alpha release to be.
> > >>>>>>> Perhaps if you do not use re-config feature may be it will just
> > work
> > >>>> for
> > >>>>>>> you?.
> > >>>>>>> There are many examples of 3.5.X being used in productions from
> my
> > >>>>> limited
> > >>>>>>> knowledge.
> > >>>>>>> ThanksPowell.
> > >>>>>>>
> > >>>>>>>   On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> > >>>>> fpj@apache.org>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> None of us expected the reconfig changes to take this long to
> > >>>> stabilize.
> > >>>>>>> Until we get there, I don't think we can do anything else with
> 3.5.
> > >>>> The
> > >>>>>>> best bet we have is to work harder to bring 3.5 into a stable
> > rather
> > >>>>> than
> > >>>>>>> trying to work around it.
> > >>>>>>>
> > >>>>>>> There are lots of people interested in seeing 3.5 stable, and if
> we
> > >>>> get
> > >>>>>>> everyone to contribute more patches and code reviews, we should
> be
> > >>>> able
> > >>>>> to
> > >>>>>>> do it sooner. After all, it is a community based effort, so the
> > >>>>> community
> > >>>>>>> shouldn't rely on just 2-3 people doing the work.
> > >>>>>>>
> > >>>>>>> -Flavio
> > >>>>>>>
> > >>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <
> cnauroth@hortonworks.com
> > >
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4
> > is
> > >>>> the
> > >>>>>>>> stable maintenance line, we are very conservative about
> > >>> back-porting
> > >>>> to
> > >>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
> > >>> not
> > >>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
> > >>>>> managing
> > >>>>>>>> risk.
> > >>>>>>>>
> > >>>>>>>> Jason, your question about release cadence is a fair one.  If
> it's
> > >>>> any
> > >>>>>>>> consolation, we are now taking the approach of trying to limit
> the
> > >>>>> scope
> > >>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
> > >>> focus
> > >>>> on
> > >>>>>>>> stabilization: resolving blocker bugs and freezing public
> APIs.  I
> > >>>>> think
> > >>>>>>>> this will help us accelerate the releases into beta and eventual
> > >>> GA.
> > >>>>>>>>
> > >>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
> > >>>> thoughts
> > >>>>>>>> from more experienced committers and PMC members about your
> > >>> proposal
> > >>>> to
> > >>>>>>>> try to cut a stable release for a limited subset of what is in
> > >>>>> branch-3.5
> > >>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
> > >>> out
> > >>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
> > >>>>> another
> > >>>>>>>> release line for an already resource-constrained volunteer staff
> > to
> > >>>>>>>> manage.  I'd prefer to dedicate those limited resources to
> overall
> > >>>> 3.5
> > >>>>>>>> stabilization.  Also, a 3.5 release in which certain features
> > >>>>> "vanished"
> > >>>>>>>> because of not meeting some stability criteria would be
> > >>> undesirable.
> > >>>>>>>>
> > >>>>>>>> --Chris Nauroth
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com>
> wrote:
> > >>>>>>>>
> > >>>>>>>>> Chris,
> > >>>>>>>>>
> > >>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
> > >>> others
> > >>>>>>> (e.g.
> > >>>>>>>>> if we don't care about certain new features, is it relatively
> > >>>> stable)?
> > >>>>>>>>> Would it be possible to cut out a version that only has the
> bits
> > >>> we
> > >>>>>>> think
> > >>>>>>>>> are stable (and release that)?
> > >>>>>>>>>
> > >>>>>>>>> From that timeline, and the historic release cadence, it would
> > >>> seem
> > >>>> to
> > >>>>>>> be
> > >>>>>>>>> a
> > >>>>>>>>> years away before we get to the stable release?
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>>
> > >>>>>>>>> Jason
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> > >>>>>>> cnauroth@hortonworks.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hello Doug,
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks for your interest in the SSL feature!
> > >>>>>>>>>>
> > >>>>>>>>>> At this point, I think we're still pretty far away from
> > >>> declaring a
> > >>>>>>>>>> stable
> > >>>>>>>>>> release in the 3.5 line.  I don't think we're close enough
> that
> > >>>>> anyone
> > >>>>>>>>>> can
> > >>>>>>>>>> offer a reliable ETA.  This is an earlier thread that
> describes
> > >>> the
> > >>>>>>>>>> high-level strategy for release planning in the 3.5 line:
> > >>>>>>>>>>
> > >>>>>>>>>> https://s.apache.org/ADK1
> > >>>>>>>>>>
> > >>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
> > >>>> resolving a
> > >>>>>>>>>> few
> > >>>>>>>>>> more blockers before we produce a release candidate.
> Hopefully
> > >>>> that
> > >>>>>>>>>> will
> > >>>>>>>>>> get done in the next few weeks.
> > >>>>>>>>>>
> > >>>>>>>>>> --Chris Nauroth
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> I know it's only been a few months, but I was wondering if
> > there
> > >>>>> was a
> > >>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is
> > there
> > >>>> any
> > >>>>>>>>>>> chance
> > >>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
> > >>>> looking
> > >>>>>>> to
> > >>>>>>>>>>> have
> > >>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> View this message in context:
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>>
> > >>>
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > >>>>>>>>>> e
> > >>>>>>>>>>> -tp7581744p7582136.html
> > >>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
> > Nabble.com.
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >
> >
>

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

We have been having regular releases, they are just not yet stable
releases. That's a factor of a few things, including that we have a
high bar and that folks contributing to the project are volunteers
with limited time. We're getting there though and with more
participation (hint hint) we'd get there faster.

Patrick

On Wed, Mar 16, 2016 at 9:33 PM, Jason Rosenberg <jb...@squareup.com> wrote:
> Forgive me, as I have not long been an active member of the zookeeper
> community (other than as a grateful user over the last 3 years or so).
>
> If I understand correclty, 3.5.X has been alpha for several years or so
> now?  I think if there isn't a plan to have a stable release soon (say
> within another year), it's a problem.  It should be about having a regular
> release cycle, with the hope that new features and bug fixes become
> available in a reasonable time.  If one feature is just not stable, then it
> shouldn't block other features, etc.  Saying a feature is a major part of
> 3.5, doesn't quite make sense in this formulation.  Instead releases
> incorporate features, and if features get delayed, they can be postponed to
> a subsequent release.
>
> The issue is that we have people saying they don't want to fix things in
> 3.4.X (or back port fixes from 3.5.X to 3.4.X).  But if 3.5.X is still
> literally still years away (after having been under development for years),
> we should re-evaluate, no?
>
> Jason
>
> On Wed, Mar 16, 2016 at 8:46 PM, Patrick Hunt <ph...@apache.org> wrote:
>
>> I'm not a huge fan of turning it off to be honest. Also just turning
>> it off at the API level wouldn't be enough, we'd need to turn it off
>> at the protocol level (otw it could still be accessed).
>>
>> I'd rather see us address it than kick it down the road. It's a major
>> feature of 3.5.
>>
>> Patrick
>>
>> On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
>> > The main issue to sort out is stability of the API. There is a security
>> concern around the fact that clients can freely reconfigure the ensemble.
>> If we follow the plan that Pat proposed some time ago:
>> >
>> >
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> <
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> >
>> >
>> > Locking the API is the main step to move it to beta. Sorting out bugs is
>> definitely necessary, but it isn't the main thing that is keeping 3.5 in
>> alpha.
>> >
>> > About making it experimental, I was raising the option of having a
>> switch that disables the API calls, not the code. The reason why that could
>> work is that anyone using 3.5 who uses the "experimental" API must explicit
>> turn on the switch and enable the calls. If they do it, they need to be
>> aware that the API can change.
>> >
>> >  I must say that I haven't really looked closely into doing it, and I'm
>> not even entirely convinced that this is a good idea, but since Jason
>> raised the point, I'm exploring options.
>> >
>> > -Flavio
>> >
>> >> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>> >>
>> >> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only
>> 3-4
>> >> are related to reconfig. Given this, and the fact that it is run in
>> >> production since 2012 in multiple companies, I don't think its more
>> >> unstable than any other part of ZooKeeper.
>> >>
>> >> There are multiple reconfig-related bugs that turned out really
>> difficult
>> >> to debug without access to the actual system or at least to the Hudson
>> >> machines where some tests are failing. In the past, Michi and I
>> physically
>> >> went to Hortonworks to debug one such issue, but this is of course not a
>> >> good method, and we weren't able to arrange such a visit again.
>> >>
>> >> Regarding making it optional - the reconfig logic has several different
>> >> parts, some would be really difficult to disable using a configuration
>> >> parameter. But the actual dynamic expansion of the cluster is triggered
>> by
>> >> the reconfig command, so it should not affect users who don't invoke it.
>> >>
>> >> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org>
>> wrote:
>> >>
>> >>> I suppose we could give it a try. How do other folks feel about it?
>> >>>
>> >>> -Flavio
>> >>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>> >>>
>> >>>> So, you could enable the dynamic reconfiguration feature behind a
>> config
>> >>>> option, and document that it should only be enabled experimentally,
>> use
>> >>> at
>> >>>> your own risk.  Keep it off by default.  Allow only static config by
>> >>>> default, until it's stable?
>> >>>>
>> >>>> Jason
>> >>>>
>> >>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>> >>> wrote:
>> >>>>
>> >>>>> Hi Jason,
>> >>>>>
>> >>>>> The consumer in Kafka is pretty independent from the core (brokers),
>> >>>>> that's how that project manages to make such a separation. We don't
>> >>> have
>> >>>>> the same with ZooKeeper as the feature we are talking about is part
>> of
>> >>>> the
>> >>>>> server and the only way I see of doing what you say is to turn off
>> >>>>> features. More specifically, we'd need to disable the reconfig API
>> and
>> >>> do
>> >>>>> not allow any change to the configuration, even though the code is
>> >>> there.
>> >>>>>
>> >>>>> Reconfig here refers to the ability of changing the configuration of
>> an
>> >>>>> ensemble (e.g., changing the set of servers).
>> >>>>>
>> >>>>> -Flavio
>> >>>>>
>> >>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>> >>>>>>
>> >>>>>> So, it would seem sensible to me to have a release where all
>> features
>> >>>> are
>> >>>>>> stable, except where noted.  E.g. mark certain features as only
>> >>> 'alpha
>> >>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>> >>>>> happily
>> >>>>>> use 3.5.X without exercising the unstable re-config bits?).
>> >>>>>>
>> >>>>>> There's precedent for doing this sort of thing in other projects,
>> >>> e.g.
>> >>>> in
>> >>>>>> Kafka, they've had several release where a new "Consumer API" is
>> >>>> shipped
>> >>>>>> that is available for beta-testing, but you can still just use the
>> >>>> older
>> >>>>>> stable consumer api, etc.
>> >>>>>>
>> >>>>>> Jason
>> >>>>>>
>> >>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>> >>>>> <powellm79@yahoo.com.invalid
>> >>>>>>> wrote:
>> >>>>>>
>> >>>>>>> Hi Doug,
>> >>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or
>> have
>> >>>> you
>> >>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled
>> as
>> >>>>> alpha
>> >>>>>>> might not be fair, since it is far more stable then what most
>> people
>> >>>>>>> associate an alpha release to be.
>> >>>>>>> Perhaps if you do not use re-config feature may be it will just
>> work
>> >>>> for
>> >>>>>>> you?.
>> >>>>>>> There are many examples of 3.5.X being used in productions from my
>> >>>>> limited
>> >>>>>>> knowledge.
>> >>>>>>> ThanksPowell.
>> >>>>>>>
>> >>>>>>>   On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>> >>>>> fpj@apache.org>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> None of us expected the reconfig changes to take this long to
>> >>>> stabilize.
>> >>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>> >>>> The
>> >>>>>>> best bet we have is to work harder to bring 3.5 into a stable
>> rather
>> >>>>> than
>> >>>>>>> trying to work around it.
>> >>>>>>>
>> >>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>> >>>> get
>> >>>>>>> everyone to contribute more patches and code reviews, we should be
>> >>>> able
>> >>>>> to
>> >>>>>>> do it sooner. After all, it is a community based effort, so the
>> >>>>> community
>> >>>>>>> shouldn't rely on just 2-3 people doing the work.
>> >>>>>>>
>> >>>>>>> -Flavio
>> >>>>>>>
>> >>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cnauroth@hortonworks.com
>> >
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4
>> is
>> >>>> the
>> >>>>>>>> stable maintenance line, we are very conservative about
>> >>> back-porting
>> >>>> to
>> >>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>> >>> not
>> >>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>> >>>>> managing
>> >>>>>>>> risk.
>> >>>>>>>>
>> >>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>> >>>> any
>> >>>>>>>> consolation, we are now taking the approach of trying to limit the
>> >>>>> scope
>> >>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>> >>> focus
>> >>>> on
>> >>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>> >>>>> think
>> >>>>>>>> this will help us accelerate the releases into beta and eventual
>> >>> GA.
>> >>>>>>>>
>> >>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>> >>>> thoughts
>> >>>>>>>> from more experienced committers and PMC members about your
>> >>> proposal
>> >>>> to
>> >>>>>>>> try to cut a stable release for a limited subset of what is in
>> >>>>> branch-3.5
>> >>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>> >>> out
>> >>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>> >>>>> another
>> >>>>>>>> release line for an already resource-constrained volunteer staff
>> to
>> >>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>> >>>> 3.5
>> >>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>> >>>>> "vanished"
>> >>>>>>>> because of not meeting some stability criteria would be
>> >>> undesirable.
>> >>>>>>>>
>> >>>>>>>> --Chris Nauroth
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>> >>>>>>>>
>> >>>>>>>>> Chris,
>> >>>>>>>>>
>> >>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>> >>> others
>> >>>>>>> (e.g.
>> >>>>>>>>> if we don't care about certain new features, is it relatively
>> >>>> stable)?
>> >>>>>>>>> Would it be possible to cut out a version that only has the bits
>> >>> we
>> >>>>>>> think
>> >>>>>>>>> are stable (and release that)?
>> >>>>>>>>>
>> >>>>>>>>> From that timeline, and the historic release cadence, it would
>> >>> seem
>> >>>> to
>> >>>>>>> be
>> >>>>>>>>> a
>> >>>>>>>>> years away before we get to the stable release?
>> >>>>>>>>>
>> >>>>>>>>> Thanks,
>> >>>>>>>>>
>> >>>>>>>>> Jason
>> >>>>>>>>>
>> >>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>> >>>>>>> cnauroth@hortonworks.com>
>> >>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Hello Doug,
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks for your interest in the SSL feature!
>> >>>>>>>>>>
>> >>>>>>>>>> At this point, I think we're still pretty far away from
>> >>> declaring a
>> >>>>>>>>>> stable
>> >>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>> >>>>> anyone
>> >>>>>>>>>> can
>> >>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>> >>> the
>> >>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>> >>>>>>>>>>
>> >>>>>>>>>> https://s.apache.org/ADK1
>> >>>>>>>>>>
>> >>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>> >>>> resolving a
>> >>>>>>>>>> few
>> >>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>> >>>> that
>> >>>>>>>>>> will
>> >>>>>>>>>> get done in the next few weeks.
>> >>>>>>>>>>
>> >>>>>>>>>> --Chris Nauroth
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> I know it's only been a few months, but I was wondering if
>> there
>> >>>>> was a
>> >>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is
>> there
>> >>>> any
>> >>>>>>>>>>> chance
>> >>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>> >>>> looking
>> >>>>>>> to
>> >>>>>>>>>>> have
>> >>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> View this message in context:
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> >>>
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>> >>>>>>>>>> e
>> >>>>>>>>>>> -tp7581744p7582136.html
>> >>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
>> Nabble.com.
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >
>>

Re: Zookeeper with SSL release date

[Jason Rosenberg <jb...@squareup.com>]

Forgive me, as I have not long been an active member of the zookeeper
community (other than as a grateful user over the last 3 years or so).

If I understand correclty, 3.5.X has been alpha for several years or so
now?  I think if there isn't a plan to have a stable release soon (say
within another year), it's a problem.  It should be about having a regular
release cycle, with the hope that new features and bug fixes become
available in a reasonable time.  If one feature is just not stable, then it
shouldn't block other features, etc.  Saying a feature is a major part of
3.5, doesn't quite make sense in this formulation.  Instead releases
incorporate features, and if features get delayed, they can be postponed to
a subsequent release.

The issue is that we have people saying they don't want to fix things in
3.4.X (or back port fixes from 3.5.X to 3.4.X).  But if 3.5.X is still
literally still years away (after having been under development for years),
we should re-evaluate, no?

Jason

On Wed, Mar 16, 2016 at 8:46 PM, Patrick Hunt <ph...@apache.org> wrote:

> I'm not a huge fan of turning it off to be honest. Also just turning
> it off at the API level wouldn't be enough, we'd need to turn it off
> at the protocol level (otw it could still be accessed).
>
> I'd rather see us address it than kick it down the road. It's a major
> feature of 3.5.
>
> Patrick
>
> On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
> > The main issue to sort out is stability of the API. There is a security
> concern around the fact that clients can freely reconfigure the ensemble.
> If we follow the plan that Pat proposed some time ago:
> >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> <
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
> >
> >
> > Locking the API is the main step to move it to beta. Sorting out bugs is
> definitely necessary, but it isn't the main thing that is keeping 3.5 in
> alpha.
> >
> > About making it experimental, I was raising the option of having a
> switch that disables the API calls, not the code. The reason why that could
> work is that anyone using 3.5 who uses the "experimental" API must explicit
> turn on the switch and enable the calls. If they do it, they need to be
> aware that the API can change.
> >
> >  I must say that I haven't really looked closely into doing it, and I'm
> not even entirely convinced that this is a good idea, but since Jason
> raised the point, I'm exploring options.
> >
> > -Flavio
> >
> >> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
> >>
> >> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only
> 3-4
> >> are related to reconfig. Given this, and the fact that it is run in
> >> production since 2012 in multiple companies, I don't think its more
> >> unstable than any other part of ZooKeeper.
> >>
> >> There are multiple reconfig-related bugs that turned out really
> difficult
> >> to debug without access to the actual system or at least to the Hudson
> >> machines where some tests are failing. In the past, Michi and I
> physically
> >> went to Hortonworks to debug one such issue, but this is of course not a
> >> good method, and we weren't able to arrange such a visit again.
> >>
> >> Regarding making it optional - the reconfig logic has several different
> >> parts, some would be really difficult to disable using a configuration
> >> parameter. But the actual dynamic expansion of the cluster is triggered
> by
> >> the reconfig command, so it should not affect users who don't invoke it.
> >>
> >> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org>
> wrote:
> >>
> >>> I suppose we could give it a try. How do other folks feel about it?
> >>>
> >>> -Flavio
> >>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
> >>>
> >>>> So, you could enable the dynamic reconfiguration feature behind a
> config
> >>>> option, and document that it should only be enabled experimentally,
> use
> >>> at
> >>>> your own risk.  Keep it off by default.  Allow only static config by
> >>>> default, until it's stable?
> >>>>
> >>>> Jason
> >>>>
> >>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
> >>> wrote:
> >>>>
> >>>>> Hi Jason,
> >>>>>
> >>>>> The consumer in Kafka is pretty independent from the core (brokers),
> >>>>> that's how that project manages to make such a separation. We don't
> >>> have
> >>>>> the same with ZooKeeper as the feature we are talking about is part
> of
> >>>> the
> >>>>> server and the only way I see of doing what you say is to turn off
> >>>>> features. More specifically, we'd need to disable the reconfig API
> and
> >>> do
> >>>>> not allow any change to the configuration, even though the code is
> >>> there.
> >>>>>
> >>>>> Reconfig here refers to the ability of changing the configuration of
> an
> >>>>> ensemble (e.g., changing the set of servers).
> >>>>>
> >>>>> -Flavio
> >>>>>
> >>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
> >>>>>>
> >>>>>> So, it would seem sensible to me to have a release where all
> features
> >>>> are
> >>>>>> stable, except where noted.  E.g. mark certain features as only
> >>> 'alpha
> >>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
> >>>>> happily
> >>>>>> use 3.5.X without exercising the unstable re-config bits?).
> >>>>>>
> >>>>>> There's precedent for doing this sort of thing in other projects,
> >>> e.g.
> >>>> in
> >>>>>> Kafka, they've had several release where a new "Consumer API" is
> >>>> shipped
> >>>>>> that is available for beta-testing, but you can still just use the
> >>>> older
> >>>>>> stable consumer api, etc.
> >>>>>>
> >>>>>> Jason
> >>>>>>
> >>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> >>>>> <powellm79@yahoo.com.invalid
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Doug,
> >>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or
> have
> >>>> you
> >>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled
> as
> >>>>> alpha
> >>>>>>> might not be fair, since it is far more stable then what most
> people
> >>>>>>> associate an alpha release to be.
> >>>>>>> Perhaps if you do not use re-config feature may be it will just
> work
> >>>> for
> >>>>>>> you?.
> >>>>>>> There are many examples of 3.5.X being used in productions from my
> >>>>> limited
> >>>>>>> knowledge.
> >>>>>>> ThanksPowell.
> >>>>>>>
> >>>>>>>   On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> >>>>> fpj@apache.org>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> None of us expected the reconfig changes to take this long to
> >>>> stabilize.
> >>>>>>> Until we get there, I don't think we can do anything else with 3.5.
> >>>> The
> >>>>>>> best bet we have is to work harder to bring 3.5 into a stable
> rather
> >>>>> than
> >>>>>>> trying to work around it.
> >>>>>>>
> >>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
> >>>> get
> >>>>>>> everyone to contribute more patches and code reviews, we should be
> >>>> able
> >>>>> to
> >>>>>>> do it sooner. After all, it is a community based effort, so the
> >>>>> community
> >>>>>>> shouldn't rely on just 2-3 people doing the work.
> >>>>>>>
> >>>>>>> -Flavio
> >>>>>>>
> >>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cnauroth@hortonworks.com
> >
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4
> is
> >>>> the
> >>>>>>>> stable maintenance line, we are very conservative about
> >>> back-porting
> >>>> to
> >>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
> >>> not
> >>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
> >>>>> managing
> >>>>>>>> risk.
> >>>>>>>>
> >>>>>>>> Jason, your question about release cadence is a fair one.  If it's
> >>>> any
> >>>>>>>> consolation, we are now taking the approach of trying to limit the
> >>>>> scope
> >>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
> >>> focus
> >>>> on
> >>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
> >>>>> think
> >>>>>>>> this will help us accelerate the releases into beta and eventual
> >>> GA.
> >>>>>>>>
> >>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
> >>>> thoughts
> >>>>>>>> from more experienced committers and PMC members about your
> >>> proposal
> >>>> to
> >>>>>>>> try to cut a stable release for a limited subset of what is in
> >>>>> branch-3.5
> >>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
> >>> out
> >>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
> >>>>> another
> >>>>>>>> release line for an already resource-constrained volunteer staff
> to
> >>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
> >>>> 3.5
> >>>>>>>> stabilization.  Also, a 3.5 release in which certain features
> >>>>> "vanished"
> >>>>>>>> because of not meeting some stability criteria would be
> >>> undesirable.
> >>>>>>>>
> >>>>>>>> --Chris Nauroth
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> >>>>>>>>
> >>>>>>>>> Chris,
> >>>>>>>>>
> >>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
> >>> others
> >>>>>>> (e.g.
> >>>>>>>>> if we don't care about certain new features, is it relatively
> >>>> stable)?
> >>>>>>>>> Would it be possible to cut out a version that only has the bits
> >>> we
> >>>>>>> think
> >>>>>>>>> are stable (and release that)?
> >>>>>>>>>
> >>>>>>>>> From that timeline, and the historic release cadence, it would
> >>> seem
> >>>> to
> >>>>>>> be
> >>>>>>>>> a
> >>>>>>>>> years away before we get to the stable release?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>>
> >>>>>>>>> Jason
> >>>>>>>>>
> >>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> >>>>>>> cnauroth@hortonworks.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hello Doug,
> >>>>>>>>>>
> >>>>>>>>>> Thanks for your interest in the SSL feature!
> >>>>>>>>>>
> >>>>>>>>>> At this point, I think we're still pretty far away from
> >>> declaring a
> >>>>>>>>>> stable
> >>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
> >>>>> anyone
> >>>>>>>>>> can
> >>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
> >>> the
> >>>>>>>>>> high-level strategy for release planning in the 3.5 line:
> >>>>>>>>>>
> >>>>>>>>>> https://s.apache.org/ADK1
> >>>>>>>>>>
> >>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
> >>>> resolving a
> >>>>>>>>>> few
> >>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
> >>>> that
> >>>>>>>>>> will
> >>>>>>>>>> get done in the next few weeks.
> >>>>>>>>>>
> >>>>>>>>>> --Chris Nauroth
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I know it's only been a few months, but I was wondering if
> there
> >>>>> was a
> >>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is
> there
> >>>> any
> >>>>>>>>>>> chance
> >>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
> >>>> looking
> >>>>>>> to
> >>>>>>>>>>> have
> >>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> View this message in context:
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >>>>>>>>>> e
> >>>>>>>>>>> -tp7581744p7582136.html
> >>>>>>>>>>> Sent from the zookeeper-user mailing list archive at
> Nabble.com.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >
>

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

I'm not a huge fan of turning it off to be honest. Also just turning
it off at the API level wouldn't be enough, we'd need to turn it off
at the protocol level (otw it could still be accessed).

I'd rather see us address it than kick it down the road. It's a major
feature of 3.5.

Patrick

On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <fp...@apache.org> wrote:
> The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:
>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>
>
> Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.
>
> About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.
>
>  I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.
>
> -Flavio
>
>> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
>>
>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
>> are related to reconfig. Given this, and the fact that it is run in
>> production since 2012 in multiple companies, I don't think its more
>> unstable than any other part of ZooKeeper.
>>
>> There are multiple reconfig-related bugs that turned out really difficult
>> to debug without access to the actual system or at least to the Hudson
>> machines where some tests are failing. In the past, Michi and I physically
>> went to Hortonworks to debug one such issue, but this is of course not a
>> good method, and we weren't able to arrange such a visit again.
>>
>> Regarding making it optional - the reconfig logic has several different
>> parts, some would be really difficult to disable using a configuration
>> parameter. But the actual dynamic expansion of the cluster is triggered by
>> the reconfig command, so it should not affect users who don't invoke it.
>>
>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
>>
>>> I suppose we could give it a try. How do other folks feel about it?
>>>
>>> -Flavio
>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>
>>>> So, you could enable the dynamic reconfiguration feature behind a config
>>>> option, and document that it should only be enabled experimentally, use
>>> at
>>>> your own risk.  Keep it off by default.  Allow only static config by
>>>> default, until it's stable?
>>>>
>>>> Jason
>>>>
>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>>> wrote:
>>>>
>>>>> Hi Jason,
>>>>>
>>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>>> that's how that project manages to make such a separation. We don't
>>> have
>>>>> the same with ZooKeeper as the feature we are talking about is part of
>>>> the
>>>>> server and the only way I see of doing what you say is to turn off
>>>>> features. More specifically, we'd need to disable the reconfig API and
>>> do
>>>>> not allow any change to the configuration, even though the code is
>>> there.
>>>>>
>>>>> Reconfig here refers to the ability of changing the configuration of an
>>>>> ensemble (e.g., changing the set of servers).
>>>>>
>>>>> -Flavio
>>>>>
>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>>>
>>>>>> So, it would seem sensible to me to have a release where all features
>>>> are
>>>>>> stable, except where noted.  E.g. mark certain features as only
>>> 'alpha
>>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>>> happily
>>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>>>
>>>>>> There's precedent for doing this sort of thing in other projects,
>>> e.g.
>>>> in
>>>>>> Kafka, they've had several release where a new "Consumer API" is
>>>> shipped
>>>>>> that is available for beta-testing, but you can still just use the
>>>> older
>>>>>> stable consumer api, etc.
>>>>>>
>>>>>> Jason
>>>>>>
>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>>> <powellm79@yahoo.com.invalid
>>>>>>> wrote:
>>>>>>
>>>>>>> Hi Doug,
>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>>> you
>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>>> alpha
>>>>>>> might not be fair, since it is far more stable then what most people
>>>>>>> associate an alpha release to be.
>>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>>> for
>>>>>>> you?.
>>>>>>> There are many examples of 3.5.X being used in productions from my
>>>>> limited
>>>>>>> knowledge.
>>>>>>> ThanksPowell.
>>>>>>>
>>>>>>>   On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>>> fpj@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> None of us expected the reconfig changes to take this long to
>>>> stabilize.
>>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>>> The
>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>>> than
>>>>>>> trying to work around it.
>>>>>>>
>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>>> get
>>>>>>> everyone to contribute more patches and code reviews, we should be
>>>> able
>>>>> to
>>>>>>> do it sooner. After all, it is a community based effort, so the
>>>>> community
>>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>>>
>>>>>>> -Flavio
>>>>>>>
>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>>> the
>>>>>>>> stable maintenance line, we are very conservative about
>>> back-porting
>>>> to
>>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>>> not
>>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>>> managing
>>>>>>>> risk.
>>>>>>>>
>>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>>> any
>>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>>> scope
>>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>>> focus
>>>> on
>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>>> think
>>>>>>>> this will help us accelerate the releases into beta and eventual
>>> GA.
>>>>>>>>
>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>>> thoughts
>>>>>>>> from more experienced committers and PMC members about your
>>> proposal
>>>> to
>>>>>>>> try to cut a stable release for a limited subset of what is in
>>>>> branch-3.5
>>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>>> out
>>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>>> another
>>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>>> 3.5
>>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>>> "vanished"
>>>>>>>> because of not meeting some stability criteria would be
>>> undesirable.
>>>>>>>>
>>>>>>>> --Chris Nauroth
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>>>
>>>>>>>>> Chris,
>>>>>>>>>
>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>>> others
>>>>>>> (e.g.
>>>>>>>>> if we don't care about certain new features, is it relatively
>>>> stable)?
>>>>>>>>> Would it be possible to cut out a version that only has the bits
>>> we
>>>>>>> think
>>>>>>>>> are stable (and release that)?
>>>>>>>>>
>>>>>>>>> From that timeline, and the historic release cadence, it would
>>> seem
>>>> to
>>>>>>> be
>>>>>>>>> a
>>>>>>>>> years away before we get to the stable release?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Jason
>>>>>>>>>
>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>>> cnauroth@hortonworks.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Doug,
>>>>>>>>>>
>>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>>>
>>>>>>>>>> At this point, I think we're still pretty far away from
>>> declaring a
>>>>>>>>>> stable
>>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>>> anyone
>>>>>>>>>> can
>>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>>> the
>>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>>>
>>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>>>
>>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>>> resolving a
>>>>>>>>>> few
>>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>>> that
>>>>>>>>>> will
>>>>>>>>>> get done in the next few weeks.
>>>>>>>>>>
>>>>>>>>>> --Chris Nauroth
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>>> was a
>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>>> any
>>>>>>>>>>> chance
>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>>> looking
>>>>>>> to
>>>>>>>>>>> have
>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context:
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>>> e
>>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>
>

Re: Zookeeper with SSL release date

[Flavio Junqueira <fp...@apache.org>]

The main issue to sort out is stability of the API. There is a security concern around the fact that clients can freely reconfigure the ensemble. If we follow the plan that Pat proposed some time ago:

https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E>

Locking the API is the main step to move it to beta. Sorting out bugs is definitely necessary, but it isn't the main thing that is keeping 3.5 in alpha.

About making it experimental, I was raising the option of having a switch that disables the API calls, not the code. The reason why that could work is that anyone using 3.5 who uses the "experimental" API must explicit turn on the switch and enable the calls. If they do it, they need to be aware that the API can change.

 I must say that I haven't really looked closely into doing it, and I'm not even entirely convinced that this is a good idea, but since Jason raised the point, I'm exploring options.

-Flavio
 
> On 16 Mar 2016, at 20:59, Alexander Shraer <sh...@gmail.com> wrote:
> 
> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
> are related to reconfig. Given this, and the fact that it is run in
> production since 2012 in multiple companies, I don't think its more
> unstable than any other part of ZooKeeper.
> 
> There are multiple reconfig-related bugs that turned out really difficult
> to debug without access to the actual system or at least to the Hudson
> machines where some tests are failing. In the past, Michi and I physically
> went to Hortonworks to debug one such issue, but this is of course not a
> good method, and we weren't able to arrange such a visit again.
> 
> Regarding making it optional - the reconfig logic has several different
> parts, some would be really difficult to disable using a configuration
> parameter. But the actual dynamic expansion of the cluster is triggered by
> the reconfig command, so it should not affect users who don't invoke it.
> 
> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:
> 
>> I suppose we could give it a try. How do other folks feel about it?
>> 
>> -Flavio
>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>> 
>>> So, you could enable the dynamic reconfiguration feature behind a config
>>> option, and document that it should only be enabled experimentally, use
>> at
>>> your own risk.  Keep it off by default.  Allow only static config by
>>> default, until it's stable?
>>> 
>>> Jason
>>> 
>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
>> wrote:
>>> 
>>>> Hi Jason,
>>>> 
>>>> The consumer in Kafka is pretty independent from the core (brokers),
>>>> that's how that project manages to make such a separation. We don't
>> have
>>>> the same with ZooKeeper as the feature we are talking about is part of
>>> the
>>>> server and the only way I see of doing what you say is to turn off
>>>> features. More specifically, we'd need to disable the reconfig API and
>> do
>>>> not allow any change to the configuration, even though the code is
>> there.
>>>> 
>>>> Reconfig here refers to the ability of changing the configuration of an
>>>> ensemble (e.g., changing the set of servers).
>>>> 
>>>> -Flavio
>>>> 
>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
>>>>> 
>>>>> So, it would seem sensible to me to have a release where all features
>>> are
>>>>> stable, except where noted.  E.g. mark certain features as only
>> 'alpha
>>>>> quality', e.g. the 're-config feature'.  (I assume it's possible to
>>>> happily
>>>>> use 3.5.X without exercising the unstable re-config bits?).
>>>>> 
>>>>> There's precedent for doing this sort of thing in other projects,
>> e.g.
>>> in
>>>>> Kafka, they've had several release where a new "Consumer API" is
>>> shipped
>>>>> that is available for beta-testing, but you can still just use the
>>> older
>>>>> stable consumer api, etc.
>>>>> 
>>>>> Jason
>>>>> 
>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>>>> <powellm79@yahoo.com.invalid
>>>>>> wrote:
>>>>> 
>>>>>> Hi Doug,
>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have
>>> you
>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as
>>>> alpha
>>>>>> might not be fair, since it is far more stable then what most people
>>>>>> associate an alpha release to be.
>>>>>> Perhaps if you do not use re-config feature may be it will just work
>>> for
>>>>>> you?.
>>>>>> There are many examples of 3.5.X being used in productions from my
>>>> limited
>>>>>> knowledge.
>>>>>> ThanksPowell.
>>>>>> 
>>>>>>   On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
>>>> fpj@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>> 
>>>>>> None of us expected the reconfig changes to take this long to
>>> stabilize.
>>>>>> Until we get there, I don't think we can do anything else with 3.5.
>>> The
>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather
>>>> than
>>>>>> trying to work around it.
>>>>>> 
>>>>>> There are lots of people interested in seeing 3.5 stable, and if we
>>> get
>>>>>> everyone to contribute more patches and code reviews, we should be
>>> able
>>>> to
>>>>>> do it sooner. After all, it is a community based effort, so the
>>>> community
>>>>>> shouldn't rely on just 2-3 people doing the work.
>>>>>> 
>>>>>> -Flavio
>>>>>> 
>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
>>> the
>>>>>>> stable maintenance line, we are very conservative about
>> back-porting
>>> to
>>>>>>> it.  Our policy is to limit back-ports to critical bug fixes and
>> not
>>>>>>> introduce any new features in the 3.4 line.  This is a matter of
>>>> managing
>>>>>>> risk.
>>>>>>> 
>>>>>>> Jason, your question about release cadence is a fair one.  If it's
>>> any
>>>>>>> consolation, we are now taking the approach of trying to limit the
>>>> scope
>>>>>>> of anything new introduced in 3.5 too.  That would allow us to
>> focus
>>> on
>>>>>>> stabilization: resolving blocker bugs and freezing public APIs.  I
>>>> think
>>>>>>> this will help us accelerate the releases into beta and eventual
>> GA.
>>>>>>> 
>>>>>>> I am new to ZooKeeper release management, so I'd like to hear
>>> thoughts
>>>>>>> from more experienced committers and PMC members about your
>> proposal
>>> to
>>>>>>> try to cut a stable release for a limited subset of what is in
>>>> branch-3.5
>>>>>>> now.  My instinct is that it would be challenging to cherry-pick
>> out
>>>>>>> pieces of branch-3.5 piecemeal at this point.  This would become
>>>> another
>>>>>>> release line for an already resource-constrained volunteer staff to
>>>>>>> manage.  I'd prefer to dedicate those limited resources to overall
>>> 3.5
>>>>>>> stabilization.  Also, a 3.5 release in which certain features
>>>> "vanished"
>>>>>>> because of not meeting some stability criteria would be
>> undesirable.
>>>>>>> 
>>>>>>> --Chris Nauroth
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>>>>>> 
>>>>>>>> Chris,
>>>>>>>> 
>>>>>>>> Can you say whether some parts of 3.5.X are more stable than
>> others
>>>>>> (e.g.
>>>>>>>> if we don't care about certain new features, is it relatively
>>> stable)?
>>>>>>>> Would it be possible to cut out a version that only has the bits
>> we
>>>>>> think
>>>>>>>> are stable (and release that)?
>>>>>>>> 
>>>>>>>> From that timeline, and the historic release cadence, it would
>> seem
>>> to
>>>>>> be
>>>>>>>> a
>>>>>>>> years away before we get to the stable release?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> 
>>>>>>>> Jason
>>>>>>>> 
>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>>>>>> cnauroth@hortonworks.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hello Doug,
>>>>>>>>> 
>>>>>>>>> Thanks for your interest in the SSL feature!
>>>>>>>>> 
>>>>>>>>> At this point, I think we're still pretty far away from
>> declaring a
>>>>>>>>> stable
>>>>>>>>> release in the 3.5 line.  I don't think we're close enough that
>>>> anyone
>>>>>>>>> can
>>>>>>>>> offer a reliable ETA.  This is an earlier thread that describes
>> the
>>>>>>>>> high-level strategy for release planning in the 3.5 line:
>>>>>>>>> 
>>>>>>>>> https://s.apache.org/ADK1
>>>>>>>>> 
>>>>>>>>> The next step is a 3.5.2-alpha release.  We're working on
>>> resolving a
>>>>>>>>> few
>>>>>>>>> more blockers before we produce a release candidate.  Hopefully
>>> that
>>>>>>>>> will
>>>>>>>>> get done in the next few weeks.
>>>>>>>>> 
>>>>>>>>> --Chris Nauroth
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>>> I know it's only been a few months, but I was wondering if there
>>>> was a
>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there
>>> any
>>>>>>>>>> chance
>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person
>>> looking
>>>>>> to
>>>>>>>>>> have
>>>>>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>> 
>>>> 
>>> 
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>>>>>> e
>>>>>>>>>> -tp7581744p7582136.html
>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
>> 

Re: Zookeeper with SSL release date

[Alexander Shraer <sh...@gmail.com>]

Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4
are related to reconfig. Given this, and the fact that it is run in
production since 2012 in multiple companies, I don't think its more
unstable than any other part of ZooKeeper.

There are multiple reconfig-related bugs that turned out really difficult
to debug without access to the actual system or at least to the Hudson
machines where some tests are failing. In the past, Michi and I physically
went to Hortonworks to debug one such issue, but this is of course not a
good method, and we weren't able to arrange such a visit again.

Regarding making it optional - the reconfig logic has several different
parts, some would be really difficult to disable using a configuration
parameter. But the actual dynamic expansion of the cluster is triggered by
the reconfig command, so it should not affect users who don't invoke it.

On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fp...@apache.org> wrote:

> I suppose we could give it a try. How do other folks feel about it?
>
> -Flavio
> On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:
>
> > So, you could enable the dynamic reconfiguration feature behind a config
> > option, and document that it should only be enabled experimentally, use
> at
> > your own risk.  Keep it off by default.  Allow only static config by
> > default, until it's stable?
> >
> > Jason
> >
> > On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org>
> wrote:
> >
> > > Hi Jason,
> > >
> > > The consumer in Kafka is pretty independent from the core (brokers),
> > > that's how that project manages to make such a separation. We don't
> have
> > > the same with ZooKeeper as the feature we are talking about is part of
> > the
> > > server and the only way I see of doing what you say is to turn off
> > > features. More specifically, we'd need to disable the reconfig API and
> do
> > > not allow any change to the configuration, even though the code is
> there.
> > >
> > > Reconfig here refers to the ability of changing the configuration of an
> > > ensemble (e.g., changing the set of servers).
> > >
> > > -Flavio
> > >
> > > > On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
> > > >
> > > > So, it would seem sensible to me to have a release where all features
> > are
> > > > stable, except where noted.  E.g. mark certain features as only
> 'alpha
> > > > quality', e.g. the 're-config feature'.  (I assume it's possible to
> > > happily
> > > > use 3.5.X without exercising the unstable re-config bits?).
> > > >
> > > > There's precedent for doing this sort of thing in other projects,
> e.g.
> > in
> > > > Kafka, they've had several release where a new "Consumer API" is
> > shipped
> > > > that is available for beta-testing, but you can still just use the
> > older
> > > > stable consumer api, etc.
> > > >
> > > > Jason
> > > >
> > > > On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > > <powellm79@yahoo.com.invalid
> > > >> wrote:
> > > >
> > > >> Hi Doug,
> > > >> Is 3.5 being an alpha release preventing you from using it?. Or have
> > you
> > > >> run into issues with it?. In general perhaps ZK 3.5 being labeled as
> > > alpha
> > > >> might not be fair, since it is far more stable then what most people
> > > >> associate an alpha release to be.
> > > >> Perhaps if you do not use re-config feature may be it will just work
> > for
> > > >> you?.
> > > >> There are many examples of 3.5.X being used in productions from my
> > > limited
> > > >> knowledge.
> > > >> ThanksPowell.
> > > >>
> > > >>    On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> > > fpj@apache.org>
> > > >> wrote:
> > > >>
> > > >>
> > > >> None of us expected the reconfig changes to take this long to
> > stabilize.
> > > >> Until we get there, I don't think we can do anything else with 3.5.
> > The
> > > >> best bet we have is to work harder to bring 3.5 into a stable rather
> > > than
> > > >> trying to work around it.
> > > >>
> > > >> There are lots of people interested in seeing 3.5 stable, and if we
> > get
> > > >> everyone to contribute more patches and code reviews, we should be
> > able
> > > to
> > > >> do it sooner. After all, it is a community based effort, so the
> > > community
> > > >> shouldn't rely on just 2-3 people doing the work.
> > > >>
> > > >> -Flavio
> > > >>
> > > >>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
> > > >> wrote:
> > > >>>
> > > >>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
> > the
> > > >>> stable maintenance line, we are very conservative about
> back-porting
> > to
> > > >>> it.  Our policy is to limit back-ports to critical bug fixes and
> not
> > > >>> introduce any new features in the 3.4 line.  This is a matter of
> > > managing
> > > >>> risk.
> > > >>>
> > > >>> Jason, your question about release cadence is a fair one.  If it's
> > any
> > > >>> consolation, we are now taking the approach of trying to limit the
> > > scope
> > > >>> of anything new introduced in 3.5 too.  That would allow us to
> focus
> > on
> > > >>> stabilization: resolving blocker bugs and freezing public APIs.  I
> > > think
> > > >>> this will help us accelerate the releases into beta and eventual
> GA.
> > > >>>
> > > >>> I am new to ZooKeeper release management, so I'd like to hear
> > thoughts
> > > >>> from more experienced committers and PMC members about your
> proposal
> > to
> > > >>> try to cut a stable release for a limited subset of what is in
> > > branch-3.5
> > > >>> now.  My instinct is that it would be challenging to cherry-pick
> out
> > > >>> pieces of branch-3.5 piecemeal at this point.  This would become
> > > another
> > > >>> release line for an already resource-constrained volunteer staff to
> > > >>> manage.  I'd prefer to dedicate those limited resources to overall
> > 3.5
> > > >>> stabilization.  Also, a 3.5 release in which certain features
> > > "vanished"
> > > >>> because of not meeting some stability criteria would be
> undesirable.
> > > >>>
> > > >>> --Chris Nauroth
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> > > >>>
> > > >>>> Chris,
> > > >>>>
> > > >>>> Can you say whether some parts of 3.5.X are more stable than
> others
> > > >> (e.g.
> > > >>>> if we don't care about certain new features, is it relatively
> > stable)?
> > > >>>> Would it be possible to cut out a version that only has the bits
> we
> > > >> think
> > > >>>> are stable (and release that)?
> > > >>>>
> > > >>>> From that timeline, and the historic release cadence, it would
> seem
> > to
> > > >> be
> > > >>>> a
> > > >>>> years away before we get to the stable release?
> > > >>>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Jason
> > > >>>>
> > > >>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> > > >> cnauroth@hortonworks.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Hello Doug,
> > > >>>>>
> > > >>>>> Thanks for your interest in the SSL feature!
> > > >>>>>
> > > >>>>> At this point, I think we're still pretty far away from
> declaring a
> > > >>>>> stable
> > > >>>>> release in the 3.5 line.  I don't think we're close enough that
> > > anyone
> > > >>>>> can
> > > >>>>> offer a reliable ETA.  This is an earlier thread that describes
> the
> > > >>>>> high-level strategy for release planning in the 3.5 line:
> > > >>>>>
> > > >>>>> https://s.apache.org/ADK1
> > > >>>>>
> > > >>>>> The next step is a 3.5.2-alpha release.  We're working on
> > resolving a
> > > >>>>> few
> > > >>>>> more blockers before we produce a release candidate.  Hopefully
> > that
> > > >>>>> will
> > > >>>>> get done in the next few weeks.
> > > >>>>>
> > > >>>>> --Chris Nauroth
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> > > >>>>>
> > > >>>>>> I know it's only been a few months, but I was wondering if there
> > > was a
> > > >>>>>> ballpark release date for a stable version of 3.5.1. Or is there
> > any
> > > >>>>>> chance
> > > >>>>>> the SSL feature would be added to 3.4.8? Just another person
> > looking
> > > >> to
> > > >>>>>> have
> > > >>>>>> that feature in a stable version. Thanks for all you do! :)
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> View this message in context:
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > > >>>>> e
> > > >>>>>> -tp7581744p7582136.html
> > > >>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> >
>

Re: Zookeeper with SSL release date

[Chris Barlock <ba...@us.ibm.com>]

This is a pretty standard way, I'd say, of hiding new function until it is 
stable.  I think it is a good idea.

Chris




From:   Flavio P JUNQUEIRA <fp...@apache.org>
To:     user@zookeeper.apache.org
Date:   03/16/2016 04:10 PM
Subject:        Re: Zookeeper with SSL release date



I suppose we could give it a try. How do other folks feel about it?

-Flavio
On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:

> So, you could enable the dynamic reconfiguration feature behind a config
> option, and document that it should only be enabled experimentally, use 
at
> your own risk.  Keep it off by default.  Allow only static config by
> default, until it's stable?
>
> Jason
>
> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org> 
wrote:
>
> > Hi Jason,
> >
> > The consumer in Kafka is pretty independent from the core (brokers),
> > that's how that project manages to make such a separation. We don't 
have
> > the same with ZooKeeper as the feature we are talking about is part of
> the
> > server and the only way I see of doing what you say is to turn off
> > features. More specifically, we'd need to disable the reconfig API and 
do
> > not allow any change to the configuration, even though the code is 
there.
> >
> > Reconfig here refers to the ability of changing the configuration of 
an
> > ensemble (e.g., changing the set of servers).
> >
> > -Flavio
> >
> > > On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
> > >
> > > So, it would seem sensible to me to have a release where all 
features
> are
> > > stable, except where noted.  E.g. mark certain features as only 
'alpha
> > > quality', e.g. the 're-config feature'.  (I assume it's possible to
> > happily
> > > use 3.5.X without exercising the unstable re-config bits?).
> > >
> > > There's precedent for doing this sort of thing in other projects, 
e.g.
> in
> > > Kafka, they've had several release where a new "Consumer API" is
> shipped
> > > that is available for beta-testing, but you can still just use the
> older
> > > stable consumer api, etc.
> > >
> > > Jason
> > >
> > > On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > <powellm79@yahoo.com.invalid
> > >> wrote:
> > >
> > >> Hi Doug,
> > >> Is 3.5 being an alpha release preventing you from using it?. Or 
have
> you
> > >> run into issues with it?. In general perhaps ZK 3.5 being labeled 
as
> > alpha
> > >> might not be fair, since it is far more stable then what most 
people
> > >> associate an alpha release to be.
> > >> Perhaps if you do not use re-config feature may be it will just 
work
> for
> > >> you?.
> > >> There are many examples of 3.5.X being used in productions from my
> > limited
> > >> knowledge.
> > >> ThanksPowell.
> > >>
> > >>    On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> > fpj@apache.org>
> > >> wrote:
> > >>
> > >>
> > >> None of us expected the reconfig changes to take this long to
> stabilize.
> > >> Until we get there, I don't think we can do anything else with 3.5.
> The
> > >> best bet we have is to work harder to bring 3.5 into a stable 
rather
> > than
> > >> trying to work around it.
> > >>
> > >> There are lots of people interested in seeing 3.5 stable, and if we
> get
> > >> everyone to contribute more patches and code reviews, we should be
> able
> > to
> > >> do it sooner. After all, it is a community based effort, so the
> > community
> > >> shouldn't rely on just 2-3 people doing the work.
> > >>
> > >> -Flavio
> > >>
> > >>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
> > >> wrote:
> > >>>
> > >>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 
is
> the
> > >>> stable maintenance line, we are very conservative about 
back-porting
> to
> > >>> it.  Our policy is to limit back-ports to critical bug fixes and 
not
> > >>> introduce any new features in the 3.4 line.  This is a matter of
> > managing
> > >>> risk.
> > >>>
> > >>> Jason, your question about release cadence is a fair one.  If it's
> any
> > >>> consolation, we are now taking the approach of trying to limit the
> > scope
> > >>> of anything new introduced in 3.5 too.  That would allow us to 
focus
> on
> > >>> stabilization: resolving blocker bugs and freezing public APIs.  I
> > think
> > >>> this will help us accelerate the releases into beta and eventual 
GA.
> > >>>
> > >>> I am new to ZooKeeper release management, so I'd like to hear
> thoughts
> > >>> from more experienced committers and PMC members about your 
proposal
> to
> > >>> try to cut a stable release for a limited subset of what is in
> > branch-3.5
> > >>> now.  My instinct is that it would be challenging to cherry-pick 
out
> > >>> pieces of branch-3.5 piecemeal at this point.  This would become
> > another
> > >>> release line for an already resource-constrained volunteer staff 
to
> > >>> manage.  I'd prefer to dedicate those limited resources to overall
> 3.5
> > >>> stabilization.  Also, a 3.5 release in which certain features
> > "vanished"
> > >>> because of not meeting some stability criteria would be 
undesirable.
> > >>>
> > >>> --Chris Nauroth
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> > >>>
> > >>>> Chris,
> > >>>>
> > >>>> Can you say whether some parts of 3.5.X are more stable than 
others
> > >> (e.g.
> > >>>> if we don't care about certain new features, is it relatively
> stable)?
> > >>>> Would it be possible to cut out a version that only has the bits 
we
> > >> think
> > >>>> are stable (and release that)?
> > >>>>
> > >>>> From that timeline, and the historic release cadence, it would 
seem
> to
> > >> be
> > >>>> a
> > >>>> years away before we get to the stable release?
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>> Jason
> > >>>>
> > >>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> > >> cnauroth@hortonworks.com>
> > >>>> wrote:
> > >>>>
> > >>>>> Hello Doug,
> > >>>>>
> > >>>>> Thanks for your interest in the SSL feature!
> > >>>>>
> > >>>>> At this point, I think we're still pretty far away from 
declaring a
> > >>>>> stable
> > >>>>> release in the 3.5 line.  I don't think we're close enough that
> > anyone
> > >>>>> can
> > >>>>> offer a reliable ETA.  This is an earlier thread that describes 
the
> > >>>>> high-level strategy for release planning in the 3.5 line:
> > >>>>>
> > >>>>> https://s.apache.org/ADK1
> > >>>>>
> > >>>>> The next step is a 3.5.2-alpha release.  We're working on
> resolving a
> > >>>>> few
> > >>>>> more blockers before we produce a release candidate.  Hopefully
> that
> > >>>>> will
> > >>>>> get done in the next few weeks.
> > >>>>>
> > >>>>> --Chris Nauroth
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> > >>>>>
> > >>>>>> I know it's only been a few months, but I was wondering if 
there
> > was a
> > >>>>>> ballpark release date for a stable version of 3.5.1. Or is 
there
> any
> > >>>>>> chance
> > >>>>>> the SSL feature would be added to 3.4.8? Just another person
> looking
> > >> to
> > >>>>>> have
> > >>>>>> that feature in a stable version. Thanks for all you do! :)
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> View this message in context:
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>
> >
> 
http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > >>>>> e
> > >>>>>> -tp7581744p7582136.html
> > >>>>>> Sent from the zookeeper-user mailing list archive at 
Nabble.com.
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>
> > >>
> > >>
> > >>
> > >>
> >
> >
>

Re: Zookeeper with SSL release date

[Flavio P JUNQUEIRA <fp...@apache.org>]

I suppose we could give it a try. How do other folks feel about it?

-Flavio
On 16 Mar 2016 19:52, "Jason Rosenberg" <jb...@squareup.com> wrote:

> So, you could enable the dynamic reconfiguration feature behind a config
> option, and document that it should only be enabled experimentally, use at
> your own risk.  Keep it off by default.  Allow only static config by
> default, until it's stable?
>
> Jason
>
> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org> wrote:
>
> > Hi Jason,
> >
> > The consumer in Kafka is pretty independent from the core (brokers),
> > that's how that project manages to make such a separation. We don't have
> > the same with ZooKeeper as the feature we are talking about is part of
> the
> > server and the only way I see of doing what you say is to turn off
> > features. More specifically, we'd need to disable the reconfig API and do
> > not allow any change to the configuration, even though the code is there.
> >
> > Reconfig here refers to the ability of changing the configuration of an
> > ensemble (e.g., changing the set of servers).
> >
> > -Flavio
> >
> > > On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
> > >
> > > So, it would seem sensible to me to have a release where all features
> are
> > > stable, except where noted.  E.g. mark certain features as only 'alpha
> > > quality', e.g. the 're-config feature'.  (I assume it's possible to
> > happily
> > > use 3.5.X without exercising the unstable re-config bits?).
> > >
> > > There's precedent for doing this sort of thing in other projects, e.g.
> in
> > > Kafka, they've had several release where a new "Consumer API" is
> shipped
> > > that is available for beta-testing, but you can still just use the
> older
> > > stable consumer api, etc.
> > >
> > > Jason
> > >
> > > On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> > <powellm79@yahoo.com.invalid
> > >> wrote:
> > >
> > >> Hi Doug,
> > >> Is 3.5 being an alpha release preventing you from using it?. Or have
> you
> > >> run into issues with it?. In general perhaps ZK 3.5 being labeled as
> > alpha
> > >> might not be fair, since it is far more stable then what most people
> > >> associate an alpha release to be.
> > >> Perhaps if you do not use re-config feature may be it will just work
> for
> > >> you?.
> > >> There are many examples of 3.5.X being used in productions from my
> > limited
> > >> knowledge.
> > >> ThanksPowell.
> > >>
> > >>    On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> > fpj@apache.org>
> > >> wrote:
> > >>
> > >>
> > >> None of us expected the reconfig changes to take this long to
> stabilize.
> > >> Until we get there, I don't think we can do anything else with 3.5.
> The
> > >> best bet we have is to work harder to bring 3.5 into a stable rather
> > than
> > >> trying to work around it.
> > >>
> > >> There are lots of people interested in seeing 3.5 stable, and if we
> get
> > >> everyone to contribute more patches and code reviews, we should be
> able
> > to
> > >> do it sooner. After all, it is a community based effort, so the
> > community
> > >> shouldn't rely on just 2-3 people doing the work.
> > >>
> > >> -Flavio
> > >>
> > >>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
> > >> wrote:
> > >>>
> > >>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is
> the
> > >>> stable maintenance line, we are very conservative about back-porting
> to
> > >>> it.  Our policy is to limit back-ports to critical bug fixes and not
> > >>> introduce any new features in the 3.4 line.  This is a matter of
> > managing
> > >>> risk.
> > >>>
> > >>> Jason, your question about release cadence is a fair one.  If it's
> any
> > >>> consolation, we are now taking the approach of trying to limit the
> > scope
> > >>> of anything new introduced in 3.5 too.  That would allow us to focus
> on
> > >>> stabilization: resolving blocker bugs and freezing public APIs.  I
> > think
> > >>> this will help us accelerate the releases into beta and eventual GA.
> > >>>
> > >>> I am new to ZooKeeper release management, so I'd like to hear
> thoughts
> > >>> from more experienced committers and PMC members about your proposal
> to
> > >>> try to cut a stable release for a limited subset of what is in
> > branch-3.5
> > >>> now.  My instinct is that it would be challenging to cherry-pick out
> > >>> pieces of branch-3.5 piecemeal at this point.  This would become
> > another
> > >>> release line for an already resource-constrained volunteer staff to
> > >>> manage.  I'd prefer to dedicate those limited resources to overall
> 3.5
> > >>> stabilization.  Also, a 3.5 release in which certain features
> > "vanished"
> > >>> because of not meeting some stability criteria would be undesirable.
> > >>>
> > >>> --Chris Nauroth
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> > >>>
> > >>>> Chris,
> > >>>>
> > >>>> Can you say whether some parts of 3.5.X are more stable than others
> > >> (e.g.
> > >>>> if we don't care about certain new features, is it relatively
> stable)?
> > >>>> Would it be possible to cut out a version that only has the bits we
> > >> think
> > >>>> are stable (and release that)?
> > >>>>
> > >>>> From that timeline, and the historic release cadence, it would seem
> to
> > >> be
> > >>>> a
> > >>>> years away before we get to the stable release?
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>> Jason
> > >>>>
> > >>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> > >> cnauroth@hortonworks.com>
> > >>>> wrote:
> > >>>>
> > >>>>> Hello Doug,
> > >>>>>
> > >>>>> Thanks for your interest in the SSL feature!
> > >>>>>
> > >>>>> At this point, I think we're still pretty far away from declaring a
> > >>>>> stable
> > >>>>> release in the 3.5 line.  I don't think we're close enough that
> > anyone
> > >>>>> can
> > >>>>> offer a reliable ETA.  This is an earlier thread that describes the
> > >>>>> high-level strategy for release planning in the 3.5 line:
> > >>>>>
> > >>>>> https://s.apache.org/ADK1
> > >>>>>
> > >>>>> The next step is a 3.5.2-alpha release.  We're working on
> resolving a
> > >>>>> few
> > >>>>> more blockers before we produce a release candidate.  Hopefully
> that
> > >>>>> will
> > >>>>> get done in the next few weeks.
> > >>>>>
> > >>>>> --Chris Nauroth
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> > >>>>>
> > >>>>>> I know it's only been a few months, but I was wondering if there
> > was a
> > >>>>>> ballpark release date for a stable version of 3.5.1. Or is there
> any
> > >>>>>> chance
> > >>>>>> the SSL feature would be added to 3.4.8? Just another person
> looking
> > >> to
> > >>>>>> have
> > >>>>>> that feature in a stable version. Thanks for all you do! :)
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> View this message in context:
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> > >>>>> e
> > >>>>>> -tp7581744p7582136.html
> > >>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>
> > >>
> > >>
> > >>
> > >>
> >
> >
>

Re: Zookeeper with SSL release date

[Jason Rosenberg <jb...@squareup.com>]

So, you could enable the dynamic reconfiguration feature behind a config
option, and document that it should only be enabled experimentally, use at
your own risk.  Keep it off by default.  Allow only static config by
default, until it's stable?

Jason

On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fp...@apache.org> wrote:

> Hi Jason,
>
> The consumer in Kafka is pretty independent from the core (brokers),
> that's how that project manages to make such a separation. We don't have
> the same with ZooKeeper as the feature we are talking about is part of the
> server and the only way I see of doing what you say is to turn off
> features. More specifically, we'd need to disable the reconfig API and do
> not allow any change to the configuration, even though the code is there.
>
> Reconfig here refers to the ability of changing the configuration of an
> ensemble (e.g., changing the set of servers).
>
> -Flavio
>
> > On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
> >
> > So, it would seem sensible to me to have a release where all features are
> > stable, except where noted.  E.g. mark certain features as only 'alpha
> > quality', e.g. the 're-config feature'.  (I assume it's possible to
> happily
> > use 3.5.X without exercising the unstable re-config bits?).
> >
> > There's precedent for doing this sort of thing in other projects, e.g. in
> > Kafka, they've had several release where a new "Consumer API" is shipped
> > that is available for beta-testing, but you can still just use the older
> > stable consumer api, etc.
> >
> > Jason
> >
> > On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
> <powellm79@yahoo.com.invalid
> >> wrote:
> >
> >> Hi Doug,
> >> Is 3.5 being an alpha release preventing you from using it?. Or have you
> >> run into issues with it?. In general perhaps ZK 3.5 being labeled as
> alpha
> >> might not be fair, since it is far more stable then what most people
> >> associate an alpha release to be.
> >> Perhaps if you do not use re-config feature may be it will just work for
> >> you?.
> >> There are many examples of 3.5.X being used in productions from my
> limited
> >> knowledge.
> >> ThanksPowell.
> >>
> >>    On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <
> fpj@apache.org>
> >> wrote:
> >>
> >>
> >> None of us expected the reconfig changes to take this long to stabilize.
> >> Until we get there, I don't think we can do anything else with 3.5. The
> >> best bet we have is to work harder to bring 3.5 into a stable rather
> than
> >> trying to work around it.
> >>
> >> There are lots of people interested in seeing 3.5 stable, and if we get
> >> everyone to contribute more patches and code reviews, we should be able
> to
> >> do it sooner. After all, it is a community based effort, so the
> community
> >> shouldn't rely on just 2-3 people doing the work.
> >>
> >> -Flavio
> >>
> >>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
> >> wrote:
> >>>
> >>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is the
> >>> stable maintenance line, we are very conservative about back-porting to
> >>> it.  Our policy is to limit back-ports to critical bug fixes and not
> >>> introduce any new features in the 3.4 line.  This is a matter of
> managing
> >>> risk.
> >>>
> >>> Jason, your question about release cadence is a fair one.  If it's any
> >>> consolation, we are now taking the approach of trying to limit the
> scope
> >>> of anything new introduced in 3.5 too.  That would allow us to focus on
> >>> stabilization: resolving blocker bugs and freezing public APIs.  I
> think
> >>> this will help us accelerate the releases into beta and eventual GA.
> >>>
> >>> I am new to ZooKeeper release management, so I'd like to hear thoughts
> >>> from more experienced committers and PMC members about your proposal to
> >>> try to cut a stable release for a limited subset of what is in
> branch-3.5
> >>> now.  My instinct is that it would be challenging to cherry-pick out
> >>> pieces of branch-3.5 piecemeal at this point.  This would become
> another
> >>> release line for an already resource-constrained volunteer staff to
> >>> manage.  I'd prefer to dedicate those limited resources to overall 3.5
> >>> stabilization.  Also, a 3.5 release in which certain features
> "vanished"
> >>> because of not meeting some stability criteria would be undesirable.
> >>>
> >>> --Chris Nauroth
> >>>
> >>>
> >>>
> >>>
> >>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> >>>
> >>>> Chris,
> >>>>
> >>>> Can you say whether some parts of 3.5.X are more stable than others
> >> (e.g.
> >>>> if we don't care about certain new features, is it relatively stable)?
> >>>> Would it be possible to cut out a version that only has the bits we
> >> think
> >>>> are stable (and release that)?
> >>>>
> >>>> From that timeline, and the historic release cadence, it would seem to
> >> be
> >>>> a
> >>>> years away before we get to the stable release?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Jason
> >>>>
> >>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> >> cnauroth@hortonworks.com>
> >>>> wrote:
> >>>>
> >>>>> Hello Doug,
> >>>>>
> >>>>> Thanks for your interest in the SSL feature!
> >>>>>
> >>>>> At this point, I think we're still pretty far away from declaring a
> >>>>> stable
> >>>>> release in the 3.5 line.  I don't think we're close enough that
> anyone
> >>>>> can
> >>>>> offer a reliable ETA.  This is an earlier thread that describes the
> >>>>> high-level strategy for release planning in the 3.5 line:
> >>>>>
> >>>>> https://s.apache.org/ADK1
> >>>>>
> >>>>> The next step is a 3.5.2-alpha release.  We're working on resolving a
> >>>>> few
> >>>>> more blockers before we produce a release candidate.  Hopefully that
> >>>>> will
> >>>>> get done in the next few weeks.
> >>>>>
> >>>>> --Chris Nauroth
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> >>>>>
> >>>>>> I know it's only been a few months, but I was wondering if there
> was a
> >>>>>> ballpark release date for a stable version of 3.5.1. Or is there any
> >>>>>> chance
> >>>>>> the SSL feature would be added to 3.4.8? Just another person looking
> >> to
> >>>>>> have
> >>>>>> that feature in a stable version. Thanks for all you do! :)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> View this message in context:
> >>>>>>
> >>>>>
> >>>>>
> >>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >>>>> e
> >>>>>> -tp7581744p7582136.html
> >>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >>
> >>
> >>
> >>
>
>

Re: Zookeeper with SSL release date

[Flavio Junqueira <fp...@apache.org>]

Hi Jason,

The consumer in Kafka is pretty independent from the core (brokers), that's how that project manages to make such a separation. We don't have the same with ZooKeeper as the feature we are talking about is part of the server and the only way I see of doing what you say is to turn off features. More specifically, we'd need to disable the reconfig API and do not allow any change to the configuration, even though the code is there. 

Reconfig here refers to the ability of changing the configuration of an ensemble (e.g., changing the set of servers).

-Flavio

> On 16 Mar 2016, at 19:14, Jason Rosenberg <jb...@squareup.com> wrote:
> 
> So, it would seem sensible to me to have a release where all features are
> stable, except where noted.  E.g. mark certain features as only 'alpha
> quality', e.g. the 're-config feature'.  (I assume it's possible to happily
> use 3.5.X without exercising the unstable re-config bits?).
> 
> There's precedent for doing this sort of thing in other projects, e.g. in
> Kafka, they've had several release where a new "Consumer API" is shipped
> that is available for beta-testing, but you can still just use the older
> stable consumer api, etc.
> 
> Jason
> 
> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti <powellm79@yahoo.com.invalid
>> wrote:
> 
>> Hi Doug,
>> Is 3.5 being an alpha release preventing you from using it?. Or have you
>> run into issues with it?. In general perhaps ZK 3.5 being labeled as alpha
>> might not be fair, since it is far more stable then what most people
>> associate an alpha release to be.
>> Perhaps if you do not use re-config feature may be it will just work for
>> you?.
>> There are many examples of 3.5.X being used in productions from my limited
>> knowledge.
>> ThanksPowell.
>> 
>>    On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <fp...@apache.org>
>> wrote:
>> 
>> 
>> None of us expected the reconfig changes to take this long to stabilize.
>> Until we get there, I don't think we can do anything else with 3.5. The
>> best bet we have is to work harder to bring 3.5 into a stable rather than
>> trying to work around it.
>> 
>> There are lots of people interested in seeing 3.5 stable, and if we get
>> everyone to contribute more patches and code reviews, we should be able to
>> do it sooner. After all, it is a community based effort, so the community
>> shouldn't rely on just 2-3 people doing the work.
>> 
>> -Flavio
>> 
>>> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>>> 
>>> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is the
>>> stable maintenance line, we are very conservative about back-porting to
>>> it.  Our policy is to limit back-ports to critical bug fixes and not
>>> introduce any new features in the 3.4 line.  This is a matter of managing
>>> risk.
>>> 
>>> Jason, your question about release cadence is a fair one.  If it's any
>>> consolation, we are now taking the approach of trying to limit the scope
>>> of anything new introduced in 3.5 too.  That would allow us to focus on
>>> stabilization: resolving blocker bugs and freezing public APIs.  I think
>>> this will help us accelerate the releases into beta and eventual GA.
>>> 
>>> I am new to ZooKeeper release management, so I'd like to hear thoughts
>>> from more experienced committers and PMC members about your proposal to
>>> try to cut a stable release for a limited subset of what is in branch-3.5
>>> now.  My instinct is that it would be challenging to cherry-pick out
>>> pieces of branch-3.5 piecemeal at this point.  This would become another
>>> release line for an already resource-constrained volunteer staff to
>>> manage.  I'd prefer to dedicate those limited resources to overall 3.5
>>> stabilization.  Also, a 3.5 release in which certain features "vanished"
>>> because of not meeting some stability criteria would be undesirable.
>>> 
>>> --Chris Nauroth
>>> 
>>> 
>>> 
>>> 
>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
>>> 
>>>> Chris,
>>>> 
>>>> Can you say whether some parts of 3.5.X are more stable than others
>> (e.g.
>>>> if we don't care about certain new features, is it relatively stable)?
>>>> Would it be possible to cut out a version that only has the bits we
>> think
>>>> are stable (and release that)?
>>>> 
>>>> From that timeline, and the historic release cadence, it would seem to
>> be
>>>> a
>>>> years away before we get to the stable release?
>>>> 
>>>> Thanks,
>>>> 
>>>> Jason
>>>> 
>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
>> cnauroth@hortonworks.com>
>>>> wrote:
>>>> 
>>>>> Hello Doug,
>>>>> 
>>>>> Thanks for your interest in the SSL feature!
>>>>> 
>>>>> At this point, I think we're still pretty far away from declaring a
>>>>> stable
>>>>> release in the 3.5 line.  I don't think we're close enough that anyone
>>>>> can
>>>>> offer a reliable ETA.  This is an earlier thread that describes the
>>>>> high-level strategy for release planning in the 3.5 line:
>>>>> 
>>>>> https://s.apache.org/ADK1
>>>>> 
>>>>> The next step is a 3.5.2-alpha release.  We're working on resolving a
>>>>> few
>>>>> more blockers before we produce a release candidate.  Hopefully that
>>>>> will
>>>>> get done in the next few weeks.
>>>>> 
>>>>> --Chris Nauroth
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>>>> 
>>>>>> I know it's only been a few months, but I was wondering if there was a
>>>>>> ballpark release date for a stable version of 3.5.1. Or is there any
>>>>>> chance
>>>>>> the SSL feature would be added to 3.4.8? Just another person looking
>> to
>>>>>> have
>>>>>> that feature in a stable version. Thanks for all you do! :)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> View this message in context:
>>>>>> 
>>>>> 
>>>>> 
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>>>> e
>>>>>> -tp7581744p7582136.html
>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>> 
>>>>> 
>>>>> 
>>> 
>> 
>> 
>> 
>> 

Re: Zookeeper with SSL release date

[Jason Rosenberg <jb...@squareup.com>]

So, it would seem sensible to me to have a release where all features are
stable, except where noted.  E.g. mark certain features as only 'alpha
quality', e.g. the 're-config feature'.  (I assume it's possible to happily
use 3.5.X without exercising the unstable re-config bits?).

There's precedent for doing this sort of thing in other projects, e.g. in
Kafka, they've had several release where a new "Consumer API" is shipped
that is available for beta-testing, but you can still just use the older
stable consumer api, etc.

Jason

On Wed, Mar 16, 2016 at 2:01 PM, powell molleti <powellm79@yahoo.com.invalid
> wrote:

> Hi Doug,
> Is 3.5 being an alpha release preventing you from using it?. Or have you
> run into issues with it?. In general perhaps ZK 3.5 being labeled as alpha
> might not be fair, since it is far more stable then what most people
> associate an alpha release to be.
> Perhaps if you do not use re-config feature may be it will just work for
> you?.
> There are many examples of 3.5.X being used in productions from my limited
> knowledge.
> ThanksPowell.
>
>     On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <fp...@apache.org>
> wrote:
>
>
>  None of us expected the reconfig changes to take this long to stabilize.
> Until we get there, I don't think we can do anything else with 3.5. The
> best bet we have is to work harder to bring 3.5 into a stable rather than
> trying to work around it.
>
> There are lots of people interested in seeing 3.5 stable, and if we get
> everyone to contribute more patches and code reviews, we should be able to
> do it sooner. After all, it is a community based effort, so the community
> shouldn't rely on just 2-3 people doing the work.
>
> -Flavio
>
> > On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
> wrote:
> >
> > Doug, I forgot to respond to your question about 3.4.  Since 3.4 is the
> > stable maintenance line, we are very conservative about back-porting to
> > it.  Our policy is to limit back-ports to critical bug fixes and not
> > introduce any new features in the 3.4 line.  This is a matter of managing
> > risk.
> >
> > Jason, your question about release cadence is a fair one.  If it's any
> > consolation, we are now taking the approach of trying to limit the scope
> > of anything new introduced in 3.5 too.  That would allow us to focus on
> > stabilization: resolving blocker bugs and freezing public APIs.  I think
> > this will help us accelerate the releases into beta and eventual GA.
> >
> > I am new to ZooKeeper release management, so I'd like to hear thoughts
> > from more experienced committers and PMC members about your proposal to
> > try to cut a stable release for a limited subset of what is in branch-3.5
> > now.  My instinct is that it would be challenging to cherry-pick out
> > pieces of branch-3.5 piecemeal at this point.  This would become another
> > release line for an already resource-constrained volunteer staff to
> > manage.  I'd prefer to dedicate those limited resources to overall 3.5
> > stabilization.  Also, a 3.5 release in which certain features "vanished"
> > because of not meeting some stability criteria would be undesirable.
> >
> > --Chris Nauroth
> >
> >
> >
> >
> > On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> >
> >> Chris,
> >>
> >> Can you say whether some parts of 3.5.X are more stable than others
> (e.g.
> >> if we don't care about certain new features, is it relatively stable)?
> >> Would it be possible to cut out a version that only has the bits we
> think
> >> are stable (and release that)?
> >>
> >> From that timeline, and the historic release cadence, it would seem to
> be
> >> a
> >> years away before we get to the stable release?
> >>
> >> Thanks,
> >>
> >> Jason
> >>
> >> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> cnauroth@hortonworks.com>
> >> wrote:
> >>
> >>> Hello Doug,
> >>>
> >>> Thanks for your interest in the SSL feature!
> >>>
> >>> At this point, I think we're still pretty far away from declaring a
> >>> stable
> >>> release in the 3.5 line.  I don't think we're close enough that anyone
> >>> can
> >>> offer a reliable ETA.  This is an earlier thread that describes the
> >>> high-level strategy for release planning in the 3.5 line:
> >>>
> >>> https://s.apache.org/ADK1
> >>>
> >>> The next step is a 3.5.2-alpha release.  We're working on resolving a
> >>> few
> >>> more blockers before we produce a release candidate.  Hopefully that
> >>> will
> >>> get done in the next few weeks.
> >>>
> >>> --Chris Nauroth
> >>>
> >>>
> >>>
> >>>
> >>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> >>>
> >>>> I know it's only been a few months, but I was wondering if there was a
> >>>> ballpark release date for a stable version of 3.5.1. Or is there any
> >>>> chance
> >>>> the SSL feature would be added to 3.4.8? Just another person looking
> to
> >>>> have
> >>>> that feature in a stable version. Thanks for all you do! :)
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> View this message in context:
> >>>>
> >>>
> >>>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >>> e
> >>>> -tp7581744p7582136.html
> >>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
> >>>>
> >>>
> >>>
> >
>
>
>
>

Re: Zookeeper with SSL release date

[powell molleti <po...@yahoo.com.INVALID>]

Hi Doug,
Is 3.5 being an alpha release preventing you from using it?. Or have you run into issues with it?. In general perhaps ZK 3.5 being labeled as alpha might not be fair, since it is far more stable then what most people associate an alpha release to be.
Perhaps if you do not use re-config feature may be it will just work for you?.
There are many examples of 3.5.X being used in productions from my limited knowledge.
ThanksPowell. 

    On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira <fp...@apache.org> wrote:
 

 None of us expected the reconfig changes to take this long to stabilize. Until we get there, I don't think we can do anything else with 3.5. The best bet we have is to work harder to bring 3.5 into a stable rather than trying to work around it.

There are lots of people interested in seeing 3.5 stable, and if we get everyone to contribute more patches and code reviews, we should be able to do it sooner. After all, it is a community based effort, so the community shouldn't rely on just 2-3 people doing the work.

-Flavio

> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com> wrote:
> 
> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is the
> stable maintenance line, we are very conservative about back-porting to
> it.  Our policy is to limit back-ports to critical bug fixes and not
> introduce any new features in the 3.4 line.  This is a matter of managing
> risk.
> 
> Jason, your question about release cadence is a fair one.  If it's any
> consolation, we are now taking the approach of trying to limit the scope
> of anything new introduced in 3.5 too.  That would allow us to focus on
> stabilization: resolving blocker bugs and freezing public APIs.  I think
> this will help us accelerate the releases into beta and eventual GA.
> 
> I am new to ZooKeeper release management, so I'd like to hear thoughts
> from more experienced committers and PMC members about your proposal to
> try to cut a stable release for a limited subset of what is in branch-3.5
> now.  My instinct is that it would be challenging to cherry-pick out
> pieces of branch-3.5 piecemeal at this point.  This would become another
> release line for an already resource-constrained volunteer staff to
> manage.  I'd prefer to dedicate those limited resources to overall 3.5
> stabilization.  Also, a 3.5 release in which certain features "vanished"
> because of not meeting some stability criteria would be undesirable.
> 
> --Chris Nauroth
> 
> 
> 
> 
> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> 
>> Chris,
>> 
>> Can you say whether some parts of 3.5.X are more stable than others (e.g.
>> if we don't care about certain new features, is it relatively stable)?
>> Would it be possible to cut out a version that only has the bits we think
>> are stable (and release that)?
>> 
>> From that timeline, and the historic release cadence, it would seem to be
>> a
>> years away before we get to the stable release?
>> 
>> Thanks,
>> 
>> Jason
>> 
>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>> 
>>> Hello Doug,
>>> 
>>> Thanks for your interest in the SSL feature!
>>> 
>>> At this point, I think we're still pretty far away from declaring a
>>> stable
>>> release in the 3.5 line.  I don't think we're close enough that anyone
>>> can
>>> offer a reliable ETA.  This is an earlier thread that describes the
>>> high-level strategy for release planning in the 3.5 line:
>>> 
>>> https://s.apache.org/ADK1
>>> 
>>> The next step is a 3.5.2-alpha release.  We're working on resolving a
>>> few
>>> more blockers before we produce a release candidate.  Hopefully that
>>> will
>>> get done in the next few weeks.
>>> 
>>> --Chris Nauroth
>>> 
>>> 
>>> 
>>> 
>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>> 
>>>> I know it's only been a few months, but I was wondering if there was a
>>>> ballpark release date for a stable version of 3.5.1. Or is there any
>>>> chance
>>>> the SSL feature would be added to 3.4.8? Just another person looking to
>>>> have
>>>> that feature in a stable version. Thanks for all you do! :)
>>>> 
>>>> 
>>>> 
>>>> --
>>>> View this message in context:
>>>> 
>>> 
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>> e
>>>> -tp7581744p7582136.html
>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>> 
>>> 
>>> 
> 


  

Re: Zookeeper with SSL release date

[Jason Rosenberg <jb...@squareup.com>]

So, is it specifically the "reconfig changes"?  Can you describe?

On Wed, Mar 16, 2016 at 5:33 AM, Flavio Junqueira <fp...@apache.org> wrote:

> None of us expected the reconfig changes to take this long to stabilize.
> Until we get there, I don't think we can do anything else with 3.5. The
> best bet we have is to work harder to bring 3.5 into a stable rather than
> trying to work around it.
>
> There are lots of people interested in seeing 3.5 stable, and if we get
> everyone to contribute more patches and code reviews, we should be able to
> do it sooner. After all, it is a community based effort, so the community
> shouldn't rely on just 2-3 people doing the work.
>
> -Flavio
>
> > On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com>
> wrote:
> >
> > Doug, I forgot to respond to your question about 3.4.  Since 3.4 is the
> > stable maintenance line, we are very conservative about back-porting to
> > it.  Our policy is to limit back-ports to critical bug fixes and not
> > introduce any new features in the 3.4 line.  This is a matter of managing
> > risk.
> >
> > Jason, your question about release cadence is a fair one.  If it's any
> > consolation, we are now taking the approach of trying to limit the scope
> > of anything new introduced in 3.5 too.  That would allow us to focus on
> > stabilization: resolving blocker bugs and freezing public APIs.  I think
> > this will help us accelerate the releases into beta and eventual GA.
> >
> > I am new to ZooKeeper release management, so I'd like to hear thoughts
> > from more experienced committers and PMC members about your proposal to
> > try to cut a stable release for a limited subset of what is in branch-3.5
> > now.  My instinct is that it would be challenging to cherry-pick out
> > pieces of branch-3.5 piecemeal at this point.  This would become another
> > release line for an already resource-constrained volunteer staff to
> > manage.  I'd prefer to dedicate those limited resources to overall 3.5
> > stabilization.  Also, a 3.5 release in which certain features "vanished"
> > because of not meeting some stability criteria would be undesirable.
> >
> > --Chris Nauroth
> >
> >
> >
> >
> > On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> >
> >> Chris,
> >>
> >> Can you say whether some parts of 3.5.X are more stable than others
> (e.g.
> >> if we don't care about certain new features, is it relatively stable)?
> >> Would it be possible to cut out a version that only has the bits we
> think
> >> are stable (and release that)?
> >>
> >> From that timeline, and the historic release cadence, it would seem to
> be
> >> a
> >> years away before we get to the stable release?
> >>
> >> Thanks,
> >>
> >> Jason
> >>
> >> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <
> cnauroth@hortonworks.com>
> >> wrote:
> >>
> >>> Hello Doug,
> >>>
> >>> Thanks for your interest in the SSL feature!
> >>>
> >>> At this point, I think we're still pretty far away from declaring a
> >>> stable
> >>> release in the 3.5 line.  I don't think we're close enough that anyone
> >>> can
> >>> offer a reliable ETA.  This is an earlier thread that describes the
> >>> high-level strategy for release planning in the 3.5 line:
> >>>
> >>> https://s.apache.org/ADK1
> >>>
> >>> The next step is a 3.5.2-alpha release.  We're working on resolving a
> >>> few
> >>> more blockers before we produce a release candidate.  Hopefully that
> >>> will
> >>> get done in the next few weeks.
> >>>
> >>> --Chris Nauroth
> >>>
> >>>
> >>>
> >>>
> >>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
> >>>
> >>>> I know it's only been a few months, but I was wondering if there was a
> >>>> ballpark release date for a stable version of 3.5.1. Or is there any
> >>>> chance
> >>>> the SSL feature would be added to 3.4.8? Just another person looking
> to
> >>>> have
> >>>> that feature in a stable version. Thanks for all you do! :)
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> View this message in context:
> >>>>
> >>>
> >>>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
> >>> e
> >>>> -tp7581744p7582136.html
> >>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
> >>>>
> >>>
> >>>
> >
>
>

Re: Zookeeper with SSL release date

[Flavio Junqueira <fp...@apache.org>]

None of us expected the reconfig changes to take this long to stabilize. Until we get there, I don't think we can do anything else with 3.5. The best bet we have is to work harder to bring 3.5 into a stable rather than trying to work around it.

There are lots of people interested in seeing 3.5 stable, and if we get everyone to contribute more patches and code reviews, we should be able to do it sooner. After all, it is a community based effort, so the community shouldn't rely on just 2-3 people doing the work.

-Flavio

> On 15 Mar 2016, at 17:28, Chris Nauroth <cn...@hortonworks.com> wrote:
> 
> Doug, I forgot to respond to your question about 3.4.  Since 3.4 is the
> stable maintenance line, we are very conservative about back-porting to
> it.  Our policy is to limit back-ports to critical bug fixes and not
> introduce any new features in the 3.4 line.  This is a matter of managing
> risk.
> 
> Jason, your question about release cadence is a fair one.  If it's any
> consolation, we are now taking the approach of trying to limit the scope
> of anything new introduced in 3.5 too.  That would allow us to focus on
> stabilization: resolving blocker bugs and freezing public APIs.  I think
> this will help us accelerate the releases into beta and eventual GA.
> 
> I am new to ZooKeeper release management, so I'd like to hear thoughts
> from more experienced committers and PMC members about your proposal to
> try to cut a stable release for a limited subset of what is in branch-3.5
> now.  My instinct is that it would be challenging to cherry-pick out
> pieces of branch-3.5 piecemeal at this point.  This would become another
> release line for an already resource-constrained volunteer staff to
> manage.  I'd prefer to dedicate those limited resources to overall 3.5
> stabilization.  Also, a 3.5 release in which certain features "vanished"
> because of not meeting some stability criteria would be undesirable.
> 
> --Chris Nauroth
> 
> 
> 
> 
> On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:
> 
>> Chris,
>> 
>> Can you say whether some parts of 3.5.X are more stable than others (e.g.
>> if we don't care about certain new features, is it relatively stable)?
>> Would it be possible to cut out a version that only has the bits we think
>> are stable (and release that)?
>> 
>> From that timeline, and the historic release cadence, it would seem to be
>> a
>> years away before we get to the stable release?
>> 
>> Thanks,
>> 
>> Jason
>> 
>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>> 
>>> Hello Doug,
>>> 
>>> Thanks for your interest in the SSL feature!
>>> 
>>> At this point, I think we're still pretty far away from declaring a
>>> stable
>>> release in the 3.5 line.  I don't think we're close enough that anyone
>>> can
>>> offer a reliable ETA.  This is an earlier thread that describes the
>>> high-level strategy for release planning in the 3.5 line:
>>> 
>>> https://s.apache.org/ADK1
>>> 
>>> The next step is a 3.5.2-alpha release.  We're working on resolving a
>>> few
>>> more blockers before we produce a release candidate.  Hopefully that
>>> will
>>> get done in the next few weeks.
>>> 
>>> --Chris Nauroth
>>> 
>>> 
>>> 
>>> 
>>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>> 
>>>> I know it's only been a few months, but I was wondering if there was a
>>>> ballpark release date for a stable version of 3.5.1. Or is there any
>>>> chance
>>>> the SSL feature would be added to 3.4.8? Just another person looking to
>>>> have
>>>> that feature in a stable version. Thanks for all you do! :)
>>>> 
>>>> 
>>>> 
>>>> --
>>>> View this message in context:
>>>> 
>>> 
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>> e
>>>> -tp7581744p7582136.html
>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>> 
>>> 
>>> 
> 

Re: Zookeeper with SSL release date

[Chris Nauroth <cn...@hortonworks.com>]

Doug, I forgot to respond to your question about 3.4.  Since 3.4 is the
stable maintenance line, we are very conservative about back-porting to
it.  Our policy is to limit back-ports to critical bug fixes and not
introduce any new features in the 3.4 line.  This is a matter of managing
risk.

Jason, your question about release cadence is a fair one.  If it's any
consolation, we are now taking the approach of trying to limit the scope
of anything new introduced in 3.5 too.  That would allow us to focus on
stabilization: resolving blocker bugs and freezing public APIs.  I think
this will help us accelerate the releases into beta and eventual GA.

I am new to ZooKeeper release management, so I'd like to hear thoughts
from more experienced committers and PMC members about your proposal to
try to cut a stable release for a limited subset of what is in branch-3.5
now.  My instinct is that it would be challenging to cherry-pick out
pieces of branch-3.5 piecemeal at this point.  This would become another
release line for an already resource-constrained volunteer staff to
manage.  I'd prefer to dedicate those limited resources to overall 3.5
stabilization.  Also, a 3.5 release in which certain features "vanished"
because of not meeting some stability criteria would be undesirable.

--Chris Nauroth




On 3/15/16, 10:12 AM, "Jason Rosenberg" <jb...@squareup.com> wrote:

>Chris,
>
>Can you say whether some parts of 3.5.X are more stable than others (e.g.
>if we don't care about certain new features, is it relatively stable)?
>Would it be possible to cut out a version that only has the bits we think
>are stable (and release that)?
>
>From that timeline, and the historic release cadence, it would seem to be
>a
>years away before we get to the stable release?
>
>Thanks,
>
>Jason
>
>On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <cn...@hortonworks.com>
>wrote:
>
>> Hello Doug,
>>
>> Thanks for your interest in the SSL feature!
>>
>> At this point, I think we're still pretty far away from declaring a
>>stable
>> release in the 3.5 line.  I don't think we're close enough that anyone
>>can
>> offer a reliable ETA.  This is an earlier thread that describes the
>> high-level strategy for release planning in the 3.5 line:
>>
>> https://s.apache.org/ADK1
>>
>> The next step is a 3.5.2-alpha release.  We're working on resolving a
>>few
>> more blockers before we produce a release candidate.  Hopefully that
>>will
>> get done in the next few weeks.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>>
>> >I know it's only been a few months, but I was wondering if there was a
>> >ballpark release date for a stable version of 3.5.1. Or is there any
>> >chance
>> >the SSL feature would be added to 3.4.8? Just another person looking to
>> >have
>> >that feature in a stable version. Thanks for all you do! :)
>> >
>> >
>> >
>> >--
>> >View this message in context:
>> >
>> 
>>http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>e
>> >-tp7581744p7582136.html
>> >Sent from the zookeeper-user mailing list archive at Nabble.com.
>> >
>>
>>

Re: Zookeeper with SSL release date

[Jason Rosenberg <jb...@squareup.com>]

Chris,

Can you say whether some parts of 3.5.X are more stable than others (e.g.
if we don't care about certain new features, is it relatively stable)?
Would it be possible to cut out a version that only has the bits we think
are stable (and release that)?

>From that timeline, and the historic release cadence, it would seem to be a
years away before we get to the stable release?

Thanks,

Jason

On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:

> Hello Doug,
>
> Thanks for your interest in the SSL feature!
>
> At this point, I think we're still pretty far away from declaring a stable
> release in the 3.5 line.  I don't think we're close enough that anyone can
> offer a reliable ETA.  This is an earlier thread that describes the
> high-level strategy for release planning in the 3.5 line:
>
> https://s.apache.org/ADK1
>
> The next step is a 3.5.2-alpha release.  We're working on resolving a few
> more blockers before we produce a release candidate.  Hopefully that will
> get done in the next few weeks.
>
> --Chris Nauroth
>
>
>
>
> On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:
>
> >I know it's only been a few months, but I was wondering if there was a
> >ballpark release date for a stable version of 3.5.1. Or is there any
> >chance
> >the SSL feature would be added to 3.4.8? Just another person looking to
> >have
> >that feature in a stable version. Thanks for all you do! :)
> >
> >
> >
> >--
> >View this message in context:
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-date
> >-tp7581744p7582136.html
> >Sent from the zookeeper-user mailing list archive at Nabble.com.
> >
>
>

Re: Zookeeper with SSL release date

[Chris Nauroth <cn...@hortonworks.com>]

Hello Doug,

Thanks for your interest in the SSL feature!

At this point, I think we're still pretty far away from declaring a stable
release in the 3.5 line.  I don't think we're close enough that anyone can
offer a reliable ETA.  This is an earlier thread that describes the
high-level strategy for release planning in the 3.5 line:

https://s.apache.org/ADK1

The next step is a 3.5.2-alpha release.  We're working on resolving a few
more blockers before we produce a release candidate.  Hopefully that will
get done in the next few weeks.

--Chris Nauroth




On 3/15/16, 9:39 AM, "Doug" <it...@gmail.com> wrote:

>I know it's only been a few months, but I was wondering if there was a
>ballpark release date for a stable version of 3.5.1. Or is there any
>chance
>the SSL feature would be added to 3.4.8? Just another person looking to
>have
>that feature in a stable version. Thanks for all you do! :)
>
>
>
>--
>View this message in context:
>http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-date
>-tp7581744p7582136.html
>Sent from the zookeeper-user mailing list archive at Nabble.com.
>

Re: Zookeeper with SSL release date

[Doug <it...@gmail.com>]

I know it's only been a few months, but I was wondering if there was a
ballpark release date for a stable version of 3.5.1. Or is there any chance
the SSL feature would be added to 3.4.8? Just another person looking to have
that feature in a stable version. Thanks for all you do! :)



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-date-tp7581744p7582136.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: Zookeeper with SSL release date

[Patrick Hunt <ph...@apache.org>]

Hi. There is no fixed date yet. We're working through a number of blocker
issues, those need to be addressed before GA would be available.

Patrick

On Fri, Nov 13, 2015 at 12:38 AM, <ma...@free.fr> wrote:

> Hi,
>
> I can see that Zookeeper supports SSL in version 3.5.1-alpha. When SSL
> support will be finally released so that it can be used in production ?
>
> Thanks
>