You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Enrico Olivelli <eo...@gmail.com> on 2020/07/01 06:36:31 UTC
Re: An official API for starting ZooKeeper from a Java program
Il giorno gio 25 giu 2020 alle ore 22:48 Patrick Hunt <ph...@apache.org> ha
scritto:
> On Thu, Jun 25, 2020 at 1:45 PM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > Il Gio 25 Giu 2020, 22:13 Patrick Hunt <ph...@apache.org> ha scritto:
> >
> > > On Thu, Jun 25, 2020 at 3:20 AM Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > > we recently got into ZOOKEEPER-3803
> > FileTxnSnapLog.fastForwardFromEdits()
> > > > throws NPE if TestingServer is started from another thread (see [1])
> > > > and I have similar cases in other non OS products that run ZooKeeper
> > from
> > > > Java.
> > > >
> > > > The case of ZOOKEEPER-3803 is for Curator TestingServer, and here we
> > are
> > > > talking about running a ZK server for testing purposes.
> > > > But I have other usecases in which the ZK server must be launched,
> for
> > > > production usage, from a Java based bootstrap launcher.
> > > >
> > > > We are now officially supporting our binary package distribution that
> > > runs
> > > > ZooKeeper from a bash script and the bash script is coupled with
> > > > ZooKeeperMain and QuorumPeerMain.
> > > >
> > > > In my opinion we should provide a well known and supported API,
> better
> > > than
> > > > using directly those two classes, to run ZooKeeper safely in
> > production,
> > > > but launched from Java.
> > > >
> > > > I am not here supporting or suggesting the idea of running ZooKeeper
> > > inside
> > > > the same process of a client application,
> > > > but only to provide a clear and stable API to start/stop and do
> minimal
> > > > health checks to a ZooKeeper peer.
> > > >
> > > > I will be happy to work on it
> > > >
> > > > Thoughts ?
> > > >
> > >
> > > Sounds reasonable to me. That said I'm not sure I follow you entirely.
> > > Isn't that the goal of the two Main classes? Is it that they are
> > deficient
> > > (and can therefore be fixed to address) or they are serving a different
> > > role entirely from what you intend to provide?
> > >
> >
> > Those classes do not have a clear interface.
> >
> > We need at least
> > - init(configuration)
> > - start()
> > - stop()
> > - boolean isAlive()
> >
> >
> Makes sense to me. Can we refactor the Main classes to include/use that and
> also doc/pub it as an public/enduser interface? Possible user/consumer
> visible regressions may make that a bad idea? Or do you want to have just
> one interface that supports any cluster type, distributed or non. I could
> see both approaches working.
>
I would prefer to not change existing code, and create a wrapper around
ZooKeeperMain and other classes.
ZooKeeper bootstrap code is already used in many fancy ways, the less we
change it the better it is for the sake of stability.
This is the JIRA with complete design choices
https://issues.apache.org/jira/browse/ZOOKEEPER-3874
I am planning to work on it as soon as possible, I would like this to be
delivered with 3.7.0
Enrico
>
> Patrick
>
>
> > Optionally we can provide some utility to get the endpoint address.
> >
> > Very similar to Curator TestingServer but:
> > - for production
> > - maintained by Zookeeper project
> >
> >
> > Enrico
> >
> >
> >
> > > Patrick
> > >
> > >
> > > >
> > > > Enrico
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/ZOOKEEPER-3803
> > > >
> > >
> >
>
Re: An official API for starting ZooKeeper from a Java program
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
The issue is trying to start ZooKeeper programmatically. ZooKeeperServerMain assumes starting from the CLI with zoo.cfg, etc. If you want to do all that programmatically it's more difficult. What is needed is a version of ZooKeeperServerMain for programmatic use.
> On Jul 1, 2020, at 3:41 PM, Christopher <ct...@apache.org> wrote:
>
> In my experience, the current ZooKeeperServerMain is adequate for
> this. I currently wrap that for my (unofficial) zookeeper-maven-plugin
> project (https://github.com/revelc/zookeeper-maven-plugin). I'm
> curious what additional benefits having a new API would add that can't
> be done with the current ones.
>
> I'm in favor of removing the reliance on the scripts to start
> ZooKeeper, but from what I can tell, the dependence on those is
> already pretty minimal (I didn't need it for my wrapping, and it also
> doesn't seem to have been necessary for Apache Accumulo's
> MiniAccumuloCluster, which also launches ZooKeeper in Java code with
> minimal effort).
>
> On Wed, Jul 1, 2020 at 2:36 AM Enrico Olivelli <eo...@gmail.com> wrote:
>>
>> Il giorno gio 25 giu 2020 alle ore 22:48 Patrick Hunt <ph...@apache.org> ha
>> scritto:
>>
>>> On Thu, Jun 25, 2020 at 1:45 PM Enrico Olivelli <eo...@gmail.com>
>>> wrote:
>>>
>>>> Il Gio 25 Giu 2020, 22:13 Patrick Hunt <ph...@apache.org> ha scritto:
>>>>
>>>>> On Thu, Jun 25, 2020 at 3:20 AM Enrico Olivelli <eo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>> we recently got into ZOOKEEPER-3803
>>>> FileTxnSnapLog.fastForwardFromEdits()
>>>>>> throws NPE if TestingServer is started from another thread (see [1])
>>>>>> and I have similar cases in other non OS products that run ZooKeeper
>>>> from
>>>>>> Java.
>>>>>>
>>>>>> The case of ZOOKEEPER-3803 is for Curator TestingServer, and here we
>>>> are
>>>>>> talking about running a ZK server for testing purposes.
>>>>>> But I have other usecases in which the ZK server must be launched,
>>> for
>>>>>> production usage, from a Java based bootstrap launcher.
>>>>>>
>>>>>> We are now officially supporting our binary package distribution that
>>>>> runs
>>>>>> ZooKeeper from a bash script and the bash script is coupled with
>>>>>> ZooKeeperMain and QuorumPeerMain.
>>>>>>
>>>>>> In my opinion we should provide a well known and supported API,
>>> better
>>>>> than
>>>>>> using directly those two classes, to run ZooKeeper safely in
>>>> production,
>>>>>> but launched from Java.
>>>>>>
>>>>>> I am not here supporting or suggesting the idea of running ZooKeeper
>>>>> inside
>>>>>> the same process of a client application,
>>>>>> but only to provide a clear and stable API to start/stop and do
>>> minimal
>>>>>> health checks to a ZooKeeper peer.
>>>>>>
>>>>>> I will be happy to work on it
>>>>>>
>>>>>> Thoughts ?
>>>>>>
>>>>>
>>>>> Sounds reasonable to me. That said I'm not sure I follow you entirely.
>>>>> Isn't that the goal of the two Main classes? Is it that they are
>>>> deficient
>>>>> (and can therefore be fixed to address) or they are serving a different
>>>>> role entirely from what you intend to provide?
>>>>>
>>>>
>>>> Those classes do not have a clear interface.
>>>>
>>>> We need at least
>>>> - init(configuration)
>>>> - start()
>>>> - stop()
>>>> - boolean isAlive()
>>>>
>>>>
>>> Makes sense to me. Can we refactor the Main classes to include/use that and
>>> also doc/pub it as an public/enduser interface? Possible user/consumer
>>> visible regressions may make that a bad idea? Or do you want to have just
>>> one interface that supports any cluster type, distributed or non. I could
>>> see both approaches working.
>>>
>>
>> I would prefer to not change existing code, and create a wrapper around
>> ZooKeeperMain and other classes.
>> ZooKeeper bootstrap code is already used in many fancy ways, the less we
>> change it the better it is for the sake of stability.
>>
>> This is the JIRA with complete design choices
>> https://issues.apache.org/jira/browse/ZOOKEEPER-3874
>>
>> I am planning to work on it as soon as possible, I would like this to be
>> delivered with 3.7.0
>>
>> Enrico
>>
>>
>>>
>>> Patrick
>>>
>>>
>>>> Optionally we can provide some utility to get the endpoint address.
>>>>
>>>> Very similar to Curator TestingServer but:
>>>> - for production
>>>> - maintained by Zookeeper project
>>>>
>>>>
>>>> Enrico
>>>>
>>>>
>>>>
>>>>> Patrick
>>>>>
>>>>>
>>>>>>
>>>>>> Enrico
>>>>>>
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-3803
>>>>>>
>>>>>
>>>>
>>>
Re: An official API for starting ZooKeeper from a Java program
Posted by Christopher <ct...@apache.org>.
In my experience, the current ZooKeeperServerMain is adequate for
this. I currently wrap that for my (unofficial) zookeeper-maven-plugin
project (https://github.com/revelc/zookeeper-maven-plugin). I'm
curious what additional benefits having a new API would add that can't
be done with the current ones.
I'm in favor of removing the reliance on the scripts to start
ZooKeeper, but from what I can tell, the dependence on those is
already pretty minimal (I didn't need it for my wrapping, and it also
doesn't seem to have been necessary for Apache Accumulo's
MiniAccumuloCluster, which also launches ZooKeeper in Java code with
minimal effort).
On Wed, Jul 1, 2020 at 2:36 AM Enrico Olivelli <eo...@gmail.com> wrote:
>
> Il giorno gio 25 giu 2020 alle ore 22:48 Patrick Hunt <ph...@apache.org> ha
> scritto:
>
> > On Thu, Jun 25, 2020 at 1:45 PM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> > > Il Gio 25 Giu 2020, 22:13 Patrick Hunt <ph...@apache.org> ha scritto:
> > >
> > > > On Thu, Jun 25, 2020 at 3:20 AM Enrico Olivelli <eo...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > > we recently got into ZOOKEEPER-3803
> > > FileTxnSnapLog.fastForwardFromEdits()
> > > > > throws NPE if TestingServer is started from another thread (see [1])
> > > > > and I have similar cases in other non OS products that run ZooKeeper
> > > from
> > > > > Java.
> > > > >
> > > > > The case of ZOOKEEPER-3803 is for Curator TestingServer, and here we
> > > are
> > > > > talking about running a ZK server for testing purposes.
> > > > > But I have other usecases in which the ZK server must be launched,
> > for
> > > > > production usage, from a Java based bootstrap launcher.
> > > > >
> > > > > We are now officially supporting our binary package distribution that
> > > > runs
> > > > > ZooKeeper from a bash script and the bash script is coupled with
> > > > > ZooKeeperMain and QuorumPeerMain.
> > > > >
> > > > > In my opinion we should provide a well known and supported API,
> > better
> > > > than
> > > > > using directly those two classes, to run ZooKeeper safely in
> > > production,
> > > > > but launched from Java.
> > > > >
> > > > > I am not here supporting or suggesting the idea of running ZooKeeper
> > > > inside
> > > > > the same process of a client application,
> > > > > but only to provide a clear and stable API to start/stop and do
> > minimal
> > > > > health checks to a ZooKeeper peer.
> > > > >
> > > > > I will be happy to work on it
> > > > >
> > > > > Thoughts ?
> > > > >
> > > >
> > > > Sounds reasonable to me. That said I'm not sure I follow you entirely.
> > > > Isn't that the goal of the two Main classes? Is it that they are
> > > deficient
> > > > (and can therefore be fixed to address) or they are serving a different
> > > > role entirely from what you intend to provide?
> > > >
> > >
> > > Those classes do not have a clear interface.
> > >
> > > We need at least
> > > - init(configuration)
> > > - start()
> > > - stop()
> > > - boolean isAlive()
> > >
> > >
> > Makes sense to me. Can we refactor the Main classes to include/use that and
> > also doc/pub it as an public/enduser interface? Possible user/consumer
> > visible regressions may make that a bad idea? Or do you want to have just
> > one interface that supports any cluster type, distributed or non. I could
> > see both approaches working.
> >
>
> I would prefer to not change existing code, and create a wrapper around
> ZooKeeperMain and other classes.
> ZooKeeper bootstrap code is already used in many fancy ways, the less we
> change it the better it is for the sake of stability.
>
> This is the JIRA with complete design choices
> https://issues.apache.org/jira/browse/ZOOKEEPER-3874
>
> I am planning to work on it as soon as possible, I would like this to be
> delivered with 3.7.0
>
> Enrico
>
>
> >
> > Patrick
> >
> >
> > > Optionally we can provide some utility to get the endpoint address.
> > >
> > > Very similar to Curator TestingServer but:
> > > - for production
> > > - maintained by Zookeeper project
> > >
> > >
> > > Enrico
> > >
> > >
> > >
> > > > Patrick
> > > >
> > > >
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/ZOOKEEPER-3803
> > > > >
> > > >
> > >
> >