You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@curator.apache.org by Vikrant Singh <vi...@gmail.com> on 2015/10/12 21:03:23 UTC
Persistent Ephemeral Node
I have some basic question on persistent ephemeral node.
Here is some background...
We have a zoo keeper based service discovery setup. Each service register
itself as a ephemeral node with zookeeper.When a service go down its
ephemeral node is removed from zookeeper and we know that service is down
and we provision it again.
At present we create plain ephemeral node. I am wondering what
benefit/risks we may get if move to persistent ephemeral ones. I see one
problem... using plane ephemeral node we can rely on state of ZK to make a
decision like service is down. This is because we are sure that if a node
get deleted with zoo keeper it will never comeback from same process. But
if moved to "persistent ephemeral" I guess same may not be the case.
Please let me know what you think of the same.
Also I would like to know what are the best scenario where one should
prefer using persistent ephemeral node over ephemeral node.
Thanks,
Vikrant
Re: Persistent Ephemeral Node
Posted by Vikrant Singh <vi...@gmail.com>.
thanks for the response... yes I want the same feature i.e. not worrying
about recreation of the node if connection loss is temporary.
Being said that.. I also want to make sure that we restart a new instance
of service when it dies. To do that I am using tree cache, a standby
process keep an eye on every node through tree cache and moment it sees one
node going down it provision itself as the same.
Problem is tree cache does not know about nature of ephemeral nodes. So
standby process will treat them same and I will end up in a situation where
two processes are trying to create same node.
On Mon, Oct 12, 2015 at 2:04 PM, Cameron McKenzie <mc...@gmail.com>
wrote:
> hey Vikrant,
> Using a persistent ephemeral node just means that your application code
> doesn't need to worry about handling recreation of the node when it
> reconnects to ZooKeeper after connection / session loss.
>
> If your ephemeral node should always be present whenever your application
> instance is running, then this would be a good use case for a persistent
> ephemeral node.
> cheers
>
>
> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <
> vikrant.subscribe@gmail.com> wrote:
>
>> I have some basic question on persistent ephemeral node.
>>
>> Here is some background...
>>
>> We have a zoo keeper based service discovery setup. Each service register
>> itself as a ephemeral node with zookeeper.When a service go down its
>> ephemeral node is removed from zookeeper and we know that service is down
>> and we provision it again.
>>
>> At present we create plain ephemeral node. I am wondering what
>> benefit/risks we may get if move to persistent ephemeral ones. I see one
>> problem... using plane ephemeral node we can rely on state of ZK to make a
>> decision like service is down. This is because we are sure that if a node
>> get deleted with zoo keeper it will never comeback from same process. But
>> if moved to "persistent ephemeral" I guess same may not be the case.
>>
>> Please let me know what you think of the same.
>>
>> Also I would like to know what are the best scenario where one should
>> prefer using persistent ephemeral node over ephemeral node.
>>
>> Thanks,
>> Vikrant
>>
>
>
Re: Persistent Ephemeral Node
Posted by Cameron McKenzie <mc...@gmail.com>.
I guess you could work it out based on the ExponentialBackoffRetry
algorithm, but it seems pretty ugly. Is it possible for you to instead just
start up a new instance of the application immediately, and then if your
crashed / seperated application comes back online, it can check if the
ephemeral node is already present and if it is just shutdown?
That would seem much less error prone.
On Thu, Oct 15, 2015 at 9:59 AM, Vikrant Singh <vi...@gmail.com>
wrote:
> Oh yes, I can do that. I can fix the path.
> I create these node using default session time outs in curator framework..
> I use ExponentialBackoffRetry with retry time out of 1s and retry count
> of 29.
>
> So when I get a remove event for a PEM node... how long I should wait to
> make sure that its client has given up on tries to rewrite it?
>
> On Wed, Oct 14, 2015 at 3:51 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> Store your PEMs in a known path. Or, store a known value in the node.
>>
>> -JZ
>>
>>
>> On Oct 14, 2015, at 5:49 PM, Vikrant Singh <vi...@gmail.com>
>> wrote:
>>
>> What I want is to differentiate between "ephemeral nodes" and
>> "persistent ephemeral node".
>>
>> On Wed, Oct 14, 2015 at 3:20 PM, Cameron McKenzie <mckenzie.cam@gmail.com
>> > wrote:
>>
>>> You could keep track of which nodes are ephemeral by looking at the stat
>>> object passed to the NODE_ADDED event in the TreeCache. Then when you get a
>>> NODE_REMOVED event you could check if it's one of your ephemeral nodes?
>>>
>>> On Thu, Oct 15, 2015 at 4:06 AM, Vikrant Singh <
>>> vikrant.subscribe@gmail.com> wrote:
>>>
>>>> yes you got it correct... a slight correction. I am trying to find a
>>>> way where I can make my tree cache event handler aware that "node removed"
>>>> event is coming from a persistent ephemeral node and delay any action till
>>>> the point the node's curator framework gives up on rewriting the node to ZK.
>>>>
>>>> On Mon, Oct 12, 2015 at 3:38 PM, Cameron McKenzie <
>>>> mckenzie.cam@gmail.com> wrote:
>>>>
>>>>> So, if I'm understanding correctly, if your application loses its
>>>>> connection to ZooKeeper (or crashes), then a new instance will be started
>>>>> in its place, and if the original instance reconnects then you don't want
>>>>> it to try and recreate its ephemeral node?
>>>>>
>>>>> If that's the case, then I think that you need additional logic beyond
>>>>> what a PersistentEphemeralNode is going to provide you, as you will need to
>>>>> check if the node already exists on reconnection.
>>>>>
>>>>> On Tue, Oct 13, 2015 at 8:43 AM, Vikrant Singh <
>>>>> vikrant.subscribe@gmail.com> wrote:
>>>>>
>>>>>> yeah I considered that.. but I ended up using tree cache because it
>>>>>> gives me more control on the state of tree . I have some custom handlers
>>>>>> for add/remove/update events. Same cache is also used for service discovery.
>>>>>>
>>>>>> On Mon, Oct 12, 2015 at 2:33 PM, David Kesler <DK...@yodle.com>
>>>>>> wrote:
>>>>>>
>>>>>>> There’s also already a Curator service discovery extension library
>>>>>>> that you can look at:
>>>>>>> http://curator.apache.org/curator-x-discovery/index.html. It’s
>>>>>>> basically boiling down to the same strategy of sticking an ephemeral node
>>>>>>> into ZK, but with some additional convenience and functionality built on
>>>>>>> top of it.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *From:* Cameron McKenzie [mailto:mckenzie.cam@gmail.com]
>>>>>>> *Sent:* Monday, October 12, 2015 5:05 PM
>>>>>>> *To:* user@curator.apache.org
>>>>>>> *Subject:* Re: Persistent Ephemeral Node
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> hey Vikrant,
>>>>>>>
>>>>>>> Using a persistent ephemeral node just means that your application
>>>>>>> code doesn't need to worry about handling recreation of the node when it
>>>>>>> reconnects to ZooKeeper after connection / session loss.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> If your ephemeral node should always be present whenever your
>>>>>>> application instance is running, then this would be a good use case for a
>>>>>>> persistent ephemeral node.
>>>>>>>
>>>>>>> cheers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <
>>>>>>> vikrant.subscribe@gmail.com> wrote:
>>>>>>>
>>>>>>> I have some basic question on persistent ephemeral node.
>>>>>>>
>>>>>>> Here is some background...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> We have a zoo keeper based service discovery setup. Each service
>>>>>>> register itself as a ephemeral node with zookeeper.When a service go down
>>>>>>> its ephemeral node is removed from zookeeper and we know that service is
>>>>>>> down and we provision it again.
>>>>>>>
>>>>>>> At present we create plain ephemeral node. I am wondering what
>>>>>>> benefit/risks we may get if move to persistent ephemeral ones. I see one
>>>>>>> problem... using plane ephemeral node we can rely on state of ZK to make a
>>>>>>> decision like service is down. This is because we are sure that if a node
>>>>>>> get deleted with zoo keeper it will never comeback from same process. But
>>>>>>> if moved to "persistent ephemeral" I guess same may not be the case.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Please let me know what you think of the same.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Also I would like to know what are the best scenario where one
>>>>>>> should prefer using persistent ephemeral node over ephemeral node.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Vikrant
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>
Re: Persistent Ephemeral Node
Posted by Vikrant Singh <vi...@gmail.com>.
Oh yes, I can do that. I can fix the path.
I create these node using default session time outs in curator framework..
I use ExponentialBackoffRetry with retry time out of 1s and retry count of
29.
So when I get a remove event for a PEM node... how long I should wait to
make sure that its client has given up on tries to rewrite it?
On Wed, Oct 14, 2015 at 3:51 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> Store your PEMs in a known path. Or, store a known value in the node.
>
> -JZ
>
>
> On Oct 14, 2015, at 5:49 PM, Vikrant Singh <vi...@gmail.com>
> wrote:
>
> What I want is to differentiate between "ephemeral nodes" and "persistent
> ephemeral node".
>
> On Wed, Oct 14, 2015 at 3:20 PM, Cameron McKenzie <mc...@gmail.com>
> wrote:
>
>> You could keep track of which nodes are ephemeral by looking at the stat
>> object passed to the NODE_ADDED event in the TreeCache. Then when you get a
>> NODE_REMOVED event you could check if it's one of your ephemeral nodes?
>>
>> On Thu, Oct 15, 2015 at 4:06 AM, Vikrant Singh <
>> vikrant.subscribe@gmail.com> wrote:
>>
>>> yes you got it correct... a slight correction. I am trying to find a way
>>> where I can make my tree cache event handler aware that "node removed"
>>> event is coming from a persistent ephemeral node and delay any action till
>>> the point the node's curator framework gives up on rewriting the node to ZK.
>>>
>>> On Mon, Oct 12, 2015 at 3:38 PM, Cameron McKenzie <
>>> mckenzie.cam@gmail.com> wrote:
>>>
>>>> So, if I'm understanding correctly, if your application loses its
>>>> connection to ZooKeeper (or crashes), then a new instance will be started
>>>> in its place, and if the original instance reconnects then you don't want
>>>> it to try and recreate its ephemeral node?
>>>>
>>>> If that's the case, then I think that you need additional logic beyond
>>>> what a PersistentEphemeralNode is going to provide you, as you will need to
>>>> check if the node already exists on reconnection.
>>>>
>>>> On Tue, Oct 13, 2015 at 8:43 AM, Vikrant Singh <
>>>> vikrant.subscribe@gmail.com> wrote:
>>>>
>>>>> yeah I considered that.. but I ended up using tree cache because it
>>>>> gives me more control on the state of tree . I have some custom handlers
>>>>> for add/remove/update events. Same cache is also used for service discovery.
>>>>>
>>>>> On Mon, Oct 12, 2015 at 2:33 PM, David Kesler <DK...@yodle.com>
>>>>> wrote:
>>>>>
>>>>>> There’s also already a Curator service discovery extension library
>>>>>> that you can look at:
>>>>>> http://curator.apache.org/curator-x-discovery/index.html. It’s
>>>>>> basically boiling down to the same strategy of sticking an ephemeral node
>>>>>> into ZK, but with some additional convenience and functionality built on
>>>>>> top of it.
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Cameron McKenzie [mailto:mckenzie.cam@gmail.com]
>>>>>> *Sent:* Monday, October 12, 2015 5:05 PM
>>>>>> *To:* user@curator.apache.org
>>>>>> *Subject:* Re: Persistent Ephemeral Node
>>>>>>
>>>>>>
>>>>>>
>>>>>> hey Vikrant,
>>>>>>
>>>>>> Using a persistent ephemeral node just means that your application
>>>>>> code doesn't need to worry about handling recreation of the node when it
>>>>>> reconnects to ZooKeeper after connection / session loss.
>>>>>>
>>>>>>
>>>>>>
>>>>>> If your ephemeral node should always be present whenever your
>>>>>> application instance is running, then this would be a good use case for a
>>>>>> persistent ephemeral node.
>>>>>>
>>>>>> cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <
>>>>>> vikrant.subscribe@gmail.com> wrote:
>>>>>>
>>>>>> I have some basic question on persistent ephemeral node.
>>>>>>
>>>>>> Here is some background...
>>>>>>
>>>>>>
>>>>>>
>>>>>> We have a zoo keeper based service discovery setup. Each service
>>>>>> register itself as a ephemeral node with zookeeper.When a service go down
>>>>>> its ephemeral node is removed from zookeeper and we know that service is
>>>>>> down and we provision it again.
>>>>>>
>>>>>> At present we create plain ephemeral node. I am wondering what
>>>>>> benefit/risks we may get if move to persistent ephemeral ones. I see one
>>>>>> problem... using plane ephemeral node we can rely on state of ZK to make a
>>>>>> decision like service is down. This is because we are sure that if a node
>>>>>> get deleted with zoo keeper it will never comeback from same process. But
>>>>>> if moved to "persistent ephemeral" I guess same may not be the case.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Please let me know what you think of the same.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Also I would like to know what are the best scenario where one should
>>>>>> prefer using persistent ephemeral node over ephemeral node.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Vikrant
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
>
Re: Persistent Ephemeral Node
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Store your PEMs in a known path. Or, store a known value in the node.
-JZ
> On Oct 14, 2015, at 5:49 PM, Vikrant Singh <vi...@gmail.com> wrote:
>
> What I want is to differentiate between "ephemeral nodes" and "persistent ephemeral node".
>
> On Wed, Oct 14, 2015 at 3:20 PM, Cameron McKenzie <mckenzie.cam@gmail.com <ma...@gmail.com>> wrote:
> You could keep track of which nodes are ephemeral by looking at the stat object passed to the NODE_ADDED event in the TreeCache. Then when you get a NODE_REMOVED event you could check if it's one of your ephemeral nodes?
>
> On Thu, Oct 15, 2015 at 4:06 AM, Vikrant Singh <vikrant.subscribe@gmail.com <ma...@gmail.com>> wrote:
> yes you got it correct... a slight correction. I am trying to find a way where I can make my tree cache event handler aware that "node removed" event is coming from a persistent ephemeral node and delay any action till the point the node's curator framework gives up on rewriting the node to ZK.
>
> On Mon, Oct 12, 2015 at 3:38 PM, Cameron McKenzie <mckenzie.cam@gmail.com <ma...@gmail.com>> wrote:
> So, if I'm understanding correctly, if your application loses its connection to ZooKeeper (or crashes), then a new instance will be started in its place, and if the original instance reconnects then you don't want it to try and recreate its ephemeral node?
>
> If that's the case, then I think that you need additional logic beyond what a PersistentEphemeralNode is going to provide you, as you will need to check if the node already exists on reconnection.
>
> On Tue, Oct 13, 2015 at 8:43 AM, Vikrant Singh <vikrant.subscribe@gmail.com <ma...@gmail.com>> wrote:
> yeah I considered that.. but I ended up using tree cache because it gives me more control on the state of tree . I have some custom handlers for add/remove/update events. Same cache is also used for service discovery.
>
> On Mon, Oct 12, 2015 at 2:33 PM, David Kesler <DKesler@yodle.com <ma...@yodle.com>> wrote:
> There’s also already a Curator service discovery extension library that you can look at: http://curator.apache.org/curator-x-discovery/index.html <http://curator.apache.org/curator-x-discovery/index.html>. It’s basically boiling down to the same strategy of sticking an ephemeral node into ZK, but with some additional convenience and functionality built on top of it.
>
>
>
> From: Cameron McKenzie [mailto:mckenzie.cam@gmail.com <ma...@gmail.com>]
> Sent: Monday, October 12, 2015 5:05 PM
> To: user@curator.apache.org <ma...@curator.apache.org>
> Subject: Re: Persistent Ephemeral Node
>
>
>
> hey Vikrant,
>
> Using a persistent ephemeral node just means that your application code doesn't need to worry about handling recreation of the node when it reconnects to ZooKeeper after connection / session loss.
>
>
>
> If your ephemeral node should always be present whenever your application instance is running, then this would be a good use case for a persistent ephemeral node.
>
> cheers
>
>
>
>
>
> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <vikrant.subscribe@gmail.com <ma...@gmail.com>> wrote:
>
> I have some basic question on persistent ephemeral node.
>
> Here is some background...
>
>
>
> We have a zoo keeper based service discovery setup. Each service register itself as a ephemeral node with zookeeper.When a service go down its ephemeral node is removed from zookeeper and we know that service is down and we provision it again.
>
> At present we create plain ephemeral node. I am wondering what benefit/risks we may get if move to persistent ephemeral ones. I see one problem... using plane ephemeral node we can rely on state of ZK to make a decision like service is down. This is because we are sure that if a node get deleted with zoo keeper it will never comeback from same process. But if moved to "persistent ephemeral" I guess same may not be the case.
>
>
>
> Please let me know what you think of the same.
>
>
>
> Also I would like to know what are the best scenario where one should prefer using persistent ephemeral node over ephemeral node.
>
>
>
> Thanks,
>
> Vikrant
>
>
>
>
>
>
>
>
Re: Persistent Ephemeral Node
Posted by Cameron McKenzie <mc...@gmail.com>.
You could store data in the persistent ephemeral nodes indicating that they
are persistent, and then track this in the NODE_ADDED event in the tree
cache. Then look it up when the NODE_REMOVED event occurs?
On Thu, Oct 15, 2015 at 9:49 AM, Vikrant Singh <vi...@gmail.com>
wrote:
> What I want is to differentiate between "ephemeral nodes" and "persistent
> ephemeral node".
>
> On Wed, Oct 14, 2015 at 3:20 PM, Cameron McKenzie <mc...@gmail.com>
> wrote:
>
>> You could keep track of which nodes are ephemeral by looking at the stat
>> object passed to the NODE_ADDED event in the TreeCache. Then when you get a
>> NODE_REMOVED event you could check if it's one of your ephemeral nodes?
>>
>> On Thu, Oct 15, 2015 at 4:06 AM, Vikrant Singh <
>> vikrant.subscribe@gmail.com> wrote:
>>
>>> yes you got it correct... a slight correction. I am trying to find a way
>>> where I can make my tree cache event handler aware that "node removed"
>>> event is coming from a persistent ephemeral node and delay any action till
>>> the point the node's curator framework gives up on rewriting the node to ZK.
>>>
>>> On Mon, Oct 12, 2015 at 3:38 PM, Cameron McKenzie <
>>> mckenzie.cam@gmail.com> wrote:
>>>
>>>> So, if I'm understanding correctly, if your application loses its
>>>> connection to ZooKeeper (or crashes), then a new instance will be started
>>>> in its place, and if the original instance reconnects then you don't want
>>>> it to try and recreate its ephemeral node?
>>>>
>>>> If that's the case, then I think that you need additional logic beyond
>>>> what a PersistentEphemeralNode is going to provide you, as you will need to
>>>> check if the node already exists on reconnection.
>>>>
>>>> On Tue, Oct 13, 2015 at 8:43 AM, Vikrant Singh <
>>>> vikrant.subscribe@gmail.com> wrote:
>>>>
>>>>> yeah I considered that.. but I ended up using tree cache because it
>>>>> gives me more control on the state of tree . I have some custom handlers
>>>>> for add/remove/update events. Same cache is also used for service discovery.
>>>>>
>>>>> On Mon, Oct 12, 2015 at 2:33 PM, David Kesler <DK...@yodle.com>
>>>>> wrote:
>>>>>
>>>>>> There’s also already a Curator service discovery extension library
>>>>>> that you can look at:
>>>>>> http://curator.apache.org/curator-x-discovery/index.html. It’s
>>>>>> basically boiling down to the same strategy of sticking an ephemeral node
>>>>>> into ZK, but with some additional convenience and functionality built on
>>>>>> top of it.
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Cameron McKenzie [mailto:mckenzie.cam@gmail.com]
>>>>>> *Sent:* Monday, October 12, 2015 5:05 PM
>>>>>> *To:* user@curator.apache.org
>>>>>> *Subject:* Re: Persistent Ephemeral Node
>>>>>>
>>>>>>
>>>>>>
>>>>>> hey Vikrant,
>>>>>>
>>>>>> Using a persistent ephemeral node just means that your application
>>>>>> code doesn't need to worry about handling recreation of the node when it
>>>>>> reconnects to ZooKeeper after connection / session loss.
>>>>>>
>>>>>>
>>>>>>
>>>>>> If your ephemeral node should always be present whenever your
>>>>>> application instance is running, then this would be a good use case for a
>>>>>> persistent ephemeral node.
>>>>>>
>>>>>> cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <
>>>>>> vikrant.subscribe@gmail.com> wrote:
>>>>>>
>>>>>> I have some basic question on persistent ephemeral node.
>>>>>>
>>>>>> Here is some background...
>>>>>>
>>>>>>
>>>>>>
>>>>>> We have a zoo keeper based service discovery setup. Each service
>>>>>> register itself as a ephemeral node with zookeeper.When a service go down
>>>>>> its ephemeral node is removed from zookeeper and we know that service is
>>>>>> down and we provision it again.
>>>>>>
>>>>>> At present we create plain ephemeral node. I am wondering what
>>>>>> benefit/risks we may get if move to persistent ephemeral ones. I see one
>>>>>> problem... using plane ephemeral node we can rely on state of ZK to make a
>>>>>> decision like service is down. This is because we are sure that if a node
>>>>>> get deleted with zoo keeper it will never comeback from same process. But
>>>>>> if moved to "persistent ephemeral" I guess same may not be the case.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Please let me know what you think of the same.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Also I would like to know what are the best scenario where one should
>>>>>> prefer using persistent ephemeral node over ephemeral node.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Vikrant
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Persistent Ephemeral Node
Posted by Vikrant Singh <vi...@gmail.com>.
What I want is to differentiate between "ephemeral nodes" and "persistent
ephemeral node".
On Wed, Oct 14, 2015 at 3:20 PM, Cameron McKenzie <mc...@gmail.com>
wrote:
> You could keep track of which nodes are ephemeral by looking at the stat
> object passed to the NODE_ADDED event in the TreeCache. Then when you get a
> NODE_REMOVED event you could check if it's one of your ephemeral nodes?
>
> On Thu, Oct 15, 2015 at 4:06 AM, Vikrant Singh <
> vikrant.subscribe@gmail.com> wrote:
>
>> yes you got it correct... a slight correction. I am trying to find a way
>> where I can make my tree cache event handler aware that "node removed"
>> event is coming from a persistent ephemeral node and delay any action till
>> the point the node's curator framework gives up on rewriting the node to ZK.
>>
>> On Mon, Oct 12, 2015 at 3:38 PM, Cameron McKenzie <mckenzie.cam@gmail.com
>> > wrote:
>>
>>> So, if I'm understanding correctly, if your application loses its
>>> connection to ZooKeeper (or crashes), then a new instance will be started
>>> in its place, and if the original instance reconnects then you don't want
>>> it to try and recreate its ephemeral node?
>>>
>>> If that's the case, then I think that you need additional logic beyond
>>> what a PersistentEphemeralNode is going to provide you, as you will need to
>>> check if the node already exists on reconnection.
>>>
>>> On Tue, Oct 13, 2015 at 8:43 AM, Vikrant Singh <
>>> vikrant.subscribe@gmail.com> wrote:
>>>
>>>> yeah I considered that.. but I ended up using tree cache because it
>>>> gives me more control on the state of tree . I have some custom handlers
>>>> for add/remove/update events. Same cache is also used for service discovery.
>>>>
>>>> On Mon, Oct 12, 2015 at 2:33 PM, David Kesler <DK...@yodle.com>
>>>> wrote:
>>>>
>>>>> There’s also already a Curator service discovery extension library
>>>>> that you can look at:
>>>>> http://curator.apache.org/curator-x-discovery/index.html. It’s
>>>>> basically boiling down to the same strategy of sticking an ephemeral node
>>>>> into ZK, but with some additional convenience and functionality built on
>>>>> top of it.
>>>>>
>>>>>
>>>>>
>>>>> *From:* Cameron McKenzie [mailto:mckenzie.cam@gmail.com]
>>>>> *Sent:* Monday, October 12, 2015 5:05 PM
>>>>> *To:* user@curator.apache.org
>>>>> *Subject:* Re: Persistent Ephemeral Node
>>>>>
>>>>>
>>>>>
>>>>> hey Vikrant,
>>>>>
>>>>> Using a persistent ephemeral node just means that your application
>>>>> code doesn't need to worry about handling recreation of the node when it
>>>>> reconnects to ZooKeeper after connection / session loss.
>>>>>
>>>>>
>>>>>
>>>>> If your ephemeral node should always be present whenever your
>>>>> application instance is running, then this would be a good use case for a
>>>>> persistent ephemeral node.
>>>>>
>>>>> cheers
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <
>>>>> vikrant.subscribe@gmail.com> wrote:
>>>>>
>>>>> I have some basic question on persistent ephemeral node.
>>>>>
>>>>> Here is some background...
>>>>>
>>>>>
>>>>>
>>>>> We have a zoo keeper based service discovery setup. Each service
>>>>> register itself as a ephemeral node with zookeeper.When a service go down
>>>>> its ephemeral node is removed from zookeeper and we know that service is
>>>>> down and we provision it again.
>>>>>
>>>>> At present we create plain ephemeral node. I am wondering what
>>>>> benefit/risks we may get if move to persistent ephemeral ones. I see one
>>>>> problem... using plane ephemeral node we can rely on state of ZK to make a
>>>>> decision like service is down. This is because we are sure that if a node
>>>>> get deleted with zoo keeper it will never comeback from same process. But
>>>>> if moved to "persistent ephemeral" I guess same may not be the case.
>>>>>
>>>>>
>>>>>
>>>>> Please let me know what you think of the same.
>>>>>
>>>>>
>>>>>
>>>>> Also I would like to know what are the best scenario where one should
>>>>> prefer using persistent ephemeral node over ephemeral node.
>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Vikrant
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Persistent Ephemeral Node
Posted by Cameron McKenzie <mc...@gmail.com>.
You could keep track of which nodes are ephemeral by looking at the stat
object passed to the NODE_ADDED event in the TreeCache. Then when you get a
NODE_REMOVED event you could check if it's one of your ephemeral nodes?
On Thu, Oct 15, 2015 at 4:06 AM, Vikrant Singh <vi...@gmail.com>
wrote:
> yes you got it correct... a slight correction. I am trying to find a way
> where I can make my tree cache event handler aware that "node removed"
> event is coming from a persistent ephemeral node and delay any action till
> the point the node's curator framework gives up on rewriting the node to ZK.
>
> On Mon, Oct 12, 2015 at 3:38 PM, Cameron McKenzie <mc...@gmail.com>
> wrote:
>
>> So, if I'm understanding correctly, if your application loses its
>> connection to ZooKeeper (or crashes), then a new instance will be started
>> in its place, and if the original instance reconnects then you don't want
>> it to try and recreate its ephemeral node?
>>
>> If that's the case, then I think that you need additional logic beyond
>> what a PersistentEphemeralNode is going to provide you, as you will need to
>> check if the node already exists on reconnection.
>>
>> On Tue, Oct 13, 2015 at 8:43 AM, Vikrant Singh <
>> vikrant.subscribe@gmail.com> wrote:
>>
>>> yeah I considered that.. but I ended up using tree cache because it
>>> gives me more control on the state of tree . I have some custom handlers
>>> for add/remove/update events. Same cache is also used for service discovery.
>>>
>>> On Mon, Oct 12, 2015 at 2:33 PM, David Kesler <DK...@yodle.com> wrote:
>>>
>>>> There’s also already a Curator service discovery extension library that
>>>> you can look at:
>>>> http://curator.apache.org/curator-x-discovery/index.html. It’s
>>>> basically boiling down to the same strategy of sticking an ephemeral node
>>>> into ZK, but with some additional convenience and functionality built on
>>>> top of it.
>>>>
>>>>
>>>>
>>>> *From:* Cameron McKenzie [mailto:mckenzie.cam@gmail.com]
>>>> *Sent:* Monday, October 12, 2015 5:05 PM
>>>> *To:* user@curator.apache.org
>>>> *Subject:* Re: Persistent Ephemeral Node
>>>>
>>>>
>>>>
>>>> hey Vikrant,
>>>>
>>>> Using a persistent ephemeral node just means that your application code
>>>> doesn't need to worry about handling recreation of the node when it
>>>> reconnects to ZooKeeper after connection / session loss.
>>>>
>>>>
>>>>
>>>> If your ephemeral node should always be present whenever your
>>>> application instance is running, then this would be a good use case for a
>>>> persistent ephemeral node.
>>>>
>>>> cheers
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <
>>>> vikrant.subscribe@gmail.com> wrote:
>>>>
>>>> I have some basic question on persistent ephemeral node.
>>>>
>>>> Here is some background...
>>>>
>>>>
>>>>
>>>> We have a zoo keeper based service discovery setup. Each service
>>>> register itself as a ephemeral node with zookeeper.When a service go down
>>>> its ephemeral node is removed from zookeeper and we know that service is
>>>> down and we provision it again.
>>>>
>>>> At present we create plain ephemeral node. I am wondering what
>>>> benefit/risks we may get if move to persistent ephemeral ones. I see one
>>>> problem... using plane ephemeral node we can rely on state of ZK to make a
>>>> decision like service is down. This is because we are sure that if a node
>>>> get deleted with zoo keeper it will never comeback from same process. But
>>>> if moved to "persistent ephemeral" I guess same may not be the case.
>>>>
>>>>
>>>>
>>>> Please let me know what you think of the same.
>>>>
>>>>
>>>>
>>>> Also I would like to know what are the best scenario where one should
>>>> prefer using persistent ephemeral node over ephemeral node.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Vikrant
>>>>
>>>>
>>>>
>>>
>>>
>>
>
Re: Persistent Ephemeral Node
Posted by Vikrant Singh <vi...@gmail.com>.
yes you got it correct... a slight correction. I am trying to find a way
where I can make my tree cache event handler aware that "node removed"
event is coming from a persistent ephemeral node and delay any action till
the point the node's curator framework gives up on rewriting the node to ZK.
On Mon, Oct 12, 2015 at 3:38 PM, Cameron McKenzie <mc...@gmail.com>
wrote:
> So, if I'm understanding correctly, if your application loses its
> connection to ZooKeeper (or crashes), then a new instance will be started
> in its place, and if the original instance reconnects then you don't want
> it to try and recreate its ephemeral node?
>
> If that's the case, then I think that you need additional logic beyond
> what a PersistentEphemeralNode is going to provide you, as you will need to
> check if the node already exists on reconnection.
>
> On Tue, Oct 13, 2015 at 8:43 AM, Vikrant Singh <
> vikrant.subscribe@gmail.com> wrote:
>
>> yeah I considered that.. but I ended up using tree cache because it gives
>> me more control on the state of tree . I have some custom handlers for
>> add/remove/update events. Same cache is also used for service discovery.
>>
>> On Mon, Oct 12, 2015 at 2:33 PM, David Kesler <DK...@yodle.com> wrote:
>>
>>> There’s also already a Curator service discovery extension library that
>>> you can look at:
>>> http://curator.apache.org/curator-x-discovery/index.html. It’s
>>> basically boiling down to the same strategy of sticking an ephemeral node
>>> into ZK, but with some additional convenience and functionality built on
>>> top of it.
>>>
>>>
>>>
>>> *From:* Cameron McKenzie [mailto:mckenzie.cam@gmail.com]
>>> *Sent:* Monday, October 12, 2015 5:05 PM
>>> *To:* user@curator.apache.org
>>> *Subject:* Re: Persistent Ephemeral Node
>>>
>>>
>>>
>>> hey Vikrant,
>>>
>>> Using a persistent ephemeral node just means that your application code
>>> doesn't need to worry about handling recreation of the node when it
>>> reconnects to ZooKeeper after connection / session loss.
>>>
>>>
>>>
>>> If your ephemeral node should always be present whenever your
>>> application instance is running, then this would be a good use case for a
>>> persistent ephemeral node.
>>>
>>> cheers
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <
>>> vikrant.subscribe@gmail.com> wrote:
>>>
>>> I have some basic question on persistent ephemeral node.
>>>
>>> Here is some background...
>>>
>>>
>>>
>>> We have a zoo keeper based service discovery setup. Each service
>>> register itself as a ephemeral node with zookeeper.When a service go down
>>> its ephemeral node is removed from zookeeper and we know that service is
>>> down and we provision it again.
>>>
>>> At present we create plain ephemeral node. I am wondering what
>>> benefit/risks we may get if move to persistent ephemeral ones. I see one
>>> problem... using plane ephemeral node we can rely on state of ZK to make a
>>> decision like service is down. This is because we are sure that if a node
>>> get deleted with zoo keeper it will never comeback from same process. But
>>> if moved to "persistent ephemeral" I guess same may not be the case.
>>>
>>>
>>>
>>> Please let me know what you think of the same.
>>>
>>>
>>>
>>> Also I would like to know what are the best scenario where one should
>>> prefer using persistent ephemeral node over ephemeral node.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Vikrant
>>>
>>>
>>>
>>
>>
>
Re: Persistent Ephemeral Node
Posted by Cameron McKenzie <mc...@gmail.com>.
So, if I'm understanding correctly, if your application loses its
connection to ZooKeeper (or crashes), then a new instance will be started
in its place, and if the original instance reconnects then you don't want
it to try and recreate its ephemeral node?
If that's the case, then I think that you need additional logic beyond what
a PersistentEphemeralNode is going to provide you, as you will need to
check if the node already exists on reconnection.
On Tue, Oct 13, 2015 at 8:43 AM, Vikrant Singh <vi...@gmail.com>
wrote:
> yeah I considered that.. but I ended up using tree cache because it gives
> me more control on the state of tree . I have some custom handlers for
> add/remove/update events. Same cache is also used for service discovery.
>
> On Mon, Oct 12, 2015 at 2:33 PM, David Kesler <DK...@yodle.com> wrote:
>
>> There’s also already a Curator service discovery extension library that
>> you can look at:
>> http://curator.apache.org/curator-x-discovery/index.html. It’s
>> basically boiling down to the same strategy of sticking an ephemeral node
>> into ZK, but with some additional convenience and functionality built on
>> top of it.
>>
>>
>>
>> *From:* Cameron McKenzie [mailto:mckenzie.cam@gmail.com]
>> *Sent:* Monday, October 12, 2015 5:05 PM
>> *To:* user@curator.apache.org
>> *Subject:* Re: Persistent Ephemeral Node
>>
>>
>>
>> hey Vikrant,
>>
>> Using a persistent ephemeral node just means that your application code
>> doesn't need to worry about handling recreation of the node when it
>> reconnects to ZooKeeper after connection / session loss.
>>
>>
>>
>> If your ephemeral node should always be present whenever your application
>> instance is running, then this would be a good use case for a persistent
>> ephemeral node.
>>
>> cheers
>>
>>
>>
>>
>>
>> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <
>> vikrant.subscribe@gmail.com> wrote:
>>
>> I have some basic question on persistent ephemeral node.
>>
>> Here is some background...
>>
>>
>>
>> We have a zoo keeper based service discovery setup. Each service register
>> itself as a ephemeral node with zookeeper.When a service go down its
>> ephemeral node is removed from zookeeper and we know that service is down
>> and we provision it again.
>>
>> At present we create plain ephemeral node. I am wondering what
>> benefit/risks we may get if move to persistent ephemeral ones. I see one
>> problem... using plane ephemeral node we can rely on state of ZK to make a
>> decision like service is down. This is because we are sure that if a node
>> get deleted with zoo keeper it will never comeback from same process. But
>> if moved to "persistent ephemeral" I guess same may not be the case.
>>
>>
>>
>> Please let me know what you think of the same.
>>
>>
>>
>> Also I would like to know what are the best scenario where one should
>> prefer using persistent ephemeral node over ephemeral node.
>>
>>
>>
>> Thanks,
>>
>> Vikrant
>>
>>
>>
>
>
Re: Persistent Ephemeral Node
Posted by Vikrant Singh <vi...@gmail.com>.
yeah I considered that.. but I ended up using tree cache because it gives
me more control on the state of tree . I have some custom handlers for
add/remove/update events. Same cache is also used for service discovery.
On Mon, Oct 12, 2015 at 2:33 PM, David Kesler <DK...@yodle.com> wrote:
> There’s also already a Curator service discovery extension library that
> you can look at: http://curator.apache.org/curator-x-discovery/index.html.
> It’s basically boiling down to the same strategy of sticking an ephemeral
> node into ZK, but with some additional convenience and functionality built
> on top of it.
>
>
>
> *From:* Cameron McKenzie [mailto:mckenzie.cam@gmail.com]
> *Sent:* Monday, October 12, 2015 5:05 PM
> *To:* user@curator.apache.org
> *Subject:* Re: Persistent Ephemeral Node
>
>
>
> hey Vikrant,
>
> Using a persistent ephemeral node just means that your application code
> doesn't need to worry about handling recreation of the node when it
> reconnects to ZooKeeper after connection / session loss.
>
>
>
> If your ephemeral node should always be present whenever your application
> instance is running, then this would be a good use case for a persistent
> ephemeral node.
>
> cheers
>
>
>
>
>
> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <
> vikrant.subscribe@gmail.com> wrote:
>
> I have some basic question on persistent ephemeral node.
>
> Here is some background...
>
>
>
> We have a zoo keeper based service discovery setup. Each service register
> itself as a ephemeral node with zookeeper.When a service go down its
> ephemeral node is removed from zookeeper and we know that service is down
> and we provision it again.
>
> At present we create plain ephemeral node. I am wondering what
> benefit/risks we may get if move to persistent ephemeral ones. I see one
> problem... using plane ephemeral node we can rely on state of ZK to make a
> decision like service is down. This is because we are sure that if a node
> get deleted with zoo keeper it will never comeback from same process. But
> if moved to "persistent ephemeral" I guess same may not be the case.
>
>
>
> Please let me know what you think of the same.
>
>
>
> Also I would like to know what are the best scenario where one should
> prefer using persistent ephemeral node over ephemeral node.
>
>
>
> Thanks,
>
> Vikrant
>
>
>
RE: Persistent Ephemeral Node
Posted by David Kesler <DK...@yodle.com>.
There’s also already a Curator service discovery extension library that you can look at: http://curator.apache.org/curator-x-discovery/index.html. It’s basically boiling down to the same strategy of sticking an ephemeral node into ZK, but with some additional convenience and functionality built on top of it.
From: Cameron McKenzie [mailto:mckenzie.cam@gmail.com]
Sent: Monday, October 12, 2015 5:05 PM
To: user@curator.apache.org
Subject: Re: Persistent Ephemeral Node
hey Vikrant,
Using a persistent ephemeral node just means that your application code doesn't need to worry about handling recreation of the node when it reconnects to ZooKeeper after connection / session loss.
If your ephemeral node should always be present whenever your application instance is running, then this would be a good use case for a persistent ephemeral node.
cheers
On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <vi...@gmail.com>> wrote:
I have some basic question on persistent ephemeral node.
Here is some background...
We have a zoo keeper based service discovery setup. Each service register itself as a ephemeral node with zookeeper.When a service go down its ephemeral node is removed from zookeeper and we know that service is down and we provision it again.
At present we create plain ephemeral node. I am wondering what benefit/risks we may get if move to persistent ephemeral ones. I see one problem... using plane ephemeral node we can rely on state of ZK to make a decision like service is down. This is because we are sure that if a node get deleted with zoo keeper it will never comeback from same process. But if moved to "persistent ephemeral" I guess same may not be the case.
Please let me know what you think of the same.
Also I would like to know what are the best scenario where one should prefer using persistent ephemeral node over ephemeral node.
Thanks,
Vikrant
Re: Persistent Ephemeral Node
Posted by Cameron McKenzie <mc...@gmail.com>.
hey Vikrant,
Using a persistent ephemeral node just means that your application code
doesn't need to worry about handling recreation of the node when it
reconnects to ZooKeeper after connection / session loss.
If your ephemeral node should always be present whenever your application
instance is running, then this would be a good use case for a persistent
ephemeral node.
cheers
On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh <vi...@gmail.com>
wrote:
> I have some basic question on persistent ephemeral node.
>
> Here is some background...
>
> We have a zoo keeper based service discovery setup. Each service register
> itself as a ephemeral node with zookeeper.When a service go down its
> ephemeral node is removed from zookeeper and we know that service is down
> and we provision it again.
>
> At present we create plain ephemeral node. I am wondering what
> benefit/risks we may get if move to persistent ephemeral ones. I see one
> problem... using plane ephemeral node we can rely on state of ZK to make a
> decision like service is down. This is because we are sure that if a node
> get deleted with zoo keeper it will never comeback from same process. But
> if moved to "persistent ephemeral" I guess same may not be the case.
>
> Please let me know what you think of the same.
>
> Also I would like to know what are the best scenario where one should
> prefer using persistent ephemeral node over ephemeral node.
>
> Thanks,
> Vikrant
>