You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@curator.apache.org by Daniel Kashtan <dj...@gmail.com> on 2015/08/31 21:06:28 UTC
NodeExistsException with Shared Value recipe in Curator
I inherited some Curator code that creates a Shared Value recipe in two
different parts of my code, using the same path. Once the second Shared
Value has .start() called, the NodeExistsException from the Zookeeper
server is created in the logs.
I am not sure how to handle this issue. Can I retrieve the Shared Value
instance from Curator instead of trying to create a second new instance of
the Shared Value on the same path?
--
-Daniel
Re: NodeExistsException with Shared Value recipe in Curator
Posted by Daniel Kashtan <dj...@gmail.com>.
I understand.
I believe that despite my "checkExists()" calls, I am seeing one case where
I still see an unexpected NodeExistsException.
On Tue, Sep 1, 2015 at 2:14 PM, Gregory Chanan <gc...@cloudera.com> wrote:
> Just to reiterate Jordan's point -- given ZooKeeper's server semantics you
> cannot avoid NodeExistsExceptions in general.
>
> If ZooKeeper had different semantics -- for example state-based APIs
> (e.g."this node should exist at the end of this call") rather than
> action-based APIs (e.g. "create") -- it would be possible to avoid these
> exceptions. Or if you could enforce single-writer at a time semantics.
>
> But given ZooKeeper's API choices, I believe most adminstrators simply
> live with the exceptions or filter them out.
>
> Greg
>
> On Tue, Sep 1, 2015 at 7:19 AM, Daniel Kashtan <dj...@gmail.com>
> wrote:
>
>> I can definitely work around this problem by configuring my log4j file to
>> WARN for the admins and then have a separate file with a verbose view set
>> to INFO.
>>
>> What would be ideal is to have an option to call the .start() method with
>> a checkExists(). I am already doing using checkExists() in other parts of
>> my code, but if I am using the Shared Value recipe I believe I cannot avoid
>> these exceptions being thrown in the server (please correct me if I am
>> wrong though).
>>
>> Personally I can tolerate the NodeExists exceptions in the logs, but when
>> administrators are running the software in production and see exceptions in
>> the logs they think something is broken.
>>
>> On Mon, Aug 31, 2015 at 7:49 PM, Jordan Zimmerman <
>> jordan@jordanzimmerman.com> wrote:
>>
>>> Why would this be needed? It adds an additional ZK call for no good
>>> reason. If the node exists, the exception is thrown. In any event, even if
>>> checkExists() succeeds there’s a chance that another process can create the
>>> node before the create() call is made. So, it doesn’t do what you want
>>> anyway.
>>>
>>> -Jordan
>>>
>>> On August 31, 2015 at 2:46:09 PM, Cameron McKenzie (
>>> mckenzie.cam@gmail.com) wrote:
>>>
>>> Would it be possible in the future to have Curator have the option to
>>> run a checkExists() before the .start() call on a recipe like Shared-Value?
>>> Is the extra overhead not worth the trouble?
>>>
>>>
>>
>>
>> --
>> -Daniel
>>
>
>
--
-Daniel
Re: NodeExistsException with Shared Value recipe in Curator
Posted by Gregory Chanan <gc...@cloudera.com>.
Just to reiterate Jordan's point -- given ZooKeeper's server semantics you
cannot avoid NodeExistsExceptions in general.
If ZooKeeper had different semantics -- for example state-based APIs
(e.g."this node should exist at the end of this call") rather than
action-based APIs (e.g. "create") -- it would be possible to avoid these
exceptions. Or if you could enforce single-writer at a time semantics.
But given ZooKeeper's API choices, I believe most adminstrators simply live
with the exceptions or filter them out.
Greg
On Tue, Sep 1, 2015 at 7:19 AM, Daniel Kashtan <dj...@gmail.com> wrote:
> I can definitely work around this problem by configuring my log4j file to
> WARN for the admins and then have a separate file with a verbose view set
> to INFO.
>
> What would be ideal is to have an option to call the .start() method with
> a checkExists(). I am already doing using checkExists() in other parts of
> my code, but if I am using the Shared Value recipe I believe I cannot avoid
> these exceptions being thrown in the server (please correct me if I am
> wrong though).
>
> Personally I can tolerate the NodeExists exceptions in the logs, but when
> administrators are running the software in production and see exceptions in
> the logs they think something is broken.
>
> On Mon, Aug 31, 2015 at 7:49 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> Why would this be needed? It adds an additional ZK call for no good
>> reason. If the node exists, the exception is thrown. In any event, even if
>> checkExists() succeeds there’s a chance that another process can create the
>> node before the create() call is made. So, it doesn’t do what you want
>> anyway.
>>
>> -Jordan
>>
>> On August 31, 2015 at 2:46:09 PM, Cameron McKenzie (
>> mckenzie.cam@gmail.com) wrote:
>>
>> Would it be possible in the future to have Curator have the option to run
>> a checkExists() before the .start() call on a recipe like Shared-Value? Is
>> the extra overhead not worth the trouble?
>>
>>
>
>
> --
> -Daniel
>
Re: NodeExistsException with Shared Value recipe in Curator
Posted by Daniel Kashtan <dj...@gmail.com>.
I can definitely work around this problem by configuring my log4j file to
WARN for the admins and then have a separate file with a verbose view set
to INFO.
What would be ideal is to have an option to call the .start() method with a
checkExists(). I am already doing using checkExists() in other parts of my
code, but if I am using the Shared Value recipe I believe I cannot avoid
these exceptions being thrown in the server (please correct me if I am
wrong though).
Personally I can tolerate the NodeExists exceptions in the logs, but when
administrators are running the software in production and see exceptions in
the logs they think something is broken.
On Mon, Aug 31, 2015 at 7:49 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> Why would this be needed? It adds an additional ZK call for no good
> reason. If the node exists, the exception is thrown. In any event, even if
> checkExists() succeeds there’s a chance that another process can create the
> node before the create() call is made. So, it doesn’t do what you want
> anyway.
>
> -Jordan
>
> On August 31, 2015 at 2:46:09 PM, Cameron McKenzie (mckenzie.cam@gmail.com)
> wrote:
>
> Would it be possible in the future to have Curator have the option to run
> a checkExists() before the .start() call on a recipe like Shared-Value? Is
> the extra overhead not worth the trouble?
>
>
--
-Daniel
Re: NodeExistsException with Shared Value recipe in Curator
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Why would this be needed? It adds an additional ZK call for no good reason. If the node exists, the exception is thrown. In any event, even if checkExists() succeeds there’s a chance that another process can create the node before the create() call is made. So, it doesn’t do what you want anyway.
-Jordan
On August 31, 2015 at 2:46:09 PM, Cameron McKenzie (mckenzie.cam@gmail.com) wrote:
Would it be possible in the future to have Curator have the option to run a checkExists() before the .start() call on a recipe like Shared-Value? Is the extra overhead not worth the trouble?
Re: NodeExistsException with Shared Value recipe in Curator
Posted by Cameron McKenzie <mc...@gmail.com>.
I'll let the other devs weigh in also, but I would prefer not to put in a
checkExists() call. While it minimises the likelihood of a
NodeExistsException occurring, it's not a guarantee. Ultimately, I think
that this should be fixed in Zookeeper itself.
Have you tried just changing the log4j configuration on the ZK server to
change the logging level for the particular appender that's logging the
INFO message? You shouldn't need to go down the path of logging to one file
and then filtering that to another file.
On Tue, Sep 1, 2015 at 7:39 AM, Daniel Kashtan <dj...@gmail.com> wrote:
> I believe that is my only other option. I think I will try to route the
> full logs to a separate file with the word verbose in it and filter out the
> PrepRequestProcessor class on the ZKServer in my zookeeper.log file.
>
> Would it be possible in the future to have Curator have the option to run
> a checkExists() before the .start() call on a recipe like Shared-Value? Is
> the extra overhead not worth the trouble?
>
> On Mon, Aug 31, 2015 at 5:28 PM, Cameron McKenzie <mc...@gmail.com>
> wrote:
>
>> hey Daniel,
>> I don't think that Curator can help in this regard. As you have
>> suggested, the logs are generated on the server side by Zookeeper itself.
>> Not sure if you could do something with the logging on the server. Maybe
>> set that particular log4j appender to only log WARN and above?
>>
>> On Tue, Sep 1, 2015 at 5:43 AM, Daniel Kashtan <dj...@gmail.com>
>> wrote:
>>
>>> I am running with the debugger in eclipse and I see the try/catch block,
>>> but it does not stop zookeeper from printing in the logs that there was a
>>> NodeExistsException. I believe that the NodeExistsException is being
>>> printed by the Zookeeper Server? I have all my logs routed to one file.
>>>
>>> The message looks like this:
>>> 2015-08-31 15:27:46,740 [myid:] - INFO [ProcessThread(sid:0
>>> cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException
>>> when processing sessionid:0x14f84fd269c0003 type:create cxid:0xa4
>>> zxid:0x62d txntype:-1 reqpath:n/a Error
>>> Path:/results/response-5932e7bc-2fe6-4745-8e36-6ed49d6f31a3
>>> Error:KeeperErrorCode = NodeExists for
>>> /results/response-5932e7bc-2fe6-4745-8e36-6ed49d6f31a3
>>>
>>> I had the same issue earlier with two different parts of my code and I
>>> could not solve this issue with try/catch blocks. I had to do a
>>> checkExists() before creating a new node.
>>>
>>> BTW I am using Curator 2.8.0
>>>
>>> On Mon, Aug 31, 2015 at 3:08 PM, Jordan Zimmerman <
>>> jordan@jordanzimmerman.com> wrote:
>>>
>>>> Can you write a test that shows this? It doesn’t seem possible given
>>>> your description. Have a look at SharedValue.start(). Note that it catches
>>>> NodeExistsException.
>>>>
>>>> -JZ
>>>>
>>>>
>>>>
>>>> On August 31, 2015 at 12:06:30 PM, Daniel Kashtan (djkashtan@gmail.com)
>>>> wrote:
>>>>
>>>> I inherited some Curator code that creates a Shared Value recipe in two
>>>> different parts of my code, using the same path. Once the second Shared
>>>> Value has .start() called, the NodeExistsException from the Zookeeper
>>>> server is created in the logs.
>>>>
>>>> I am not sure how to handle this issue. Can I retrieve the Shared Value
>>>> instance from Curator instead of trying to create a second new instance of
>>>> the Shared Value on the same path?
>>>>
>>>> --
>>>> -Daniel
>>>>
>>>>
>>>
>>>
>>> --
>>> -Daniel
>>>
>>
>>
>
>
> --
> -Daniel
>
Re: NodeExistsException with Shared Value recipe in Curator
Posted by Daniel Kashtan <dj...@gmail.com>.
I believe that is my only other option. I think I will try to route the
full logs to a separate file with the word verbose in it and filter out the
PrepRequestProcessor class on the ZKServer in my zookeeper.log file.
Would it be possible in the future to have Curator have the option to run a
checkExists() before the .start() call on a recipe like Shared-Value? Is
the extra overhead not worth the trouble?
On Mon, Aug 31, 2015 at 5:28 PM, Cameron McKenzie <mc...@gmail.com>
wrote:
> hey Daniel,
> I don't think that Curator can help in this regard. As you have suggested,
> the logs are generated on the server side by Zookeeper itself. Not sure if
> you could do something with the logging on the server. Maybe set that
> particular log4j appender to only log WARN and above?
>
> On Tue, Sep 1, 2015 at 5:43 AM, Daniel Kashtan <dj...@gmail.com>
> wrote:
>
>> I am running with the debugger in eclipse and I see the try/catch block,
>> but it does not stop zookeeper from printing in the logs that there was a
>> NodeExistsException. I believe that the NodeExistsException is being
>> printed by the Zookeeper Server? I have all my logs routed to one file.
>>
>> The message looks like this:
>> 2015-08-31 15:27:46,740 [myid:] - INFO [ProcessThread(sid:0
>> cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException
>> when processing sessionid:0x14f84fd269c0003 type:create cxid:0xa4
>> zxid:0x62d txntype:-1 reqpath:n/a Error
>> Path:/results/response-5932e7bc-2fe6-4745-8e36-6ed49d6f31a3
>> Error:KeeperErrorCode = NodeExists for
>> /results/response-5932e7bc-2fe6-4745-8e36-6ed49d6f31a3
>>
>> I had the same issue earlier with two different parts of my code and I
>> could not solve this issue with try/catch blocks. I had to do a
>> checkExists() before creating a new node.
>>
>> BTW I am using Curator 2.8.0
>>
>> On Mon, Aug 31, 2015 at 3:08 PM, Jordan Zimmerman <
>> jordan@jordanzimmerman.com> wrote:
>>
>>> Can you write a test that shows this? It doesn’t seem possible given
>>> your description. Have a look at SharedValue.start(). Note that it catches
>>> NodeExistsException.
>>>
>>> -JZ
>>>
>>>
>>>
>>> On August 31, 2015 at 12:06:30 PM, Daniel Kashtan (djkashtan@gmail.com)
>>> wrote:
>>>
>>> I inherited some Curator code that creates a Shared Value recipe in two
>>> different parts of my code, using the same path. Once the second Shared
>>> Value has .start() called, the NodeExistsException from the Zookeeper
>>> server is created in the logs.
>>>
>>> I am not sure how to handle this issue. Can I retrieve the Shared Value
>>> instance from Curator instead of trying to create a second new instance of
>>> the Shared Value on the same path?
>>>
>>> --
>>> -Daniel
>>>
>>>
>>
>>
>> --
>> -Daniel
>>
>
>
--
-Daniel
Re: NodeExistsException with Shared Value recipe in Curator
Posted by Cameron McKenzie <mc...@gmail.com>.
hey Daniel,
I don't think that Curator can help in this regard. As you have suggested,
the logs are generated on the server side by Zookeeper itself. Not sure if
you could do something with the logging on the server. Maybe set that
particular log4j appender to only log WARN and above?
On Tue, Sep 1, 2015 at 5:43 AM, Daniel Kashtan <dj...@gmail.com> wrote:
> I am running with the debugger in eclipse and I see the try/catch block,
> but it does not stop zookeeper from printing in the logs that there was a
> NodeExistsException. I believe that the NodeExistsException is being
> printed by the Zookeeper Server? I have all my logs routed to one file.
>
> The message looks like this:
> 2015-08-31 15:27:46,740 [myid:] - INFO [ProcessThread(sid:0
> cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException
> when processing sessionid:0x14f84fd269c0003 type:create cxid:0xa4
> zxid:0x62d txntype:-1 reqpath:n/a Error
> Path:/results/response-5932e7bc-2fe6-4745-8e36-6ed49d6f31a3
> Error:KeeperErrorCode = NodeExists for
> /results/response-5932e7bc-2fe6-4745-8e36-6ed49d6f31a3
>
> I had the same issue earlier with two different parts of my code and I
> could not solve this issue with try/catch blocks. I had to do a
> checkExists() before creating a new node.
>
> BTW I am using Curator 2.8.0
>
> On Mon, Aug 31, 2015 at 3:08 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> Can you write a test that shows this? It doesn’t seem possible given your
>> description. Have a look at SharedValue.start(). Note that it catches
>> NodeExistsException.
>>
>> -JZ
>>
>>
>>
>> On August 31, 2015 at 12:06:30 PM, Daniel Kashtan (djkashtan@gmail.com)
>> wrote:
>>
>> I inherited some Curator code that creates a Shared Value recipe in two
>> different parts of my code, using the same path. Once the second Shared
>> Value has .start() called, the NodeExistsException from the Zookeeper
>> server is created in the logs.
>>
>> I am not sure how to handle this issue. Can I retrieve the Shared Value
>> instance from Curator instead of trying to create a second new instance of
>> the Shared Value on the same path?
>>
>> --
>> -Daniel
>>
>>
>
>
> --
> -Daniel
>
Re: NodeExistsException with Shared Value recipe in Curator
Posted by Daniel Kashtan <dj...@gmail.com>.
I am running with the debugger in eclipse and I see the try/catch block,
but it does not stop zookeeper from printing in the logs that there was a
NodeExistsException. I believe that the NodeExistsException is being
printed by the Zookeeper Server? I have all my logs routed to one file.
The message looks like this:
2015-08-31 15:27:46,740 [myid:] - INFO [ProcessThread(sid:0
cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when
processing sessionid:0x14f84fd269c0003 type:create cxid:0xa4 zxid:0x62d
txntype:-1 reqpath:n/a Error
Path:/results/response-5932e7bc-2fe6-4745-8e36-6ed49d6f31a3
Error:KeeperErrorCode = NodeExists for
/results/response-5932e7bc-2fe6-4745-8e36-6ed49d6f31a3
I had the same issue earlier with two different parts of my code and I
could not solve this issue with try/catch blocks. I had to do a
checkExists() before creating a new node.
BTW I am using Curator 2.8.0
On Mon, Aug 31, 2015 at 3:08 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> Can you write a test that shows this? It doesn’t seem possible given your
> description. Have a look at SharedValue.start(). Note that it catches
> NodeExistsException.
>
> -JZ
>
>
>
> On August 31, 2015 at 12:06:30 PM, Daniel Kashtan (djkashtan@gmail.com)
> wrote:
>
> I inherited some Curator code that creates a Shared Value recipe in two
> different parts of my code, using the same path. Once the second Shared
> Value has .start() called, the NodeExistsException from the Zookeeper
> server is created in the logs.
>
> I am not sure how to handle this issue. Can I retrieve the Shared Value
> instance from Curator instead of trying to create a second new instance of
> the Shared Value on the same path?
>
> --
> -Daniel
>
>
--
-Daniel
Re: NodeExistsException with Shared Value recipe in Curator
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Can you write a test that shows this? It doesn’t seem possible given your description. Have a look at SharedValue.start(). Note that it catches NodeExistsException.
-JZ
On August 31, 2015 at 12:06:30 PM, Daniel Kashtan (djkashtan@gmail.com) wrote:
I inherited some Curator code that creates a Shared Value recipe in two different parts of my code, using the same path. Once the second Shared Value has .start() called, the NodeExistsException from the Zookeeper server is created in the logs.
I am not sure how to handle this issue. Can I retrieve the Shared Value instance from Curator instead of trying to create a second new instance of the Shared Value on the same path?
--
-Daniel