You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by me...@gigaspaces.com, me...@gigaspaces.com on 2018/05/16 10:26:39 UTC

IllegalStateException in performBackgroundOperation during close

Hi, I ran to a similar issue as described in CURATOR-126

version :
[org.apache.curator.utils.Compatibility] - Running in ZooKeeper 3.4.x compatibility mode
[org.apache.zookeeper.ZooKeeper] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT

This is a 3xZK cluster, where each process holds an embedded ZooKeeper and a client.
The client is executing the Leader Election recipe.

The exception below happens when I close one of the processes. First I close the leader selector and then close the client. The leader selector has autoRequeue enabled, but even when disabled, this exception occurs.

-----------------------
2018-05-15 17:43:40,514 Close leader selector and unregister from leader election
2018-05-15 17:43:40,935 Released leadership semaphore [sessionId=0x16364e4bf6c0001]
2018-05-15 17:43:40,940  'onLeadershipLost' [sessionId=0x16364e4bf6c0001]
2018-05-15 17:43:40,942  Close ZooKeeper client
2018-05-15 17:43:40,984 SEVERE [org.apache.curator.framework.recipes.leader.LeaderSelector] - The leader threw an exception; Caused by: java.lang.IllegalStateException: instance must be started before calling this method
        at org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:444)
        at org.apache.curator.framework.imps.CuratorFrameworkImpl.delete(CuratorFrameworkImpl.java:424)
        at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath(LockInternals.java:347)
        at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(LockInternals.java:124)
        at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:154)
        at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:449)
        at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:466)
        at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:65)
        at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:246)
        at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:240)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)

2018-05-15 17:43:40,984 [org.apache.curator.framework.imps.CuratorFrameworkImpl] - backgroundOperationsLoop exiting
2018-05-15 17:43:41,016 [org.apache.zookeeper.ZooKeeper] - Session: 0x16364e4bf6c0001 closed
-----------------------

Has anyone came across this? is there some other sequence to close and avoid this exception appearing in the logs?

Re: IllegalStateException in performBackgroundOperation during close

Posted by Andor Molnar <an...@cloudera.com>.
Hi Meron,

I've seen this issue intermittently when testing the leader election recipe
in Sentry. Unfortunately I'm not sure about how to fix this or close the
handles properly, however it seemed to me that it didn't cause any issue.

To be honest it's a Curator question anyway, so I think you'll get better
answers on Curator's users mailing list.

Regards,
Andor



On Wed, May 16, 2018 at 12:26 PM, meron.avigdor@gigaspaces.com <
meron.avigdor@gigaspaces.com> wrote:

> Hi, I ran to a similar issue as described in CURATOR-126
>
> version :
> [org.apache.curator.utils.Compatibility] - Running in ZooKeeper 3.4.x
> compatibility mode
> [org.apache.zookeeper.ZooKeeper] - Client environment:zookeeper.version=
> 3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017
> 10:13 GMT
>
> This is a 3xZK cluster, where each process holds an embedded ZooKeeper and
> a client.
> The client is executing the Leader Election recipe.
>
> The exception below happens when I close one of the processes. First I
> close the leader selector and then close the client. The leader selector
> has autoRequeue enabled, but even when disabled, this exception occurs.
>
> -----------------------
> 2018-05-15 17:43:40,514 Close leader selector and unregister from leader
> election
> 2018-05-15 17:43:40,935 Released leadership semaphore
> [sessionId=0x16364e4bf6c0001]
> 2018-05-15 17:43:40,940  'onLeadershipLost' [sessionId=0x16364e4bf6c0001]
> 2018-05-15 17:43:40,942  Close ZooKeeper client
> 2018-05-15 17:43:40,984 SEVERE [org.apache.curator.framework.recipes.leader.LeaderSelector]
> - The leader threw an exception; Caused by: java.lang.IllegalStateException:
> instance must be started before calling this method
>         at org.apache.curator.shaded.com.google.common.base.
> Preconditions.checkState(Preconditions.java:444)
>         at org.apache.curator.framework.imps.CuratorFrameworkImpl.
> delete(CuratorFrameworkImpl.java:424)
>         at org.apache.curator.framework.recipes.locks.LockInternals.
> deleteOurPath(LockInternals.java:347)
>         at org.apache.curator.framework.recipes.locks.LockInternals.
> releaseLock(LockInternals.java:124)
>         at org.apache.curator.framework.recipes.locks.
> InterProcessMutex.release(InterProcessMutex.java:154)
>         at org.apache.curator.framework.recipes.leader.LeaderSelector.
> doWork(LeaderSelector.java:449)
>         at org.apache.curator.framework.recipes.leader.LeaderSelector.
> doWorkLoop(LeaderSelector.java:466)
>         at org.apache.curator.framework.recipes.leader.LeaderSelector.
> access$100(LeaderSelector.java:65)
>         at org.apache.curator.framework.recipes.leader.LeaderSelector$
> 2.call(LeaderSelector.java:246)
>         at org.apache.curator.framework.recipes.leader.LeaderSelector$
> 2.call(LeaderSelector.java:240)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:748)
>
> 2018-05-15 17:43:40,984 [org.apache.curator.framework.imps.CuratorFrameworkImpl]
> - backgroundOperationsLoop exiting
> 2018-05-15 17:43:41,016 [org.apache.zookeeper.ZooKeeper] - Session:
> 0x16364e4bf6c0001 closed
> -----------------------
>
> Has anyone came across this? is there some other sequence to close and
> avoid this exception appearing in the logs?
>