You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@curator.apache.org by "Keith Turner (JIRA)" <ji...@apache.org> on 2018/06/01 22:15:00 UTC
[jira] [Updated] (CURATOR-468) Closing an active leader selector
always logs spurious error
[ https://issues.apache.org/jira/browse/CURATOR-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner updated CURATOR-468:
---------------------------------
Description:
Closing a a leader selector seems to always log spurious error messages. This only happens when it has assumed leader because of CURATOR-337. I think this spurious error logging highlights a conceptual problem with the code. The primary reason that [LeaderSelector.java line 449|https://github.com/apache/curator/blob/ee5d65463b73abb2a2b732fb02729239b183519d/curator-recipes/src/main/java/org/apache/curator/framework/recipes/leader/LeaderSelector.java#L449] is executed is because the thread was interrupted. Since the interrupt is set for the thread the call to release the mutex will never work, so its a bit of a catch-22.
Below is an example of the exception I am seeing.
{noformat}
2018-05-31 20:23:51,068 [leader.LeaderSelector] ERROR: The leader threw an exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:871)
at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:250)
at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:244)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109)
at org.apache.curator.framework.imps.DeleteBuilderImpl.pathInForeground(DeleteBuilderImpl.java:241)
at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:225)
at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:35)
at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath(LockInternals.java:339)
at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(LockInternals.java:123)
at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:154)
at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:449)
at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:466)
at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:65)
at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:246)
at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:240)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
Below is an example of an exception I see when closing a leader selector and then closing the curator framework.
{noformat}
2018-05-31 20:23:49,905 [leader.LeaderSelector] ERROR: The leader threw an exception
java.lang.IllegalStateException: instance must be started before calling this method
at org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:176)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.delete(CuratorFrameworkImpl.java:359)
at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath(LockInternals.java:339)
at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(LockInternals.java:123)
at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:154)
at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:449)
at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:466)
at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:65)
at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:246)
at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:240)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
was:
Closing a a leader selector seems to always log spurious error messages. This only happens when it has assumed leader because of CURATOR-337. I think this spurious error logging highlights a conceptual problem with the code. The primary reason that [LeaderSelector.java line 449|https://github.com/apache/curator/blob/ee5d65463b73abb2a2b732fb02729239b183519d/curator-recipes/src/main/java/org/apache/curator/framework/recipes/leader/LeaderSelector.java#L449] is executed is because the thread was interrupted. Since the interrupt is set for the thread the call to release the mutex will never work, so its a bit of a catch-22.
Below is an example of the exception I am seeing.
{{noformat}}
2018-05-31 20:23:51,068 [leader.LeaderSelector] ERROR: The leader threw an exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:871)
at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:250)
at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:244)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109)
at org.apache.curator.framework.imps.DeleteBuilderImpl.pathInForeground(DeleteBuilderImpl.java:241)
at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:225)
at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:35)
at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath(LockInternals.java:339)
at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(LockInternals.java:123)
at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:154)
at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:449)
at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:466)
at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:65)
at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:246)
at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:240)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{{noformat}}
Below is an example of an exception I see when closing a leader selector and then closing the curator framework.
{{noformat}}
2018-05-31 20:23:49,905 [leader.LeaderSelector] ERROR: The leader threw an exception
java.lang.IllegalStateException: instance must be started before calling this method
at org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:176)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.delete(CuratorFrameworkImpl.java:359)
at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath(LockInternals.java:339)
at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(LockInternals.java:123)
at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:154)
at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:449)
at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:466)
at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:65)
at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:246)
at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:240)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{{noformat}}
> Closing an active leader selector always logs spurious error
> ------------------------------------------------------------
>
> Key: CURATOR-468
> URL: https://issues.apache.org/jira/browse/CURATOR-468
> Project: Apache Curator
> Issue Type: Improvement
> Reporter: Keith Turner
> Assignee: Jordan Zimmerman
> Priority: Major
>
> Closing a a leader selector seems to always log spurious error messages. This only happens when it has assumed leader because of CURATOR-337. I think this spurious error logging highlights a conceptual problem with the code. The primary reason that [LeaderSelector.java line 449|https://github.com/apache/curator/blob/ee5d65463b73abb2a2b732fb02729239b183519d/curator-recipes/src/main/java/org/apache/curator/framework/recipes/leader/LeaderSelector.java#L449] is executed is because the thread was interrupted. Since the interrupt is set for the thread the call to release the mutex will never work, so its a bit of a catch-22.
> Below is an example of the exception I am seeing.
> {noformat}
> 2018-05-31 20:23:51,068 [leader.LeaderSelector] ERROR: The leader threw an exception
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:871)
> at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:250)
> at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:244)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109)
> at org.apache.curator.framework.imps.DeleteBuilderImpl.pathInForeground(DeleteBuilderImpl.java:241)
> at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:225)
> at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:35)
> at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath(LockInternals.java:339)
> at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(LockInternals.java:123)
> at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:154)
> at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:449)
> at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:466)
> at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:65)
> at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:246)
> at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:240)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> Below is an example of an exception I see when closing a leader selector and then closing the curator framework.
> {noformat}
> 2018-05-31 20:23:49,905 [leader.LeaderSelector] ERROR: The leader threw an exception
> java.lang.IllegalStateException: instance must be started before calling this method
> at org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:176)
> at org.apache.curator.framework.imps.CuratorFrameworkImpl.delete(CuratorFrameworkImpl.java:359)
> at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath(LockInternals.java:339)
> at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(LockInternals.java:123)
> at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:154)
> at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:449)
> at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:466)
> at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:65)
> at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:246)
> at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:240)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)