You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@curator.apache.org by Raji Muthupandian <ra...@gmail.com> on 2014/02/07 22:35:38 UTC

Curator Framework Close() is NOT always shutting down ConnectionStateManager threads

Hi Team,
     We have 5 server Zookeeper ensemble. If any one of the server goes
down, connections are balanced to remaining servers. When the down server
comes back, there will not be any connections to this server unless new
clients are connected or connectionLoss happens at client side. This makes
a unequal connection distribution to the servers.

To avoid this, we thought of having a connection balancer task which closes
the existing connection and creates new connection on every configured time
interval. As part of connection close, we call Curatorframework.close().
        But in some instances , ConnectionStateManager threads are not
closed. Threads are still in WAITING state to take events from eventqueue.

"ConnectionStateManager-0" - Thread t@272
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for <3be45251> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
    at
com.netflix.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:170)
    at
com.netflix.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:40)
    at
com.netflix.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:104)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:724

This creates lot of dangling threads over a period of time. Is there a way
to cleanly close the connections ?

is there any better approach to handle connection balancing....

Thanks
Raji

Re: Curator Framework Close() is NOT always shutting down ConnectionStateManager threads

Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.

Close is supposed to shutdown everything. What version are you using? Can you create a test case that exhibits the problem?

====================
Jordan Zimmerman

> On Feb 7, 2014, at 4:35 PM, Raji Muthupandian <ra...@gmail.com> wrote:
> 
> Hi Team,
>      We have 5 server Zookeeper ensemble. If any one of the server goes down, connections are balanced to remaining servers. When the down server comes back, there will not be any connections to this server unless new clients are connected or connectionLoss happens at client side. This makes a unequal connection distribution to the servers.
> 
> To avoid this, we thought of having a connection balancer task which closes the existing connection and creates new connection on every configured time interval. As part of connection close, we call Curatorframework.close().
>         But in some instances , ConnectionStateManager threads are not closed. Threads are still in WAITING state to take events from eventqueue. 
> 
> "ConnectionStateManager-0" - Thread t@272
>    java.lang.Thread.State: WAITING
>     at sun.misc.Unsafe.park(Native Method)
>     - parking to wait for <3be45251> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>     at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>     at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
>     at com.netflix.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:170)
>     at com.netflix.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:40)
>     at com.netflix.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:104)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>     at java.lang.Thread.run(Thread.java:724
> 
> This creates lot of dangling threads over a period of time. Is there a way to cleanly close the connections ?
> 
> is there any better approach to handle connection balancing....
> 
> Thanks
> Raji