You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by Shameera Rathnayaka <sh...@apache.org> on 2015/05/28 17:58:33 UTC

Apache Curator Integration Summary

Hi Devs,

Following actions have been taken with Apache Curator Integration.

1. Remove Zookeeper dependency from Orchestrator module[1].
2. Use one CuratorFramework object everywhere in Gfac[2].
3. Removed unused utils methods related to Zookeeper.
4. Use latest Curator release (2.8.0)

TODO,
need to review zookeeper data structure.

[1]
https://github.com/apache/airavata/commit/915ed04d54c61e083ef0f2b7dccb7083ef3fee9e
[2]
https://github.com/apache/airavata/commit/4a06178022c0cd1a215c47e3c65a7b6ffd342bbb

Thanks,
Shameera.

Re: Apache Curator Integration Summary

Posted by Shameera Rathnayaka <sh...@gmail.com>.
Hi Suresh,


> I am curious what kind of connection issues have you seen lately (when we
> are directly using zookeeper client)?
>

If you have started to debug the gfac, then i am pretty sure you will see
zookeeper connection issues. Not only in debug mode, in normal mode also we
are getting this issue time to time.
​ See following logs, actual log file is too big to attach here.​ Apache
Curator will handle this connection issue and provide clean way to deal
with Zookeeper. They expose their api in Fluent Style way which is more
clear that using native Zookeeper client.


2015-05-22 11:55:46,146 [main-EventThread] INFO
 org.apache.airavata.gfac.server.GfacServerHandler  - Disconnected
2015-05-22 11:55:46,147 [main-EventThread] INFO
 org.apache.airavata.gfac.server.GfacServerHandler  - ZK Connection is
Disconnected
2015-05-22 11:55:46,188 [pool-8-thread-2] ERROR
org.apache.airavata.gfac.GFacException  - Cannot invoke OutHandlers
KeeperErrorCode = ConnectionLoss for
/gfac-experiments/gfac-node1/SLM1-US-Stampede2015-05-21_14-34-28_31011639-019b-4a2b-9ef3-e6365caa5f74/org.apache.airavata.gfac.ssh.handler.NewSSHOutputHandler
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for
/gfac-experiments/gfac-node1/SLM1-US-Stampede2015-05-21_14-34-28_31011639-019b-4a2b-9ef3-e6365caa5f74/org.apache.airavata.gfac.ssh.handler.NewSSHOutputHandler
        at
org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1003)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1031)
        at
org.apache.airavata.gfac.core.utils.GFacUtils.createHandlerZnode(GFacUtils.java:310)
        at
org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeOutFlowHandlers(BetterGfacImpl.java:969)
        at
org.apache.airavata.gfac.core.utils.OutHandlerWorker.run(OutHandlerWorker.java:66)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
2015-05-22 11:55:46,207 [pool-8-thread-2] ERROR
org.apache.airavata.gfac.GFacException  - Cannot invoke OutHandlers
Cannot invoke OutHandlers
KeeperErrorCode = ConnectionLoss for
/gfac-experiments/gfac-node1/SLM1-US-Stampede2015-05-21_14-34-28_31011639-019b-4a2b-9ef3-e6365caa5f74/org.apache.airavata.gfac.ssh.handler.NewSSHOutputHandler
org.apache.airavata.gfac.GFacException: Cannot invoke OutHandlers
KeeperErrorCode = ConnectionLoss for
/gfac-experiments/gfac-node1/SLM1-US-Stampede2015-05-21_14-34-28_31011639-019b-4a2b-9ef3-e6365caa5f74/org.apache.airavata.gfac.ssh.handler.NewSSHOutputHandler
        at
org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeOutFlowHandlers(BetterGfacImpl.java:1005)
        at
org.apache.airavata.gfac.core.utils.OutHandlerWorker.run(OutHandlerWorker.java:66)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for
/gfac-experiments/gfac-node1/SLM1-US-Stampede2015-05-21_14-34-28_31011639-019b-4a2b-9ef3-e6365caa5f74/org.apache.airavata.gfac.ssh.handler.NewSSHOutputHandler
        at
org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1003)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1031)
        at
org.apache.airavata.gfac.core.utils.GFacUtils.createHandlerZnode(GFacUtils.java:310)
        at
org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeOutFlowHandlers(BetterGfacImpl.java:969)
        ... 4 more


​Thanks,
Shameera.
​

>
> Suresh
>
> On May 28, 2015, at 11:58 AM, Shameera Rathnayaka <sh...@apache.org>
> wrote:
>
> Hi Devs,
>
> Following actions have been taken with Apache Curator Integration.
>
> 1. Remove Zookeeper dependency from Orchestrator module[1].
> 2. Use one CuratorFramework object everywhere in Gfac[2].
> 3. Removed unused utils methods related to Zookeeper.
> 4. Use latest Curator release (2.8.0)
>
> TODO,
> need to review zookeeper data structure.
>
> [1]
> https://github.com/apache/airavata/commit/915ed04d54c61e083ef0f2b7dccb7083ef3fee9e
> [2]
> https://github.com/apache/airavata/commit/4a06178022c0cd1a215c47e3c65a7b6ffd342bbb
>
> Thanks,
> Shameera.
>
>
>


-- 
Best Regards,
Shameera Rathnayaka.

email: shameera AT apache.org , shameerainfo AT gmail.com
Blog : http://shameerarathnayaka.blogspot.com/

Re: Apache Curator Integration Summary

Posted by Suresh Marru <sm...@apache.org>.
Hi Shameera,

I am not opposed to moving to curator, it indeed might help since as Jordan described on a different thread, curator will monitor connections and retry them without airavata not having to do the similar handling of connection failures. But my understanding is they are for more transient. 

I am curious what kind of connection issues have you seen lately (when we are directly using zookeeper client)? 

Suresh

> On May 28, 2015, at 11:58 AM, Shameera Rathnayaka <sh...@apache.org> wrote:
> 
> Hi Devs, 
> 
> Following actions have been taken with Apache Curator Integration.
> 
> 1. Remove Zookeeper dependency from Orchestrator module[1]. 
> 2. Use one CuratorFramework object everywhere in Gfac[2]. 
> 3. Removed unused utils methods related to Zookeeper.
> 4. Use latest Curator release (2.8.0)
> 
> TODO, 
> need to review zookeeper data structure. 
> 
> [1] https://github.com/apache/airavata/commit/915ed04d54c61e083ef0f2b7dccb7083ef3fee9e <https://github.com/apache/airavata/commit/915ed04d54c61e083ef0f2b7dccb7083ef3fee9e>
> [2] https://github.com/apache/airavata/commit/4a06178022c0cd1a215c47e3c65a7b6ffd342bbb <https://github.com/apache/airavata/commit/4a06178022c0cd1a215c47e3c65a7b6ffd342bbb>
> 
> Thanks, 
> Shameera.