[jira] [Updated] (GOBBLIN-1205) restarting gobblin on yarn fails with error

["Abhishek Tiwari (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GOBBLIN-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Abhishek Tiwari updated GOBBLIN-1205:
-------------------------------------
    Fix Version/s:     (was: 0.15.0)
                   0.16.0

> restarting gobblin on yarn fails with error
> -------------------------------------------
>
>                 Key: GOBBLIN-1205
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1205
>             Project: Apache Gobblin
>          Issue Type: Bug
>    Affects Versions: 0.15.0
>            Reporter: Jay Sen
>            Priority: Major
>             Fix For: 0.16.0
>
>
> restarting gobblin deployed on yarn mode occasionally fails starting up with following error, may be the path is still on hold by the previous process, it may just need bit time between stop/start.
> {code:java}
> WARN [ZKHelixAdmin] Root directory exists.Cleaning the root directory:/GobblinYarnHelixAppWARN [ZKHelixAdmin] Root directory exists.Cleaning the root directory:/GobblinYarnHelixAppWARN [ZkClient] Failed to delete path /GobblinYarnHelixApp/CONTROLLER! org.I0Itec.zkclient.exception.ZkException: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLERERROR [ZkClient] Failed to delete /GobblinYarnHelixApp/CONTROLLERorg.I0Itec.zkclient.exception.ZkException: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLER at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:68) at org.apache.helix.manager.zk.zookeeper.ZkClient.retryUntilConnected(ZkClient.java:1160) at org.apache.helix.manager.zk.zookeeper.ZkClient.delete(ZkClient.java:1215) at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:949) at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:942) at org.apache.helix.manager.zk.ZKHelixAdmin.addCluster(ZKHelixAdmin.java:698) at org.apache.helix.tools.ClusterSetup.addCluster(ClusterSetup.java:162) at org.apache.gobblin.cluster.HelixUtils.createGobblinHelixCluster(HelixUtils.java:96) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.launch(GobblinYarnAppLauncher.java:337) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.main(GobblinYarnAppLauncher.java:1067)Caused by: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLER at org.apache.zookeeper.KeeperException.create(KeeperException.java:128) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:882) at org.apache.helix.manager.zk.zookeeper.ZkConnection.delete(ZkConnection.java:119) at org.apache.helix.manager.zk.zookeeper.ZkClient$9.call(ZkClient.java:1219) at org.apache.helix.manager.zk.zookeeper.ZkClient.retryUntilConnected(ZkClient.java:1150) ... 8 more
> ==> logs/yarn.err <==Exception in thread "main" org.apache.helix.HelixException: Failed to delete /GobblinYarnHelixApp/CONTROLLER at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:952) at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:942) at org.apache.helix.manager.zk.ZKHelixAdmin.addCluster(ZKHelixAdmin.java:698) at org.apache.helix.tools.ClusterSetup.addCluster(ClusterSetup.java:162) at org.apache.gobblin.cluster.HelixUtils.createGobblinHelixCluster(HelixUtils.java:96) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.launch(GobblinYarnAppLauncher.java:337) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.main(GobblinYarnAppLauncher.java:1067)Caused by: org.I0Itec.zkclient.exception.ZkException: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLER at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:68) at org.apache.helix.manager.zk.zookeeper.ZkClient.retryUntilConnected(ZkClient.java:1160) at org.apache.helix.manager.zk.zookeeper.ZkClient.delete(ZkClient.java:1215) at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:949) ... 6 moreCaused by: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLER at org.apache.zookeeper.KeeperException.create(KeeperException.java:128) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:882) at org.apache.helix.manager.zk.zookeeper.ZkConnection.delete(ZkConnection.java:119) at org.apache.helix.manager.zk.zookeeper.ZkClient$9.call(ZkClient.java:1219) at org.apache.helix.manager.zk.zookeeper.ZkClient.retryUntilConnected(ZkClient.java:1150) ... 8 moreException in thread "Thread-6" org.apache.helix.HelixException: HelixManager (ZkClient) is not connected. Call HelixManager#connect() at org.apache.helix.manager.zk.ZKHelixManager.checkConnected(ZKHelixManager.java:363) at org.apache.helix.manager.zk.ZKHelixManager.getClusterManagmentTool(ZKHelixManager.java:908) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.disableLiveHelixInstances(GobblinYarnAppLauncher.java:533) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.stop(GobblinYarnAppLauncher.java:436) at org.apache.gobblin.yarn.GobblinYarnAppLauncher$2.run(GobblinYarnAppLauncher.java:1054)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)