[jira] [Created] (GOBBLIN-1205) restarting gobblin on yarn fails with error

["Jay Sen (Jira)" <ji...@apache.org>]

Jay Sen created GOBBLIN-1205:
--------------------------------

             Summary: restarting gobblin on yarn fails with error
                 Key: GOBBLIN-1205
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1205
             Project: Apache Gobblin
          Issue Type: Bug
    Affects Versions: 0.15.0
            Reporter: Jay Sen
             Fix For: 0.15.0


restarting gobblin deployed on yarn mode occasionally fails starting up with following error, may be the path is still on hold by the previous process, it may just need bit time between stop/start.
{code:java}
WARN [ZKHelixAdmin] Root directory exists.Cleaning the root directory:/GobblinYarnHelixAppWARN [ZKHelixAdmin] Root directory exists.Cleaning the root directory:/GobblinYarnHelixAppWARN [ZkClient] Failed to delete path /GobblinYarnHelixApp/CONTROLLER! org.I0Itec.zkclient.exception.ZkException: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLERERROR [ZkClient] Failed to delete /GobblinYarnHelixApp/CONTROLLERorg.I0Itec.zkclient.exception.ZkException: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLER at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:68) at org.apache.helix.manager.zk.zookeeper.ZkClient.retryUntilConnected(ZkClient.java:1160) at org.apache.helix.manager.zk.zookeeper.ZkClient.delete(ZkClient.java:1215) at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:949) at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:942) at org.apache.helix.manager.zk.ZKHelixAdmin.addCluster(ZKHelixAdmin.java:698) at org.apache.helix.tools.ClusterSetup.addCluster(ClusterSetup.java:162) at org.apache.gobblin.cluster.HelixUtils.createGobblinHelixCluster(HelixUtils.java:96) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.launch(GobblinYarnAppLauncher.java:337) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.main(GobblinYarnAppLauncher.java:1067)Caused by: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLER at org.apache.zookeeper.KeeperException.create(KeeperException.java:128) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:882) at org.apache.helix.manager.zk.zookeeper.ZkConnection.delete(ZkConnection.java:119) at org.apache.helix.manager.zk.zookeeper.ZkClient$9.call(ZkClient.java:1219) at org.apache.helix.manager.zk.zookeeper.ZkClient.retryUntilConnected(ZkClient.java:1150) ... 8 more
==> logs/yarn.err <==Exception in thread "main" org.apache.helix.HelixException: Failed to delete /GobblinYarnHelixApp/CONTROLLER at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:952) at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:942) at org.apache.helix.manager.zk.ZKHelixAdmin.addCluster(ZKHelixAdmin.java:698) at org.apache.helix.tools.ClusterSetup.addCluster(ClusterSetup.java:162) at org.apache.gobblin.cluster.HelixUtils.createGobblinHelixCluster(HelixUtils.java:96) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.launch(GobblinYarnAppLauncher.java:337) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.main(GobblinYarnAppLauncher.java:1067)Caused by: org.I0Itec.zkclient.exception.ZkException: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLER at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:68) at org.apache.helix.manager.zk.zookeeper.ZkClient.retryUntilConnected(ZkClient.java:1160) at org.apache.helix.manager.zk.zookeeper.ZkClient.delete(ZkClient.java:1215) at org.apache.helix.manager.zk.zookeeper.ZkClient.deleteRecursively(ZkClient.java:949) ... 6 moreCaused by: org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /GobblinYarnHelixApp/CONTROLLER at org.apache.zookeeper.KeeperException.create(KeeperException.java:128) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:882) at org.apache.helix.manager.zk.zookeeper.ZkConnection.delete(ZkConnection.java:119) at org.apache.helix.manager.zk.zookeeper.ZkClient$9.call(ZkClient.java:1219) at org.apache.helix.manager.zk.zookeeper.ZkClient.retryUntilConnected(ZkClient.java:1150) ... 8 moreException in thread "Thread-6" org.apache.helix.HelixException: HelixManager (ZkClient) is not connected. Call HelixManager#connect() at org.apache.helix.manager.zk.ZKHelixManager.checkConnected(ZKHelixManager.java:363) at org.apache.helix.manager.zk.ZKHelixManager.getClusterManagmentTool(ZKHelixManager.java:908) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.disableLiveHelixInstances(GobblinYarnAppLauncher.java:533) at org.apache.gobblin.yarn.GobblinYarnAppLauncher.stop(GobblinYarnAppLauncher.java:436) at org.apache.gobblin.yarn.GobblinYarnAppLauncher$2.run(GobblinYarnAppLauncher.java:1054)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)