[jira] [Created] (SPARK-9491) App running on secure YARN with no HBase config will hang

["Marcelo Vanzin (JIRA)" <ji...@apache.org>]

Marcelo Vanzin created SPARK-9491:
-------------------------------------

             Summary: App running on secure YARN with no HBase config will hang
                 Key: SPARK-9491
                 URL: https://issues.apache.org/jira/browse/SPARK-9491
             Project: Spark
          Issue Type: Bug
          Components: YARN
    Affects Versions: 1.5.0
            Reporter: Marcelo Vanzin
            Priority: Blocker


Because HBase may not be available, or the default config may be pointing at the wrong information for HBase, the YARN backend may end up waiting forever at this point:

{noformat}
"main" prio=10 tid=0x00007f96c8016000 nid=0x1aa6 waiting on condition [0x00007f96cda96000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:443)
        at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:60)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1123)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1110)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1067)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:902)
        at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:78)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
        at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95)
        at org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callBlockingMethod(CoprocessorRpcChannel.java:73)
        at org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$BlockingStub.getAuthenticationToken(AuthenticationProtos.java:4512)
        at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:86)
        at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:69)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.deploy.yarn.Client$.obtainTokenForHBase(Client.scala:1299)
        at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:270)
{noformat}

The code shouldn't try to fetch HBase delegation tokens when HBase is not configured.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org