[jira] [Updated] (HBASE-10432) Rpc retries non-recoverable error

["Nick Dimiduk (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-10432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nick Dimiduk updated HBASE-10432:
---------------------------------

    Attachment: HBASE-10432.02.patch

Rebased to trunk.

This patch corrects RpcRetryingCaller#translateException to wrap an Error in a DoNotRetryIOException, so perhaps the case in AsyncProcess#sendMultiAction is not relevant. [~nkeywal] any thoughts on a test for this?

> Rpc retries non-recoverable error
> ---------------------------------
>
>                 Key: HBASE-10432
>                 URL: https://issues.apache.org/jira/browse/HBASE-10432
>             Project: HBase
>          Issue Type: Bug
>          Components: IPC/RPC
>    Affects Versions: 0.98.0, 0.96.2, 0.99.0
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>            Priority: Minor
>         Attachments: HBASE-10432.00.patch, HBASE-10432.01.patch, HBASE-10432.02.patch, exception.txt
>
>
> I'm recently working with hbase/trunk + hive/trunk. I had a hive command eventually timeout with the following exception (stacktrace truncated).
> {noformat}
> Caused by: java.io.IOException: Could not set up IO Streams
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:922)
>         at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1536)
>         at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1425)
>         at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
>         at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
>         at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:28857)
>         at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:302)
>         at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:157)
>         at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:57)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
>         ... 43 more
> Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.net.NetUtils.getInputStream(Ljava/net/Socket;)Lorg/apache/hadoop/net/SocketInputWrapper;
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:861)
>         ... 52 more
> {noformat}
> The root cause looks like a dependency version missmatch (Hive compiled vs hadoop1, HBase vs hadoop2). However, we still retry this exception, even though it'll never actually complete. We should be more careful where we blindly catch Throwables.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)