You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Jinglun (JIRA)" <ji...@apache.org> on 2018/11/26 13:21:00 UTC
[jira] [Commented] (HDFS-13888) RequestHedgingProxyProvider shows
InterruptedException
[ https://issues.apache.org/jira/browse/HDFS-13888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16698910#comment-16698910 ]
Jinglun commented on HDFS-13888:
--------------------------------
hi [~elgoiri], I take a look on the cause of the log, and here is the reason:
1. RequestHedgingProxyProvider starts one thread for each rpc call (in a thread pool);
2. One rpc call returns successfully and RequestHedgingProxyProvider calls shutDownNow(), which interrupts all threads in the thread pool;
3. The rpc thread being interrupted handles the interrupt in 3 ways:
3.1 If the rpc thread being interrupted happens to wait for the sending thread's return, it will log "interrupted waiting to send rpc request to server" with full trace.
3.2. If the rpc thread happens to wait for the response, then an InterruptedIOException("Call interrupted") will be throwed. As the thread doesn't catch any Exception, the InterruptedIOException will ends the thread without any log.
3.3. If the rpc thread has got the response, it will return the result and the thread ends.
3.1 is the reason of the full stack trace log, i will change the log to debug.
> RequestHedgingProxyProvider shows InterruptedException
> ------------------------------------------------------
>
> Key: HDFS-13888
> URL: https://issues.apache.org/jira/browse/HDFS-13888
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.1.1
> Reporter: Íñigo Goiri
> Priority: Minor
>
> RequestHedgingProxyProvider shows InterruptedException when running:
> {code}
> 2018-08-30 23:52:48,883 WARN ipc.Client: interrupted waiting to send rpc request to server
> java.lang.InterruptedException
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at org.apache.hadoop.ipc.Client$Connection.sendRpcRequest(Client.java:1142)
> at org.apache.hadoop.ipc.Client.call(Client.java:1395)
> at org.apache.hadoop.ipc.Client.call(Client.java:1353)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy11.getFileInfo(Unknown Source)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:900)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:135)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> It looks like this is the case of the background request that is killed once the main one succeeds. We should not log the full stack trace for this and maybe just a debug log.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org