You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org> on 2012/09/21 10:51:08 UTC
[jira] [Commented] (HDFS-3965) DFSInputStream should not eat up
exceptions if file is under construction
[ https://issues.apache.org/jira/browse/HDFS-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460364#comment-13460364 ]
Amareshwari Sriramadasu commented on HDFS-3965:
-----------------------------------------------
We saw a scenario where datanode is unresponsive and the client silently exits with following log:
{noformat}
21 Sep 2012 00:00:00,050 [IPC Client (47) connection to /a.b.c.d:50020
from blk_5626968076878217612_215416] DEBUG org.apache.hadoop.ipc.Client -
closing ipc connection to /a.b.c.d:50020: 60000 millis timeout while
waiting for channel to be ready for read. ch :
java.nio.channels.SocketChannel[connected local=/a.b.c.d:55558
remote=/a.b.c.d:50020]
java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel
to be ready for read. ch : java.nio.channels.SocketChannel[connected
local=/a.b.c.d:55558 remote=/a.b.c.d:50020]
at
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at
org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:375)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at java.io.DataInputStream.readInt(DataInputStream.java:370)
at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:815)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:724)
21 Sep 2012 00:00:00,055 DEBUG
org.apache.hadoop.hdfs.DFSClient - DFSClient file
/myfile
is being concurrently append to but datanode a.b.c.d probably does not
have block blk_5626968076878217612_215416
{noformat}
Instead these exceptions should be thrown out so that the read fails and users can retry later
> DFSInputStream should not eat up exceptions if file is under construction
> -------------------------------------------------------------------------
>
> Key: HDFS-3965
> URL: https://issues.apache.org/jira/browse/HDFS-3965
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 1.0.3
> Reporter: Amareshwari Sriramadasu
>
> Currently DFSclient eats up exceptions if it could not open the file under construction:
> {noformat}
> 1881 try {
> 1882 primary = createClientDatanodeProtocolProxy(primaryNode, conf,
> 1883 last.getBlock(), last.getBlockToken(), socketTimeout);
> 1884 Block newBlock = primary.getBlockInfo(last.getBlock());
> 1885 long newBlockSize = newBlock.getNumBytes();
> 1886 long delta = newBlockSize - last.getBlockSize();
> 1887 // if the size of the block on the datanode is different
> 1888 // from what the NN knows about, the datanode wins!
> 1889 last.getBlock().setNumBytes(newBlockSize);
> 1890 long newlength = newInfo.getFileLength() + delta;
> 1891 newInfo.setFileLength(newlength);
> 1892 LOG.debug("DFSClient setting last block " + last + " to length "
> 1893 + newBlockSize + " filesize is now " + newInfo.getFileLength());
> 1894 } catch (IOException e) {
> 1895 if (e.getMessage().startsWith(
> 1896 "java.io.IOException: java.lang.NoSuchMethodException: "
> 1897 + "org.apache.hadoop.hdfs.protocol"
> 1898 + ".ClientDatanodeProtocol.getBlockInfo")) {
> 1899 // We're talking to a server that doesn't implement HDFS-200.
> 1900 serverSupportsHdfs200 = false;
> 1901 } else {
> 1902 LOG.debug("DFSClient file " + src
> 1903 + " is being concurrently append to" + " but datanode "
> 1904 + primaryNode.getHostName() + " probably does not have block "
> 1905 + last.getBlock());
> 1906 }
> 1907 }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira