[jira] [Updated] (HDFS-7523) Setting a socket receive buffer size in DFSClient

["Liang Xie (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Liang Xie updated HDFS-7523:
----------------------------
    Attachment: HDFS-7523-001.txt

Attached a small patch. similiar with the writing stuff in DFSOutputStream:
{code}
   * Create a socket for a write pipeline
   * @param first the first datanode 
   * @param length the pipeline length
   * @param client client
   * @return the socket connected to the first datanode
   */
  static Socket createSocketForPipeline(final DatanodeInfo first,
      final int length, final DFSClient client) throws IOException {
    final String dnAddr = first.getXferAddr(
        client.getConf().connectToDnViaHostname);
    if (DFSClient.LOG.isDebugEnabled()) {
      DFSClient.LOG.debug("Connecting to datanode " + dnAddr);
    }
    final InetSocketAddress isa = NetUtils.createSocketAddr(dnAddr);
    final Socket sock = client.socketFactory.createSocket();
    final int timeout = client.getDatanodeReadTimeout(length);
    NetUtils.connect(sock, isa, client.getRandomLocalInterfaceAddr(), client.getConf().socketTimeout);
    sock.setSoTimeout(timeout);
    sock.setSendBufferSize(HdfsConstants.DEFAULT_DATA_SOCKET_SIZE);
{code}

> Setting a socket receive buffer size in DFSClient
> -------------------------------------------------
>
>                 Key: HDFS-7523
>                 URL: https://issues.apache.org/jira/browse/HDFS-7523
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: dfsclient
>    Affects Versions: 2.6.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>         Attachments: HDFS-7523-001.txt
>
>
> It would be nice if we have a socket receive buffer size while creating socket from client(HBase) view, in old version it should be in DFSInputStream, in trunk it seems should be at:
> {code}
>   @Override // RemotePeerFactory
>   public Peer newConnectedPeer(InetSocketAddress addr,
>       Token<BlockTokenIdentifier> blockToken, DatanodeID datanodeId)
>       throws IOException {
>     Peer peer = null;
>     boolean success = false;
>     Socket sock = null;
>     try {
>       sock = socketFactory.createSocket();
>       NetUtils.connect(sock, addr,
>         getRandomLocalInterfaceAddr(),
>         dfsClientConf.socketTimeout);
>       peer = TcpPeerServer.peerFromSocketAndKey(saslClient, sock, this,
>           blockToken, datanodeId);
>       peer.setReadTimeout(dfsClientConf.socketTimeout);
> {code}
> e.g: sock.setReceiveBufferSize(HdfsConstants.DEFAULT_DATA_SOCKET_SIZE);
> the default socket buffer size in Linux+JDK7 seems is 8k if i am not wrong, this value sometimes is small for HBase 64k block reading in a 10G network(at least, more system call)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)