[jira] [Commented] (HADOOP-10667) implement TCP connection reuse for native client

["Abraham Elmahrek (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042963#comment-14042963 ] 

Abraham Elmahrek commented on HADOOP-10667:
-------------------------------------------

# What is 'conns_created' for? Future logging purposes? Analytics?
{code}
diff --git a/hadoop-native-core/src/main/native/rpc/reactor.h b/hadoop-native-core/src/main/native/rpc/reactor.h
index c8a38e6..29f12b8 100644
--- a/hadoop-native-core/src/main/native/rpc/reactor.h
+++ b/hadoop-native-core/src/main/native/rpc/reactor.h
@@ -72,6 +72,11 @@ struct hrpc_reactor_inbox {
  */
 struct hrpc_reactor {
     /**
+     * Number of connections we've created.
+     */
+    uint64_t conns_created;
+
+    /**
      * The inbox for incoming work for this reactor thread.
      */
     struct hrpc_reactor_inbox inbox;
{code}
# I noticed that proxies give 'calls' 'socketaddr_in' structs. The proxy struct seems to be allocated on the stack in several places. ie in ndfc.c. Seems a bit dangerous unless we know for sure call structs will be deallocated before proxies?
{code}
diff --git a/hadoop-native-core/src/main/native/rpc/proxy.c b/hadoop-native-core/src/main/native/rpc/proxy.c
index 8ea04cc..18d33d7 100644
--- a/hadoop-native-core/src/main/native/rpc/proxy.c
+++ b/hadoop-native-core/src/main/native/rpc/proxy.c
@@ -59,7 +59,7 @@ void hrpc_proxy_init(struct hrpc_proxy *proxy,
     proxy->msgr = msgr;
     proxy->protocol = protocol;
     proxy->username = username;
-    proxy->call.remote = *remote;
+    proxy->remote = *remote;
 }
 
 struct hadoop_err *hrpc_proxy_activate(struct hrpc_proxy *proxy)
@@ -129,6 +129,7 @@ void hrpc_proxy_start(struct hrpc_proxy *proxy,
     uint8_t *buf;
     struct hrpc_call *call = &proxy->call;
 
+    call->remote = &proxy->remote;
     call->cb = cb;
     call->cb_data = cb_data;
     call->protocol = proxy->protocol;
{code}

In general +1 with the exception of the questions above!

> implement TCP connection reuse for native client
> ------------------------------------------------
>
>                 Key: HADOOP-10667
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10667
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: native
>    Affects Versions: HADOOP-10388
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HADOOP-10667-pnative.001.patch, HADOOP-10667-pnative.002.patch, HADOOP-10667-pnative.003.patch
>
>
> The HDFS / YARN native clients should re-use TCP connections to avoid the overhead of the three-way handshake, similar to how the Java code does.



--
This message was sent by Atlassian JIRA
(v6.2#6252)