You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Doug Cutting (JIRA)" <ji...@apache.org> on 2008/10/10 00:32:44 UTC

[jira] Commented: (HADOOP-4386) RPC support for large data transfers.

    [ https://issues.apache.org/jira/browse/HADOOP-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638416#action_12638416 ] 

Doug Cutting commented on HADOOP-4386:
--------------------------------------

I think there are a couple of things here.  Perhaps they should become sub-issues.
  - async RPC calls.  One should be able to invoke an RPC but not block waiting for the response, instead using a 'select'-style api to recieve responses asynchronously.
  - non-copied parameters and values.  One should be able to have results from an RPC read directly from the socket into a user-provided object (a parameter?) avoiding an extra copy.  On the server side, one should also be able to transfer data directly from a file to the socket.

> RPC support for large data transfers.
> -------------------------------------
>
>                 Key: HADOOP-4386
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4386
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs, ipc
>            Reporter: Raghu Angadi
>
> Currently HDFS has a socket level protocol for serving HDFS data to clients. Clients do not use RPCs to read or write data. Fundamentally there is no reason why this data transfer  can not use RPCs.
> This jira is place holder for any porting Datanode transfers to RPC. This topic has been discussed in varying detail many times, the latest being in the context of HADOOP-3856. There are quite a few issues to be resolved both at API level and at implementation level. 
> We should probably copy some of the comments from HADOOP-3856 to here.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.