[jira] [Comment Edited] (ARROW-1055) [C++] GPU support library development

["Siu Kwan Lam (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129015#comment-16129015 ] 

Siu Kwan Lam edited comment on ARROW-1055 at 8/16/17 5:52 PM:
--------------------------------------------------------------

For starter, it will be great if there is a arrow serializer and deserializer that minimize host<->device transfer.  Initially, arrow needs to handle device memory pointer only and it doesn't need to deal with device IPC handles.   The schema is usually small that it is okay to be processed on the host side.  For the recordbatches, it is best to keep the payload on the device.  

EDIT: 
Adding a reference to gpu arrow deserializer implementation in libgdf: https://github.com/gpuopenanalytics/libgdf/blob/master/src/ipc.cu


was (Author: sklam):
For starter, it will be great if there is a arrow serializer and deserializer that minimize host<->device transfer.  Initially, arrow needs to handle device memory pointer only and it doesn't need to deal with device IPC handles.   The schema is usually small that it is okay to be processed on the host side.  For the recordbatches, it is best to keep the payload on the device.  

> [C++] GPU support library development
> -------------------------------------
>
>                 Key: ARROW-1055
>                 URL: https://issues.apache.org/jira/browse/ARROW-1055
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: C++
>            Reporter: Wes McKinney
>
> Umbrella issue to cover GPU buffer memory management, IPC loading, mixed-device schema/record batch management, etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)