You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Yibo Cai (Jira)" <ji...@apache.org> on 2022/01/07 14:01:00 UTC

[jira] [Updated] (ARROW-15282) [C++][FlightRPC] Support non-grpc data planes

     [ https://issues.apache.org/jira/browse/ARROW-15282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yibo Cai updated ARROW-15282:
-----------------------------
    Description: 
The goal is to decouple FlightData payload transmission from gRPC and provide a convenient framework to support other accelerated data planes (ucx, dpdk, xdp, etc.).

As a PoC, I implemented a shared memory data plane for local rpc, the performance is great compared with current best (grpc over unix socket).

Tested on xeon gold 5218.
||streams||grpc over unix socket||shared memory data plane||
|1|3324 GB/s|7045 GB/s|
|2|6289 GB/s|13311 GB/s|
|4|10037 GB/s|25012 GB/s|

  was:
The goal is to decouple FlightData payload transmission from gRPC and provide a convenient framework to support other accelerated data planes (ucx, dpdk, xdp, etc.).

As a PoC, I implemented a shared memory data plane for local rpc, the performance is great compared with current best (grpc over unix socket).
||streams||grpc over unix socket||shared memory data plane||
|1|3324 GB/s|7045 GB/s|
|2|6289 GB/s|13311 GB/s|
|4|10037 GB/s|25012 GB/s|


> [C++][FlightRPC] Support non-grpc data planes
> ---------------------------------------------
>
>                 Key: ARROW-15282
>                 URL: https://issues.apache.org/jira/browse/ARROW-15282
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: C++, FlightRPC
>            Reporter: Yibo Cai
>            Assignee: Yibo Cai
>            Priority: Major
>
> The goal is to decouple FlightData payload transmission from gRPC and provide a convenient framework to support other accelerated data planes (ucx, dpdk, xdp, etc.).
> As a PoC, I implemented a shared memory data plane for local rpc, the performance is great compared with current best (grpc over unix socket).
> Tested on xeon gold 5218.
> ||streams||grpc over unix socket||shared memory data plane||
> |1|3324 GB/s|7045 GB/s|
> |2|6289 GB/s|13311 GB/s|
> |4|10037 GB/s|25012 GB/s|



--
This message was sent by Atlassian Jira
(v8.20.1#820001)