You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Antoine Pitrou (Jira)" <ji...@apache.org> on 2020/10/15 17:01:00 UTC

[jira] [Updated] (ARROW-8845) [C++] Selective compression on the wire

     [ https://issues.apache.org/jira/browse/ARROW-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Antoine Pitrou updated ARROW-8845:
----------------------------------
    Summary: [C++] Selective compression on the wire  (was: [c++] Selective compression on the wire)

> [C++] Selective compression on the wire
> ---------------------------------------
>
>                 Key: ARROW-8845
>                 URL: https://issues.apache.org/jira/browse/ARROW-8845
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: FlightRPC
>            Reporter: Amol Umbarkar
>            Priority: Major
>
> Dask seems to be selectively do compression if it is found to be useful. They sort of pick 10kb of sample upfront to calculate compression and if the results are good then the whole batch is compressed. This seems to save de-compression effort on receiver side.
>  
> Please take a look at [https://blog.dask.org/2016/04/14/dask-distributed-optimizing-protocol#problem-3-unwanted-compression]
>  
> Thought this could be relevant to arrow batch transfers as well. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)