[jira] [Updated] (KUDU-1448) Improve bitshuffle to enable AVX2 at runtime

["Todd Lipcon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated KUDU-1448:
------------------------------
    Attachment: Selection_029.png

Here's a screenshot of scan rates running 10 concurrent TPCH SF3000 Q6s on a 9-node cluster with bitshuffle and dictionary coding used. The drop in the middle is my restarting to a build with the AVX2-enabled bitshuffle, and the performance is about 30% better after the restart.

> Improve bitshuffle to enable AVX2 at runtime
> --------------------------------------------
>
>                 Key: KUDU-1448
>                 URL: https://issues.apache.org/jira/browse/KUDU-1448
>             Project: Kudu
>          Issue Type: Improvement
>          Components: cfile, perf
>    Affects Versions: 0.8.0
>            Reporter: Todd Lipcon
>         Attachments: Selection_029.png
>
>
> I've been doing a little testing with the BITSHUFFLE encoding, and found that scan performance is improved about 30% on a Haswell cluster by enabling AVX2 when building the bitshuffle library. Unfortunately, that would cause a crash on earlier boxes, so we can't ship like that.
> I filed https://github.com/kiyo-masui/bitshuffle/issues/41 to do the switching at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)