You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Antoine Pitrou (Jira)" <ji...@apache.org> on 2020/09/21 18:53:00 UTC

[jira] [Created] (ARROW-10058) [C++] Investigate performance of LevelsToBitmap without BMI2

Antoine Pitrou created ARROW-10058:
--------------------------------------

             Summary: [C++] Investigate performance of LevelsToBitmap without BMI2
                 Key: ARROW-10058
                 URL: https://issues.apache.org/jira/browse/ARROW-10058
             Project: Apache Arrow
          Issue Type: Sub-task
          Components: C++
            Reporter: Antoine Pitrou


Currently, when some Parquet nested data involves some repetition levels, converting the levels to bitmap goes through a slow scalar path unless the BMI2 instruction set is available and efficient (the latter using the PEXT instruction to process 16 levels at once).

It may be possible to emulate PEXT for 5- or 6-bit masks by using a lookup table, allowing to process 5-6 levels at once.

(also, it would be good to add nested reading benchmarks for non-trivial nesting; currently we only benchmark one-level struct and one-level list)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)