You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "Micah Kornfield (Jira)" <ji...@apache.org> on 2020/04/17 04:53:00 UTC
[jira] [Comment Edited] (PARQUET-1841) [C++] Experiment to see if
using SIMD shuffle operations for DecodeSpaced improves performance
[ https://issues.apache.org/jira/browse/PARQUET-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17085431#comment-17085431 ]
Micah Kornfield edited comment on PARQUET-1841 at 4/17/20, 4:52 AM:
--------------------------------------------------------------------
For AVX512 enabled processor the mask_expand like a good approach when available. I thought doing something clever with _mm_shuffle_* or mm_permute might be possible. Note that the values that values corresponding to nulls don't need to be zeroed out they just need to be assigned.
@wesm @pitrou do you know if we have any ability to run integration tests on hosts supporting AVX512?
was (Author: emkornfield@gmail.com):
For AVX512 enabled processor the mask_expand like a good approach when available. I thought doing something clever with _mm_shuffle_* might be possible. Note that the values that values corresponding to nulls don't need to be zeroed out they just need to be assigned.
@wesm @pitrou do you know if we have any ability to run integration tests on hosts supporting AVX512?
> [C++] Experiment to see if using SIMD shuffle operations for DecodeSpaced improves performance
> ----------------------------------------------------------------------------------------------
>
> Key: PARQUET-1841
> URL: https://issues.apache.org/jira/browse/PARQUET-1841
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-cpp
> Reporter: Micah Kornfield
> Assignee: Micah Kornfield
> Priority: Major
> Attachments: image-2020-04-14-15-01-48-222.png
>
>
> Followup from PARQUET-1840 for current benchmarks it seems that doing removing the memset somehow either has no impact or is slightly worse. We should investigate using SIMD operations to speed up spacing.
>
> As part of this we can see if moving the memset to only cover uninitialized values after moving all required values provides any speedup.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)