You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/01/09 15:14:51 UTC

[GitHub] [arrow-rs] jhorstmann opened a new pull request #1146: Implement SIMD comparison operations for types with less than 4 lanes (i128)

jhorstmann opened a new pull request #1146:
URL: https://github.com/apache/arrow-rs/pull/1146


   # Which issue does this PR close?
   
   Implements comparison for simd types with less than 8 lanes.
   
   Closes #1136 .
   
   # What changes are included in this PR?
   
   This PR changes the comparison kernel so that the simd portion can always append 64 bits at a time. Since the simd types are 512 bits wide, this means the inner comparison is unrolled, for example 8 times for Float64 (8 x 8lanes) or 4 times for Float32 (4 x 16lanes). For Int8 types it does not get unrolled since one comparison already results in 64 bits.
   
   This should even speed up the comparison kernel a bit for common types, because there is less loop overhead.
   
   On my laptop the simd version for `i128` `MonthDayNano` types is not actually faster than the scalar version, on a more modern or server class machine there should be a slight speedup.
   
   Unrelated to this change I also noticed that the code generation for non-avx512 machines is sub-optimal since the compiler has to emulate the 512bit wider operations using smaller vector registers, and for the bitmap generating code this has some overhead.
   
   
   
   
   <!---
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   
   # Are there any user-facing changes?
   
   
   <!---
   If there are user-facing changes then we may require documentation to be updated before approving the PR.
   -->
   
   <!---
   If there are any breaking changes to public APIs, please add the `breaking change` label.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow-rs] alamb commented on pull request #1146: Implement SIMD comparison operations for types with less than 4 lanes (i128)

Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #1146:
URL: https://github.com/apache/arrow-rs/pull/1146#issuecomment-1009377030


   Thanks @jhorstmann 
   
   fyi @tustvold 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow-rs] alamb commented on a change in pull request #1146: Implement SIMD comparison operations for types with less than 4 lanes (i128)

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #1146:
URL: https://github.com/apache/arrow-rs/pull/1146#discussion_r781582735



##########
File path: arrow/src/compute/kernels/comparison.rs
##########
@@ -2723,8 +2743,6 @@ mod tests {
         );
     }
 
-    // Fails when simd is enabled: https://github.com/apache/arrow-rs/issues/1136

Review comment:
       πŸ‘ 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow-rs] alamb merged pull request #1146: Implement SIMD comparison operations for types with less than 4 lanes (i128)

Posted by GitBox <gi...@apache.org>.
alamb merged pull request #1146:
URL: https://github.com/apache/arrow-rs/pull/1146


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow-rs] codecov-commenter edited a comment on pull request #1146: Implement SIMD comparison operations for types with less than 4 lanes (i128)

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1146:
URL: https://github.com/apache/arrow-rs/pull/1146#issuecomment-1008318204


   # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1146](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb99dab) into [master](https://codecov.io/gh/apache/arrow-rs/commit/719096b2d342dd3bf1f3f2226a26b93e19602852?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (719096b) will **decrease** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   > :exclamation: Current head bb99dab differs from pull request most recent head 63beecd. Consider uploading reports for the commit 63beecd to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow-rs/pull/1146/graphs/tree.svg?width=650&height=150&src=pr&token=pq9V9qWZ1N&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #1146      +/-   ##
   ==========================================
   - Coverage   82.55%   82.55%   -0.01%     
   ==========================================
     Files         169      169              
     Lines       50456    50456              
   ==========================================
   - Hits        41655    41653       -2     
   - Misses       8801     8803       +2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Ξ” | |
   |---|---|---|
   | [arrow/src/compute/kernels/comparison.rs](https://codecov.io/gh/apache/arrow-rs/pull/1146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2NvbXB1dGUva2VybmVscy9jb21wYXJpc29uLnJz) | `91.96% <ΓΈ> (ΓΈ)` | |
   | [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `65.98% <0.00%> (-0.46%)` | :arrow_down: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Ξ” = absolute <relative> (impact)`, `ΓΈ = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [719096b...63beecd](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow-rs] paddyhoran commented on a change in pull request #1146: Implement SIMD comparison operations for types with less than 4 lanes (i128)

Posted by GitBox <gi...@apache.org>.
paddyhoran commented on a change in pull request #1146:
URL: https://github.com/apache/arrow-rs/pull/1146#discussion_r781201456



##########
File path: arrow/src/compute/kernels/comparison.rs
##########
@@ -1471,53 +1471,65 @@ where
 
     let null_bit_buffer = combine_option_bitmap(left.data_ref(), right.data_ref(), len)?;
 
+    // we process the data in chunks to that each iteration results in one u64 of comparison result bits

Review comment:
       ```suggestion
       // we process the data in chunks so that each iteration results in one u64 of comparison result bits
   ```

##########
File path: arrow/src/compute/kernels/comparison.rs
##########
@@ -1551,16 +1563,20 @@ where
 
     let len = left.len();
 
+    // we process the data in chunks to that each iteration results in one u64 of comparison result bits

Review comment:
       ```suggestion
       // we process the data in chunks so that each iteration results in one u64 of comparison result bits
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow-rs] codecov-commenter commented on pull request #1146: Implement SIMD comparison operations for types with less than 4 lanes (i128)

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1146:
URL: https://github.com/apache/arrow-rs/pull/1146#issuecomment-1008318204


   # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1146](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1e25b3c) into [master](https://codecov.io/gh/apache/arrow-rs/commit/719096b2d342dd3bf1f3f2226a26b93e19602852?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (719096b) will **decrease** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow-rs/pull/1146/graphs/tree.svg?width=650&height=150&src=pr&token=pq9V9qWZ1N&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #1146      +/-   ##
   ==========================================
   - Coverage   82.55%   82.54%   -0.01%     
   ==========================================
     Files         169      169              
     Lines       50456    50456              
   ==========================================
   - Hits        41655    41651       -4     
   - Misses       8801     8805       +4     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Ξ” | |
   |---|---|---|
   | [arrow/src/compute/kernels/comparison.rs](https://codecov.io/gh/apache/arrow-rs/pull/1146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2NvbXB1dGUva2VybmVscy9jb21wYXJpc29uLnJz) | `91.96% <ΓΈ> (ΓΈ)` | |
   | [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `65.98% <0.00%> (-0.46%)` | :arrow_down: |
   | [parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow-rs/pull/1146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZW5jb2RpbmdzL2VuY29kaW5nLnJz) | `93.52% <0.00%> (-0.20%)` | :arrow_down: |
   | [arrow/src/array/transform/mod.rs](https://codecov.io/gh/apache/arrow-rs/pull/1146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2FycmF5L3RyYW5zZm9ybS9tb2QucnM=) | `85.56% <0.00%> (-0.14%)` | :arrow_down: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Ξ” = absolute <relative> (impact)`, `ΓΈ = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [719096b...1e25b3c](https://codecov.io/gh/apache/arrow-rs/pull/1146?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org