You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "rmuir (via GitHub)" <gi...@apache.org> on 2023/11/01 05:37:34 UTC

[PR] speedup arm int functions? [lucene]

rmuir opened a new pull request, #12743:
URL: https://github.com/apache/lucene/pull/12743

   I was looking at this curious little guy which seems close to what we want? https://github.com/openjdk/jdk/blob/b3fec6b5f32c338ae1a84dd20bdcbd3d9b7186f3/src/hotspot/cpu/aarch64/aarch64_vector_ad.m4#L1375-L1390
   
   Not sure what it is there for, maybe to speedup some hashing function or crypto or something.
   
   it has a test: https://github.com/openjdk/jdk/blob/b3fec6b5f32c338ae1a84dd20bdcbd3d9b7186f3/test/hotspot/jtreg/compiler/loopopts/superword/Vec_MulAddS2I.java#L81
   
   So i changed an integer function to look more like that and it shows speedups on arm without slowdown on x86. needs more investigation. maybe can be applied to some of the other functions, not sure, i am sleepy.
   
   ```
   master:
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt    5  3.121 ± 0.001  ops/us
   VectorUtilBenchmark.binaryDotProductVector    1024  thrpt    5  6.742 ± 0.037  ops/us
   
   patch:
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt    5  4.523 ± 0.077  ops/us
   VectorUtilBenchmark.binaryDotProductVector    1024  thrpt    5  6.682 ± 0.066  ops/us
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1792630498

   equivalent on intel ice lake: https://www.felixcloutier.com/x86/vpdpbusd
   IMO, we should figure out a path to using these, to get the best performance from the binary vectors. it isn't useful to spend time trying to do many other hacks around the situation when the "problem has been solved", we just have to make use of the hardware.
   
   but i think this little patch won't hurt anyone and improves the scalar baseline.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1788889191

   I see reasonable speedups on both x64 and ARM, but sadly see no vectorization in the disassembly.  The speed seems to come from the 2x instructions/pipelining (?) per strip mined loop iteration, rather than any vectorization (which is not happening).  [ I included the Panama/Vector output just for comparison purposes ]
   
   Mac M2:
   
   main:
   ```
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt    5  3.151 ± 0.044  ops/us
   VectorUtilBenchmark.binaryDotProductVector    1024  thrpt    5  7.112 ± 0.030  ops/us
   ```
   
   PR branch:
   ```
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt    5  4.722 ± 0.031  ops/us
   VectorUtilBenchmark.binaryDotProductVector    1024  thrpt    5  7.127 ± 0.042  ops/us
   ```
   
   I see a reasonable speed up on Linux / x64. 
   
   main:
   ```
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt    5   2.121 ± 0.010  ops/us
   VectorUtilBenchmark.binaryDotProductVector    1024  thrpt    5  21.341 ± 0.039  ops/us
   ```
   
   PR branch:
   ```
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt    5   2.799 ± 0.022  ops/us
   VectorUtilBenchmark.binaryDotProductVector    1024  thrpt    5  21.337 ± 0.014  ops/us
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1788910222

   thank you for the explanation, so now it just adds another mystery for me.  Your x64 is still i5-11400 ? I will look into this more.
   
   Yes on the ARM, the assembly didn't look like what i wanted but inputs are `byte` and not `short` which might be relevant. But performance improved, so that means we should investigate wtf is happening and try to take it to the limit :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1803265618

   speeds up as many machines as it slows down.
   
   cascadelake: `['0', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz', '1', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz']`
   
   main
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.504 ± 0.005  ops/us
   ```
   patch
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  1.850 ± 0.013  ops/us
   ```
   
   graviton2: `['0', '1']`
   
   main
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.387 ± 0.001  ops/us
   ```
   patch
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.121 ± 0.004  ops/us
   ```
   
   graviton3: `['0', '1']`
   
   main
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.370 ± 0.002  ops/us
   ```
   patch
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  3.346 ± 0.025  ops/us
   ```
   
   haswell: `['0', 'GenuineIntel', 'Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz', '1', 'GenuineIntel', 'Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz']`
   
   main
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.096 ± 0.016  ops/us
   ```
   patch
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  1.678 ± 0.039  ops/us
   ```
   
   icelake: `['0', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz', '1', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz']`
   
   main
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.586 ± 0.010  ops/us
   ```
   patch
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.240 ± 0.006  ops/us
   ```
   
   sapphirerapids: `['0', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8488C', '1', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8488C']`
   
   main
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.408 ± 0.009  ops/us
   ```
   patch
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.729 ± 0.058  ops/us
   ```
   
   zen2: `['0', 'AuthenticAMD', 'AMD EPYC 7R32', '1', 'AuthenticAMD', 'AMD EPYC 7R32']`
   
   main
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  1.544 ± 0.004  ops/us
   ```
   patch
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  1.435 ± 0.004  ops/us
   ```
   
   zen3: `['0', 'AuthenticAMD', 'AMD EPYC 7R13 Processor', '1', 'AuthenticAMD', 'AMD EPYC 7R13 Processor']`
   
   main
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  1.580 ± 0.002  ops/us
   ```
   patch
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  1.791 ± 0.004  ops/us
   ```
   zen4: `['0', 'AuthenticAMD', 'AMD EPYC 9R14', '1', 'AuthenticAMD', 'AMD EPYC 9R14']`
   
   main
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  1.856 ± 0.002  ops/us
   ```
   patch
   ```
   Benchmark                                   (size)   Mode  Cnt  Score   Error   Units
   VectorUtilBenchmark.binaryDotProductScalar    1024  thrpt   15  2.364 ± 0.007  ops/us
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1792596894

   @ChrisHegarty I did some investigation, looked at the assembly on ARM machines, did some experiments, etc. I didn't mess around with intel, but i think the situation is the same. My thoughts:
   * lots of overhead here for each loop iteration other than just `mul`/`madd` for what we do here. so there's some room left on the table on some cpus.
   * does not help the other functions as they have more going on already
   * does not help vector functions, specific to this one
   * cannot find a way to get more parallelism
   
   e.g. on the arm it is more than just `ldrsb` to load from byte array, convert to int, sign-extend, etc. There is the overhead of the loop too. so we just get a little more parallelism out of this simple function. for the other functions here this trick is not helpful: they already have a "meatier" loop body if that makes sense.
   
   So I think we should commit the little trick for now. The way to get more parallelization long-term for this method is to use the correct cpu instructions (e.g. `sdot`): https://developer.arm.com/documentation/102651/a/What-are-dot-product-intructions- . As the documentation states, with 128-bit vectors it can do 64 dot products per cycle. I assume possibly twice that on 256-bit vectors.
   
   on intel there is a similar feature. I don't even know where to begin/suggest when looking at vector API, how such instructions would be exposed. It is like a VectorOperator which is both ternary and conversion :). The best idea i have, would be to expose it as ternary operator on ByteVector that returns ByteVector and force the user to reinterpret the underlying bits as IntVector.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1789007292

   > Your x64 is still i5-11400 ? I will look into this more.
   
   Yes.   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1794112273

   When looking at `SDOT` to do this, i was able to accomplish it with another vector API, just as basis for comparison: 
   
   https://godbolt.org/z/9cv5WaGaT
   
   Even if you lower `-march=armv8-a` to exclude `SDOT`, it just uses `smull`/`smull2` and `saddw`/`saddw2` to work on lower/upper halves of vector.
   
   On the other hand here I am struggling with crazy conversions and generics part numbers and slicing abstractions :( I don't actually want to convert anything!
   
   The only lone use of long multiplication i find in openjdk arm support is for the "curious little guy".


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir closed pull request #12743: speedup arm int functions?
URL: https://github.com/apache/lucene/pull/12743


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1788919960

   and what is the compiler's excuse this time? :) no floating point here. I should just be able to write a simple loop and get good performance!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1789014829

   jmh prof output from my Linux x64 rocket lake - https://gist.github.com/ChrisHegarty/508bb1857cb50df0d757f711c81fd740


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1789229323

   thanks @ChrisHegarty ! I will try to dig in more to this. Again i don't see speedups on my x86 so something has changed (maybe architecture) thats allowing more to happen in parallel on your CPU or something. wikichip has been down so I don't know any answers yet. nor if it can apply to the vector case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] speedup arm int functions? [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1794797146

   we should test more. I can test on more machines later too. Maybe good motivation to get full automation going for doing that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org