You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "rmuir (via GitHub)" <gi...@apache.org> on 2023/11/01 05:37:34 UTC
[PR] speedup arm int functions? [lucene]
rmuir opened a new pull request, #12743:
URL: https://github.com/apache/lucene/pull/12743
I was looking at this curious little guy which seems close to what we want? https://github.com/openjdk/jdk/blob/b3fec6b5f32c338ae1a84dd20bdcbd3d9b7186f3/src/hotspot/cpu/aarch64/aarch64_vector_ad.m4#L1375-L1390
Not sure what it is there for, maybe to speedup some hashing function or crypto or something.
it has a test: https://github.com/openjdk/jdk/blob/b3fec6b5f32c338ae1a84dd20bdcbd3d9b7186f3/test/hotspot/jtreg/compiler/loopopts/superword/Vec_MulAddS2I.java#L81
So i changed an integer function to look more like that and it shows speedups on arm without slowdown on x86. needs more investigation. maybe can be applied to some of the other functions, not sure, i am sleepy.
```
master:
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 5 3.121 ± 0.001 ops/us
VectorUtilBenchmark.binaryDotProductVector 1024 thrpt 5 6.742 ± 0.037 ops/us
patch:
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 5 4.523 ± 0.077 ops/us
VectorUtilBenchmark.binaryDotProductVector 1024 thrpt 5 6.682 ± 0.066 ops/us
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1792630498
equivalent on intel ice lake: https://www.felixcloutier.com/x86/vpdpbusd
IMO, we should figure out a path to using these, to get the best performance from the binary vectors. it isn't useful to spend time trying to do many other hacks around the situation when the "problem has been solved", we just have to make use of the hardware.
but i think this little patch won't hurt anyone and improves the scalar baseline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1788889191
I see reasonable speedups on both x64 and ARM, but sadly see no vectorization in the disassembly. The speed seems to come from the 2x instructions/pipelining (?) per strip mined loop iteration, rather than any vectorization (which is not happening). [ I included the Panama/Vector output just for comparison purposes ]
Mac M2:
main:
```
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 5 3.151 ± 0.044 ops/us
VectorUtilBenchmark.binaryDotProductVector 1024 thrpt 5 7.112 ± 0.030 ops/us
```
PR branch:
```
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 5 4.722 ± 0.031 ops/us
VectorUtilBenchmark.binaryDotProductVector 1024 thrpt 5 7.127 ± 0.042 ops/us
```
I see a reasonable speed up on Linux / x64.
main:
```
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 5 2.121 ± 0.010 ops/us
VectorUtilBenchmark.binaryDotProductVector 1024 thrpt 5 21.341 ± 0.039 ops/us
```
PR branch:
```
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 5 2.799 ± 0.022 ops/us
VectorUtilBenchmark.binaryDotProductVector 1024 thrpt 5 21.337 ± 0.014 ops/us
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1788910222
thank you for the explanation, so now it just adds another mystery for me. Your x64 is still i5-11400 ? I will look into this more.
Yes on the ARM, the assembly didn't look like what i wanted but inputs are `byte` and not `short` which might be relevant. But performance improved, so that means we should investigate wtf is happening and try to take it to the limit :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1803265618
speeds up as many machines as it slows down.
cascadelake: `['0', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz', '1', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz']`
main
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.504 ± 0.005 ops/us
```
patch
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 1.850 ± 0.013 ops/us
```
graviton2: `['0', '1']`
main
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.387 ± 0.001 ops/us
```
patch
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.121 ± 0.004 ops/us
```
graviton3: `['0', '1']`
main
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.370 ± 0.002 ops/us
```
patch
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 3.346 ± 0.025 ops/us
```
haswell: `['0', 'GenuineIntel', 'Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz', '1', 'GenuineIntel', 'Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz']`
main
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.096 ± 0.016 ops/us
```
patch
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 1.678 ± 0.039 ops/us
```
icelake: `['0', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz', '1', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz']`
main
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.586 ± 0.010 ops/us
```
patch
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.240 ± 0.006 ops/us
```
sapphirerapids: `['0', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8488C', '1', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8488C']`
main
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.408 ± 0.009 ops/us
```
patch
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.729 ± 0.058 ops/us
```
zen2: `['0', 'AuthenticAMD', 'AMD EPYC 7R32', '1', 'AuthenticAMD', 'AMD EPYC 7R32']`
main
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 1.544 ± 0.004 ops/us
```
patch
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 1.435 ± 0.004 ops/us
```
zen3: `['0', 'AuthenticAMD', 'AMD EPYC 7R13 Processor', '1', 'AuthenticAMD', 'AMD EPYC 7R13 Processor']`
main
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 1.580 ± 0.002 ops/us
```
patch
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 1.791 ± 0.004 ops/us
```
zen4: `['0', 'AuthenticAMD', 'AMD EPYC 9R14', '1', 'AuthenticAMD', 'AMD EPYC 9R14']`
main
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 1.856 ± 0.002 ops/us
```
patch
```
Benchmark (size) Mode Cnt Score Error Units
VectorUtilBenchmark.binaryDotProductScalar 1024 thrpt 15 2.364 ± 0.007 ops/us
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1792596894
@ChrisHegarty I did some investigation, looked at the assembly on ARM machines, did some experiments, etc. I didn't mess around with intel, but i think the situation is the same. My thoughts:
* lots of overhead here for each loop iteration other than just `mul`/`madd` for what we do here. so there's some room left on the table on some cpus.
* does not help the other functions as they have more going on already
* does not help vector functions, specific to this one
* cannot find a way to get more parallelism
e.g. on the arm it is more than just `ldrsb` to load from byte array, convert to int, sign-extend, etc. There is the overhead of the loop too. so we just get a little more parallelism out of this simple function. for the other functions here this trick is not helpful: they already have a "meatier" loop body if that makes sense.
So I think we should commit the little trick for now. The way to get more parallelization long-term for this method is to use the correct cpu instructions (e.g. `sdot`): https://developer.arm.com/documentation/102651/a/What-are-dot-product-intructions- . As the documentation states, with 128-bit vectors it can do 64 dot products per cycle. I assume possibly twice that on 256-bit vectors.
on intel there is a similar feature. I don't even know where to begin/suggest when looking at vector API, how such instructions would be exposed. It is like a VectorOperator which is both ternary and conversion :). The best idea i have, would be to expose it as ternary operator on ByteVector that returns ByteVector and force the user to reinterpret the underlying bits as IntVector.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1789007292
> Your x64 is still i5-11400 ? I will look into this more.
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1794112273
When looking at `SDOT` to do this, i was able to accomplish it with another vector API, just as basis for comparison:
https://godbolt.org/z/9cv5WaGaT
Even if you lower `-march=armv8-a` to exclude `SDOT`, it just uses `smull`/`smull2` and `saddw`/`saddw2` to work on lower/upper halves of vector.
On the other hand here I am struggling with crazy conversions and generics part numbers and slicing abstractions :( I don't actually want to convert anything!
The only lone use of long multiplication i find in openjdk arm support is for the "curious little guy".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir closed pull request #12743: speedup arm int functions?
URL: https://github.com/apache/lucene/pull/12743
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1788919960
and what is the compiler's excuse this time? :) no floating point here. I should just be able to write a simple loop and get good performance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1789014829
jmh prof output from my Linux x64 rocket lake - https://gist.github.com/ChrisHegarty/508bb1857cb50df0d757f711c81fd740
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1789229323
thanks @ChrisHegarty ! I will try to dig in more to this. Again i don't see speedups on my x86 so something has changed (maybe architecture) thats allowing more to happen in parallel on your CPU or something. wikichip has been down so I don't know any answers yet. nor if it can apply to the vector case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] speedup arm int functions? [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1794797146
we should test more. I can test on more machines later too. Maybe good motivation to get full automation going for doing that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org