You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2025/10/09 13:07:42 UTC
[I] [C++] Add dynamic dispatch on ARM64 [arrow]
pitrou opened a new issue, #47769:
URL: https://github.com/apache/arrow/issues/47769
### Describe the enhancement requested
On ARM64, Arrow C++ currently enables NEON at compile-time, and doesn't support any form of dynamic dispatch.
We should add dynamic dispatch for SVE and SVE2, and look into optimizing specific operations (such as those used when reading Parquet files) for SVE and/or SVE2 vectors.
### About SVE vs. SVE2
According to https://developer.arm.com/documentation/102340/0100/Introducing-SVE2, _"the main difference between SVE2 and SVE is the functional coverage of the instruction set. SVE was designed for HPC and ML applications. SVE2 extends the SVE instruction set to enable data-processing domains beyond HPC and ML."_
More details can be found in https://developer.arm.com/documentation/102340/0100/New-features-in-SVE2
### Testing on SVE2
It's not easy to find cloud instances that have SVE2 enabled. Possible picks:
* Axion-based instances on Google Cloud
* Graviton4-based instances on AWS (but not earlier generations of Graviton CPUs)
* Cobalt 100-based instances on Azure
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4243577822
So we'd want to dynamic dispatch to SVE128? Not sure how it compares to Neon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4223792508
AFAIK, the only SVE vector sizes in the wild are 128 and 256, so we should choose our macros accordingly.
> This may be departing from this, as we would need to supply multiple ones in the dynamic case (but only one in the static case), though we can make it work with some more CMake checks.
I'm not sure sure it's possible to enable two different SVE sizes at the same time, so we may need distinct compilations for vector width anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4243007121
We have the answer here (assuming that it's always the same instance type, which is of course not guaranteed):
https://github.com/apache/arrow/actions/runs/24392607522/job/71242267674?pr=49736
```
Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: ARM
Model name: Neoverse-N2
Model: 0
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Stepping: r0p0
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16
```
So it has SVE and even SVE2, and the SVE vector length is 128 bits according to various sources.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4243873327
> So we'd want to dynamic dispatch to SVE128? Not sure how it compares to Neon.
Yes, to SVE-128 and SVE-256 (and perhaps SVE2-128 and SVE2-256). Finding out how it compares to Neon is part of the interest, and that will certainly be informative to our client.
cc @cyb70289
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4334500835
Issue resolved by pull request 49756
https://github.com/apache/arrow/pull/49756
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "cyb70289 (via GitHub)" <gi...@apache.org>.
cyb70289 commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4248616817
SVE-128 is often no faster than Neon for similar instructions (e.g., vaddq -> svadd). Actually, it's sometimes slightly slower than Neon because most SVE operations require an additional predicate argument to activate only partial lanes, even though it's always all true for traditional SIMD coding.
SVE-256 will definitely help. AFAIK only Gravtion3 supports SVE-256 (no SVE2). Gravtion4 and 5 returns to SVE-128 (with SVE2).
Some new instructions introduced in SVE can be useful. E.g., SVE2 svmatch can boost special token detection in CSV parser [1], by matching multiple tokens in one shot [2].
[1] https://github.com/apache/arrow/blob/main/cpp/src/arrow/csv/lexing_internal.h#L177
[2] https://developer.arm.com/community/arm-community-blogs/b/architectures-and-processors-blog/posts/multi-token-search-strings-svmatch-instruction
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4249946761
> SVE-128 is often no faster than Neon for similar instructions (e.g., vaddq -> svadd). Actually, it's sometimes slightly slower than Neon because most SVE operations require an additional predicate argument to activate only partial lanes, even though it's always all true for traditional SIMD coding. SVE-256 will definitely help. AFAIK only Gravtion3 supports SVE-256 (no SVE2). Gravtion4 and 5 returns to SVE-128 (with SVE2).
I see, thanks. Unfortunately, it seems most SVE implementations nowadays are 128-bit?
> Some new instructions introduced in SVE can be useful. E.g., SVE2 svmatch can boost special token detection in CSV parser [1], by matching multiple tokens in one shot [2].
Is there an exhaustive list of new instructions in SVE (vs Neon) and SVE2 (vs SVE) somewhere? I couldn't find anything on my own.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "cyb70289 (via GitHub)" <gi...@apache.org>.
cyb70289 commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4250106439
>
> I see, thanks. Unfortunately, it seems most SVE implementations nowadays are 128-bit?
>
Yes. Looks some hardware seek for more execution units (higher parallelism), not wider vector.
>
> Is there an exhaustive list of new instructions in SVE (vs Neon) and SVE2 (vs SVE) somewhere? I couldn't find anything on my own.
You can check [ACLE](https://developer.arm.com/Architectures/Arm%20C%20Language%20Extensions) for related documents. [Arm C Language Extension for SVE](https://developer.arm.com/documentation/100987/0000/?lang=en) lists SVE and SVE2 intrinsics respectively.
No SVE vs Neon comparison, AFAIK there's not much new features (predicated loop might be one but not widely used due to incompatible to traditional simd coding).
[Arm intrinsics](https://developer.arm.com/architectures/instruction-sets/intrinsics/) is also a good reference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4222732099
I've been looking into this.
First xsimd does not currently support SVE2, but as Antoine said, it is not easy to find instances, meaning the demand is likely limited at the present time.
Second, SVE introduce a significant departure from the general SIMD model, which is that the register/batch size is a dynamic constant. One can write SVE code, and it can work with 256 or 2048 register sizes, sort of a built-in dynamic dispatch!
The issue with this is that xsimd had not yet adapted this model, and requires to know the size at compile time (there is compiler option to force it to a given size). Regardless of this, few of our algorithm (integer unpacking, byte stream split that I know of) also currently require to know the size at compile time.
As such I suggest we introduce hard coded macros `ARROW_HAVE_RUNTIME_SVE(256|512)`, `ARROW_RUNTIME_SIMD_LEVEL+=SVE256,SVE512` (these two sizes feel like a good start).
We currently have `ARROW_HAVE_SVE${SVE_VECTOR_BITS}`, which is gated behind `if(SVE_VECTOR_BITS)` (else `ARROW_HAVE_SVE_SIZELESS`). This may be departing from this, as we would need to supply multiple ones in the dynamic case (but only one in the static case), though we can make it work with some more CMake checks.
I am not aware of any code in Arrow using this macros, perhaps some are using it with auto vectorization capabilities.
What do you think @pitrou, seems like a good direction?
Also CC @raulcd on whether we have an SVE CI (otherwise how to set it up).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4223798382
> Also CC [@raulcd](https://github.com/raulcd) on whether we have an SVE CI (otherwise how to set it up).
Can probably run `lscpu` on one of our ARM64 Linux jobs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #47769:
URL: https://github.com/apache/arrow/issues/47769#issuecomment-4242943836
We should check what features do GitHub runner arm64 CPU provide. If they don't have the specific CPU capabilities we want to test we could potentially spin up an AWS instance and add a nightly job using runs-on to test those.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Add dynamic dispatch on ARM64 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #47769: [C++] Add dynamic dispatch on ARM64
URL: https://github.com/apache/arrow/issues/47769
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org