You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by "chiranmoyFujitsu (via GitHub)" <gi...@apache.org> on 2025/06/10 06:02:44 UTC

[D] Prospects of SVE based optimizations [arrow]

GitHub user chiranmoyFujitsu created a discussion: Prospects of SVE based optimizations

Are SIMD optimizations with regard to SVE acceptable by the community?
I see that there are many functions utilizing NEON while benchmarking but barely any that use SVE.
There are lots of functions using, and in turn utilizing SSE, AVX and NEON, but not for SVE.

GitHub link: https://github.com/apache/arrow/discussions/46762

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org


Re: [D] Prospects of SVE based optimizations [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
GitHub user lidavidm added a comment to the discussion: Prospects of SVE based optimizations

I don't think M-series processors support SVE (not even M4, [according to Wikipedia](https://en.wikipedia.org/wiki/Apple_M4), which otherwise supports ARMv9). It's only on Fujitsu's A64X and relatively recent ARM designs.

It appears Linux ARM runners from GitHub should support it, being either Neoverse V1 or Neoverse V2, but apparently they're missing support: https://github.com/orgs/community/discussions/19197#discussioncomment-12315925

GitHub link: https://github.com/apache/arrow/discussions/46762#discussioncomment-13418041

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org


Re: [D] Prospects of SVE based optimizations [arrow]

Posted by "chiranmoyFujitsu (via GitHub)" <gi...@apache.org>.
GitHub user chiranmoyFujitsu added a comment to the discussion: Prospects of SVE based optimizations

> I don't think M-series processors support SVE (not even M4, [according to Wikipedia](https://en.wikipedia.org/wiki/Apple_M4), which otherwise supports ARMv9). It's only on Fujitsu's A64X and relatively recent ARM designs.

Not just Fujitsu A64X, the [Graviton](https://github.com/aws/aws-graviton-getting-started?tab=readme-ov-file) series from AWS also supports SVE.

> It appears Linux ARM runners from GitHub should support it, being either Neoverse V1 or Neoverse V2, but apparently they're missing support:

Thank you for the information about the CI. For now, we will try to look into possible optimisations with SVE, and the community can make a decision based on the utility

GitHub link: https://github.com/apache/arrow/discussions/46762#discussioncomment-13418918

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org


Re: [D] Prospects of SVE based optimizations [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
GitHub user lidavidm added a comment to the discussion: Prospects of SVE based optimizations

It's frustratingly hard to find documentation on exact processors but it looks like "maybe in the future" is the answer, at least for GitHub hosted runners.

GitHub link: https://github.com/apache/arrow/discussions/46762#discussioncomment-13418062

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org


Re: [D] Prospects of SVE based optimizations [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
GitHub user kou added a comment to the discussion: Prospects of SVE based optimizations

I don't know...
If `macos-14` or `macos-15` support SVE, we have it.

GitHub link: https://github.com/apache/arrow/discussions/46762#discussioncomment-13417837

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org


Re: [D] Prospects of SVE based optimizations [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
GitHub user kou added a comment to the discussion: Prospects of SVE based optimizations

Note that we need not only CI but also at least one maintainer for maintain.
If we don't have any maintainer, we need to remove it eventually. For example: https://lists.apache.org/thread/f5ndxf0o6v4wbzgoy6nsrj4xkn34f380

GitHub link: https://github.com/apache/arrow/discussions/46762#discussioncomment-13425613

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org


Re: [D] Prospects of SVE based optimizations [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
GitHub user kou added a comment to the discussion: Prospects of SVE based optimizations

It's acceptable if we can maintain. See also: https://github.com/apache/arrow/issues/46002

GitHub link: https://github.com/apache/arrow/discussions/46762#discussioncomment-13417110

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org


Re: [D] Prospects of SVE based optimizations [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
GitHub user lidavidm added a comment to the discussion: Prospects of SVE based optimizations

We don't have any CI runner that supports SVE currently, right?

GitHub link: https://github.com/apache/arrow/discussions/46762#discussioncomment-13417743

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org


Re: [D] Prospects of SVE based optimizations [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
GitHub user pitrou added a comment to the discussion: Prospects of SVE based optimizations

> Are SIMD optimizations with regard to SVE acceptable by the community?

Hand-written SIMD optimizations should almost always use [xsimd](https://xsimd.readthedocs.io/en/latest/).

> I see that there are many functions utilizing NEON while benchmarking but barely any that use SVE.

How did you come to this conclusion ("many functions utilizing NEON"?). NEON is enabled by default when compiling for ARM/ARM64, but we have very few hand-written optimizations. So this might simply be code vectorized by the compiler.



GitHub link: https://github.com/apache/arrow/discussions/46762#discussioncomment-13429228

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org


Re: [D] [C++] Prospects of SVE based optimizations [arrow]

Posted by "chiranmoyf (via GitHub)" <gi...@apache.org>.
GitHub user chiranmoyf edited a discussion: [C++] Prospects of SVE based optimizations

Are SIMD optimizations with regard to SVE acceptable by the community?
I see that there are many functions utilizing NEON while benchmarking but barely any that use SVE.
There are lots of functions using, and in turn utilizing SSE, AVX and NEON, but not for SVE.

GitHub link: https://github.com/apache/arrow/discussions/46762

----
This is an automatically sent email for user@arrow.apache.org.
To unsubscribe, please send an email to: user-unsubscribe@arrow.apache.org