You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "parthchandra (via GitHub)" <gi...@apache.org> on 2026/02/02 19:29:47 UTC
[PR] graviton specific build [datafusion-comet]
parthchandra opened a new pull request, #3366:
URL: https://github.com/apache/datafusion-comet/pull/3366
## Which issue does this PR close?
## Rationale for this change
While running an Iceberg benchmark on a graviton cluster, @hsiang-c found the following calls taking ~12% of the execution time
```
_dl_tlsdesc_dynamic
__aarch64_cas8_acq_rel
__aarch64_swp8_acq_rel
__aarch64_ldadd8_rel
```
The `__aarch64_cas8_acq_rel, __aarch64_swp8_acq_rel, __aarch64_ldadd8_rel` calls can be optimized on graviton V2 and newer ARM processors into a single instruction.
The `_dl_tlsdesc_dynamic` call can be optimized to reduce the overhead of thread local storage resolution by the using a static linking model (https://maskray.me/blog/2021-02-14-all-about-thread-local-storage)
## What changes are included in this PR?
Adds a separate build for a graviton binary. Loading of the native binary checks for graviton processors and then loads the graviton version if necessary.
This can be overridden by specifying the environment variable - `NATIVE_VARIANT_ENV`
The build for graviton uses `nightly` in the build container. This is for the `-Z tls-model=initial-exec` flag which is only available in nightly.
The build also adds explicit dependencies on `serde` and `serde_urlencoded` which are depended on transitively by `reqwest` and are somehow not resolved while building on graviton.
The build adds to the size of the uber jar
```
-rwxr-xr-x@ 1 parth staff 90455632 Feb 2 10:32 ./common/target/classes/org/apache/comet/linux/aarch64/libcomet.so
-rwxr-xr-x@ 1 parth staff 98510816 Feb 2 10:20 ./common/target/classes/org/apache/comet/linux/amd64/libcomet.so
-rwxr-xr-x@ 1 parth staff 95560232 Feb 2 10:43 ./common/target/classes/org/apache/comet/linux/graviton/libcomet.so
```
The `build-comet-release.sh` script is also updated and makes it a little easier to specify which platforms to build the binary for. Building for graviton is _optional_
## How are these changes tested?
Manually by running the `build-release-comet.sh` script. Script and binary independently verified by @hsiang-c
Contributor note: @hsiang-c contributed to this PR. Also Claude Code contributed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] build: graviton specific build [datafusion-comet]
Posted by "parthchandra (via GitHub)" <gi...@apache.org>.
parthchandra closed pull request #3366: build: graviton specific build
URL: https://github.com/apache/datafusion-comet/pull/3366
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] graviton specific build [datafusion-comet]
Posted by "parthchandra (via GitHub)" <gi...@apache.org>.
parthchandra commented on code in PR #3366:
URL: https://github.com/apache/datafusion-comet/pull/3366#discussion_r2755890610
##########
Makefile:
##########
@@ -65,6 +65,13 @@ ifdef HAS_OSXCROSS
cd native && cargo build -j 2 --target aarch64-apple-darwin --release $(FEATURES_ARG)
endif
+# build native libs for Graviton architecture Linux on a Graviton machine/container
+# Requires Rust nightly for -Z tls-model flag
+# Uses initial-exec TLS model, LSE, and native CPU optimizations for Graviton
+core-graviton-libs:
+ cd native && RUSTFLAGS="-Z tls-model=initial-exec -C target-feature=+lse -C target-cpu=native" \
Review Comment:
The `-Z tls-model=initial-exec` flag requires `nightly`. The gain from this change only came to about 1% so we can revert this if we don't want to use `nightly`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] build: graviton specific build [datafusion-comet]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #3366:
URL: https://github.com/apache/datafusion-comet/pull/3366#issuecomment-3837668231
I propose just changing the `target-cpu` on ARM builds to `neoverse-n1` which gets you LSE (atomics) and is a reasonable baseline for ARM architectures available in 2026 from cloud vendors. It's also a subset of what Apple Silicon provides. Newer `neoverse` targets would be great, but I don't know if we want to require new Graviton and eliminate stuff like Ampere Altra. I'd need to do a survey of GCP and Azure ARM cores.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] build: graviton specific build [datafusion-comet]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #3366:
URL: https://github.com/apache/datafusion-comet/pull/3366#issuecomment-3837922309
## [Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/3366?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `12.50000%` with `28 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 16.45%. Comparing base ([`f09f8af`](https://app.codecov.io/gh/apache/datafusion-comet/commit/f09f8af64c6599255e116a376f4f008f2fd63b43?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e0eb578`](https://app.codecov.io/gh/apache/datafusion-comet/commit/e0eb578ef239891fa32caa6875ef586165d25cc2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 918 commits behind head on main.
| [Files with missing lines](https://app.codecov.io/gh/apache/datafusion-comet/pull/3366?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...mon/src/main/java/org/apache/comet/NativeBase.java](https://app.codecov.io/gh/apache/datafusion-comet/pull/3366?src=pr&el=tree&filepath=common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fcomet%2FNativeBase.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9jb21ldC9OYXRpdmVCYXNlLmphdmE=) | 12.50% | [24 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/datafusion-comet/pull/3366?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #3366 +/- ##
=============================================
- Coverage 56.12% 16.45% -39.68%
+ Complexity 976 402 -574
=============================================
Files 119 175 +56
Lines 11743 16182 +4439
Branches 2251 2666 +415
=============================================
- Hits 6591 2662 -3929
- Misses 4012 13107 +9095
+ Partials 1140 413 -727
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/datafusion-comet/pull/3366?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] build: graviton specific build [datafusion-comet]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #3366:
URL: https://github.com/apache/datafusion-comet/pull/3366#issuecomment-3838423224
> > I propose just changing the `target-cpu` on ARM builds to `neoverse-n1` which gets you LSE (atomics) and is a reasonable baseline for ARM architectures available in 2026 from cloud vendors. It's also a subset of what Apple Silicon provides. Newer `neoverse` targets would be great, but I don't know if we want to require new Graviton and eliminate stuff like Ampere Altra. I'd need to do a survey of GCP and Azure ARM cores.
>
> One reason for doing it this way was that building for LSE and running on an ARM version that doesn't support LSE causes an instant crash. I don't have any real world data on how many people may be on older versions of ARM but didn't want to take the chance.
As best as I can tell, LSE was added in ARMv-8.1-A, which predates anything I think we'd find in modern data centers. Azure and GCP use something newer than `neoverse-n1`, and `n1` seems like a reasonable baselines for current `Graviton` offerings. In the same way probably don't want to ship binaries for anything that doesn't have AVX2, I think it's reasonable to cut off ARM without LSE, at least for shipped binaries. Folks are free to compile their own if they need, in the same way that they'll get a faster build if they compile on their target architecture.
I summarized my thoughts in https://github.com/apache/datafusion-comet/pull/3368
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] build: graviton specific build [datafusion-comet]
Posted by "parthchandra (via GitHub)" <gi...@apache.org>.
parthchandra commented on PR #3366:
URL: https://github.com/apache/datafusion-comet/pull/3366#issuecomment-3837861789
> I propose just changing the `target-cpu` on ARM builds to `neoverse-n1` which gets you LSE (atomics) and is a reasonable baseline for ARM architectures available in 2026 from cloud vendors. It's also a subset of what Apple Silicon provides. Newer `neoverse` targets would be great, but I don't know if we want to require new Graviton and eliminate stuff like Ampere Altra. I'd need to do a survey of GCP and Azure ARM cores.
One reason for doing it this way was that building for LSE and running on an ARM version that doesn't support LSE causes an instant crash. I don't have any real world data on how many people may be on older versions of ARM but didn't want to take the chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org