You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "liuneng1994 (via GitHub)" <gi...@apache.org> on 2026/02/24 05:54:14 UTC
[PR] [CORE] Add Bloop integration for faster Scala incremental compilation [incubator-gluten]
liuneng1994 opened a new pull request, #11645:
URL: https://github.com/apache/incubator-gluten/pull/11645
## Summary
This PR adds Bloop build server integration to accelerate Scala development workflows. Bloop maintains a persistent JVM with warm Zinc compiler state, dramatically reducing incremental compilation times.
### Key Changes
- **pom.xml**: Add bloop-maven-plugin (v2.0.3) with version property, pluginManagement entry, and `-Pbloop` profile to skip style checks during config generation
- **dev/bloop-setup.sh**: Script to generate Bloop configuration with JVM test options injection (required for Java 17+) and `-release` option removal for compatibility
- **dev/bloop-test.sh**: Convenience wrapper with `SPARK_ANSI_SQL_MODE=false` default for Spark 4.x test compatibility
- **docs/developers/bloop-integration.md**: Comprehensive usage guide with setup instructions and troubleshooting
### Benchmark Results
| Scenario | Maven | Bloop | Speedup |
|----------|-------|-------|---------|
| gluten-core incremental | 8.26s | 0.23s | **35.9x** |
| gluten-core clean+compile | 29.62s | 14.90s | **2.0x** |
| Full project incremental | ~2 min | 2.82s | **40.1x** |
### Usage
```bash
# Generate bloop config (first time / when changing profiles)
./dev/bloop-setup.sh -Pspark-3.5,scala-2.12,backends-velox
# Compile with watch mode (auto-recompile on save)
bloop compile gluten-core -w
# Run tests
bloop test backends-velox -o '*VeloxHashJoinSuite*'
# Or use the convenience wrapper
./dev/bloop-test.sh -pl backends-velox -s VeloxHashJoinSuite
```
## Test plan
- [x] Verified bloop config generation with `./dev/bloop-setup.sh -Pjava-17,spark-4.1,scala-2.13,backends-velox`
- [x] Verified compilation: `bloop compile gluten-core` - passes
- [x] Verified tests: `bloop test gluten-core` - 6 suites, 24 tests pass
- [x] Verified Spark 4.1 tests with ANSI mode disabled: `SPARK_ANSI_SQL_MODE=false bloop test backends-velox -o '*VeloxHashJoinSuite*'` - 7 tests pass
- [x] Verified benchmark results show significant speedup
- [x] Default Maven builds remain unaffected (bloop profile is opt-in)
🤖 Generated with [Claude Code](https://claude.ai/code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE] Add Bloop integration for faster Scala incremental compilation [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11645:
URL: https://github.com/apache/incubator-gluten/pull/11645#issuecomment-3949322606
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE] Add Bloop integration for faster Scala incremental compilation [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11645:
URL: https://github.com/apache/incubator-gluten/pull/11645#issuecomment-3949309591
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE] Add Bloop integration for faster Scala incremental compilation [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11645:
URL: https://github.com/apache/incubator-gluten/pull/11645#issuecomment-3956306602
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org