You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/01/27 00:39:58 UTC
[spark] branch branch-3.2 updated: [SPARK-42201][BUILD] `build/sbt` should allow `SBT_OPTS` to override JVM memory setting
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new ab5109e24e1 [SPARK-42201][BUILD] `build/sbt` should allow `SBT_OPTS` to override JVM memory setting
ab5109e24e1 is described below
commit ab5109e24e142efd4e26a0dba2cd0e1e5a788b3d
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Thu Jan 26 16:39:17 2023 -0800
[SPARK-42201][BUILD] `build/sbt` should allow `SBT_OPTS` to override JVM memory setting
### What changes were proposed in this pull request?
This PR aims to fix a bug which `build/sbt` doesn't allow JVM memory setting via `SBT_OPTS`.
### Why are the changes needed?
`SBT_OPTS` is supposed to be used in this way in the community.
https://github.com/apache/spark/blob/e30bb538e480940b1963eb14c3267662912d8584/appveyor.yml#L54
However, `SBT_OPTS` memory setting like the following is ignored because ` -Xms4096m -Xmx4096m -XX:ReservedCodeCacheSize=512m` is injected by default after `SBT_OPTS`. We should switch the order.
```
$ SBT_OPTS="-Xmx6g" build/sbt package
```
https://github.com/apache/spark/blob/e30bb538e480940b1963eb14c3267662912d8584/build/sbt-launch-lib.bash#L124
### Does this PR introduce _any_ user-facing change?
No. This is a dev-only change.
### How was this patch tested?
Manually run the following.
```
$ SBT_OPTS="-Xmx6g" build/sbt package
```
While running the above command, check the JVM options.
```
$ ps aux | grep java
dongjoon 36683 434.3 3.1 418465456 1031888 s001 R+ 1:11PM 0:19.86 /Users/dongjoon/.jenv/versions/temurin17/bin/java -Xms4096m -Xmx4096m -XX:ReservedCodeCacheSize=512m -Xmx6g -jar build/sbt-launch-1.8.2.jar package
```
Closes #39758 from dongjoon-hyun/SPARK-42201.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit 66ec1eb630a4682f5ad2ed2ee989ffcce9031608)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
build/sbt-launch-lib.bash | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/build/sbt-launch-lib.bash b/build/sbt-launch-lib.bash
index 8fb6672bddc..01ba6b929f9 100755
--- a/build/sbt-launch-lib.bash
+++ b/build/sbt-launch-lib.bash
@@ -183,8 +183,8 @@ run() {
# run sbt
execRunner "$java_cmd" \
- ${SBT_OPTS:-$default_sbt_opts} \
$(get_mem_opts $sbt_mem) \
+ ${SBT_OPTS:-$default_sbt_opts} \
${java_opts} \
${java_args[@]} \
-jar "$sbt_jar" \
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org