You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by va...@apache.org on 2018/02/26 18:28:56 UTC
spark git commit: [SPARK-23449][K8S] Preserve extraJavaOptions
ordering
Repository: spark
Updated Branches:
refs/heads/master b308182f2 -> 185f5bc7d
[SPARK-23449][K8S] Preserve extraJavaOptions ordering
For some JVM options, like `-XX:+UnlockExperimentalVMOptions` ordering is necessary.
## What changes were proposed in this pull request?
Keep original `extraJavaOptions` ordering, when passing them through environment variables inside the Docker container.
## How was this patch tested?
Ran base branch a couple of times and checked startup command in logs. Ordering differed every time. Added sorting, ordering was consistent to what user had in `extraJavaOptions`.
Author: Andrew Korzhuev <ko...@andrusha.me>
Closes #20628 from andrusha/patch-2.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/185f5bc7
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/185f5bc7
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/185f5bc7
Branch: refs/heads/master
Commit: 185f5bc7dd52cebe8fac9393ecb2bd0968bc5867
Parents: b308182
Author: Andrew Korzhuev <ko...@andrusha.me>
Authored: Mon Feb 26 10:28:45 2018 -0800
Committer: Marcelo Vanzin <va...@cloudera.com>
Committed: Mon Feb 26 10:28:45 2018 -0800
----------------------------------------------------------------------
.../kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/185f5bc7/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh
----------------------------------------------------------------------
diff --git a/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh b/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh
index b9090dc..3d67b0a 100755
--- a/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh
+++ b/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh
@@ -41,7 +41,7 @@ fi
shift 1
SPARK_CLASSPATH="$SPARK_CLASSPATH:${SPARK_HOME}/jars/*"
-env | grep SPARK_JAVA_OPT_ | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt
+env | grep SPARK_JAVA_OPT_ | sort -t_ -k4 -n | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt
readarray -t SPARK_JAVA_OPTS < /tmp/java_opts.txt
if [ -n "$SPARK_MOUNTED_CLASSPATH" ]; then
SPARK_CLASSPATH="$SPARK_CLASSPATH:$SPARK_MOUNTED_CLASSPATH"
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org