You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by yi...@apache.org on 2023/02/21 09:02:59 UTC
[spark-docker] branch master updated: [SPARK-42505] Apply entrypoint template change to 3.3.0/3.3.1
This is an automated email from the ASF dual-hosted git repository.
yikun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-docker.git
The following commit(s) were added to refs/heads/master by this push:
new 02bc905 [SPARK-42505] Apply entrypoint template change to 3.3.0/3.3.1
02bc905 is described below
commit 02bc9054d757f8defbc2baf6af1d2a9aa84b2b35
Author: Yikun Jiang <yi...@gmail.com>
AuthorDate: Tue Feb 21 17:02:29 2023 +0800
[SPARK-42505] Apply entrypoint template change to 3.3.0/3.3.1
### What changes were proposed in this pull request?
Apply entrypoint template change to 3.3.0/3.3.1
### Why are the changes needed?
We remove the redundant PySpark related vars in https://github.com/apache/spark-docker/commit/e8f5b0a1151c349d9c7fdb09cf76300b42a6946b . This change also should be apply to 3.3.0/3.3.1.
### Does this PR introduce _any_ user-facing change?
No, because the image hasn't plublished yet.
### How was this patch tested?
CI for 3.3.0/3.3.1 passed
Closes #31 from Yikun/SPARK-42505.
Authored-by: Yikun Jiang <yi...@gmail.com>
Signed-off-by: Yikun Jiang <yi...@gmail.com>
---
3.3.0/scala2.12-java11-r-ubuntu/entrypoint.sh | 7 -------
3.3.0/scala2.12-java11-ubuntu/entrypoint.sh | 7 -------
3.3.1/scala2.12-java11-r-ubuntu/entrypoint.sh | 7 -------
3.3.1/scala2.12-java11-ubuntu/entrypoint.sh | 7 -------
4 files changed, 28 deletions(-)
diff --git a/3.3.0/scala2.12-java11-r-ubuntu/entrypoint.sh b/3.3.0/scala2.12-java11-r-ubuntu/entrypoint.sh
index 4bb1557..159d539 100644
--- a/3.3.0/scala2.12-java11-r-ubuntu/entrypoint.sh
+++ b/3.3.0/scala2.12-java11-r-ubuntu/entrypoint.sh
@@ -45,13 +45,6 @@ if [ -n "$SPARK_EXTRA_CLASSPATH" ]; then
SPARK_CLASSPATH="$SPARK_CLASSPATH:$SPARK_EXTRA_CLASSPATH"
fi
-if ! [ -z ${PYSPARK_PYTHON+x} ]; then
- export PYSPARK_PYTHON
-fi
-if ! [ -z ${PYSPARK_DRIVER_PYTHON+x} ]; then
- export PYSPARK_DRIVER_PYTHON
-fi
-
# If HADOOP_HOME is set and SPARK_DIST_CLASSPATH is not set, set it here so Hadoop jars are available to the executor.
# It does not set SPARK_DIST_CLASSPATH if already set, to avoid overriding customizations of this value from elsewhere e.g. Docker/K8s.
if [ -n "${HADOOP_HOME}" ] && [ -z "${SPARK_DIST_CLASSPATH}" ]; then
diff --git a/3.3.0/scala2.12-java11-ubuntu/entrypoint.sh b/3.3.0/scala2.12-java11-ubuntu/entrypoint.sh
index 4bb1557..159d539 100644
--- a/3.3.0/scala2.12-java11-ubuntu/entrypoint.sh
+++ b/3.3.0/scala2.12-java11-ubuntu/entrypoint.sh
@@ -45,13 +45,6 @@ if [ -n "$SPARK_EXTRA_CLASSPATH" ]; then
SPARK_CLASSPATH="$SPARK_CLASSPATH:$SPARK_EXTRA_CLASSPATH"
fi
-if ! [ -z ${PYSPARK_PYTHON+x} ]; then
- export PYSPARK_PYTHON
-fi
-if ! [ -z ${PYSPARK_DRIVER_PYTHON+x} ]; then
- export PYSPARK_DRIVER_PYTHON
-fi
-
# If HADOOP_HOME is set and SPARK_DIST_CLASSPATH is not set, set it here so Hadoop jars are available to the executor.
# It does not set SPARK_DIST_CLASSPATH if already set, to avoid overriding customizations of this value from elsewhere e.g. Docker/K8s.
if [ -n "${HADOOP_HOME}" ] && [ -z "${SPARK_DIST_CLASSPATH}" ]; then
diff --git a/3.3.1/scala2.12-java11-r-ubuntu/entrypoint.sh b/3.3.1/scala2.12-java11-r-ubuntu/entrypoint.sh
index 4bb1557..159d539 100644
--- a/3.3.1/scala2.12-java11-r-ubuntu/entrypoint.sh
+++ b/3.3.1/scala2.12-java11-r-ubuntu/entrypoint.sh
@@ -45,13 +45,6 @@ if [ -n "$SPARK_EXTRA_CLASSPATH" ]; then
SPARK_CLASSPATH="$SPARK_CLASSPATH:$SPARK_EXTRA_CLASSPATH"
fi
-if ! [ -z ${PYSPARK_PYTHON+x} ]; then
- export PYSPARK_PYTHON
-fi
-if ! [ -z ${PYSPARK_DRIVER_PYTHON+x} ]; then
- export PYSPARK_DRIVER_PYTHON
-fi
-
# If HADOOP_HOME is set and SPARK_DIST_CLASSPATH is not set, set it here so Hadoop jars are available to the executor.
# It does not set SPARK_DIST_CLASSPATH if already set, to avoid overriding customizations of this value from elsewhere e.g. Docker/K8s.
if [ -n "${HADOOP_HOME}" ] && [ -z "${SPARK_DIST_CLASSPATH}" ]; then
diff --git a/3.3.1/scala2.12-java11-ubuntu/entrypoint.sh b/3.3.1/scala2.12-java11-ubuntu/entrypoint.sh
index 4bb1557..159d539 100644
--- a/3.3.1/scala2.12-java11-ubuntu/entrypoint.sh
+++ b/3.3.1/scala2.12-java11-ubuntu/entrypoint.sh
@@ -45,13 +45,6 @@ if [ -n "$SPARK_EXTRA_CLASSPATH" ]; then
SPARK_CLASSPATH="$SPARK_CLASSPATH:$SPARK_EXTRA_CLASSPATH"
fi
-if ! [ -z ${PYSPARK_PYTHON+x} ]; then
- export PYSPARK_PYTHON
-fi
-if ! [ -z ${PYSPARK_DRIVER_PYTHON+x} ]; then
- export PYSPARK_DRIVER_PYTHON
-fi
-
# If HADOOP_HOME is set and SPARK_DIST_CLASSPATH is not set, set it here so Hadoop jars are available to the executor.
# It does not set SPARK_DIST_CLASSPATH if already set, to avoid overriding customizations of this value from elsewhere e.g. Docker/K8s.
if [ -n "${HADOOP_HOME}" ] && [ -z "${SPARK_DIST_CLASSPATH}" ]; then
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org