You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "Yikun (via GitHub)" <gi...@apache.org> on 2023/05/25 09:30:00 UTC

[GitHub] [spark-docker] Yikun opened a new pull request, #42: [SPARK-43793] Fix SPARK_EXECUTOR_JAVA_OPTS assignment bug

Yikun opened a new pull request, #42:
URL: https://github.com/apache/spark-docker/pull/42

   ### What changes were proposed in this pull request?
   In previous code, this is susceptible to a few bugs particularly around newlines in values.
   ```
   env | grep SPARK_JAVA_OPT_ | sort -t_ -k4 -n | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt
   readarray -t SPARK_EXECUTOR_JAVA_OPTS < /tmp/java_opts.txt
   ```
   
   ### Why are the changes needed?
   To address DOI comments: https://github.com/docker-library/official-images/pull/13089#issuecomment-1533540388 , question 6.
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   
   ### How was this patch tested?
   1. Test mannully
   ```
   export SPARK_JAVA_OPT_0="foo=bar"
   export SPARK_JAVA_OPT_1="foo1=bar1"
   
   for v in "${!SPARK_JAVA_OPT_@}"; do
       SPARK_EXECUTOR_JAVA_OPTS+=( "${!v}" )
   done
   
   for v in ${SPARK_EXECUTOR_JAVA_OPTS[@]}; do
       echo $v
   done
   
   # foo=bar
   # foo1=bar1
   ```
   2. CI passed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark-docker] Yikun closed pull request #42: [SPARK-43793] Fix SPARK_EXECUTOR_JAVA_OPTS assignment bug

Posted by "Yikun (via GitHub)" <gi...@apache.org>.
Yikun closed pull request #42: [SPARK-43793] Fix SPARK_EXECUTOR_JAVA_OPTS assignment bug
URL: https://github.com/apache/spark-docker/pull/42


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark-docker] Yikun commented on pull request #42: [SPARK-43793] Fix SPARK_EXECUTOR_JAVA_OPTS assignment bug

Posted by "Yikun (via GitHub)" <gi...@apache.org>.
Yikun commented on PR #42:
URL: https://github.com/apache/spark-docker/pull/42#issuecomment-1562712765

   @HyukjinKwon Thanks! merged.
   
   also cc @dongjoon-hyun 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org