You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by "Yikun (via GitHub)" <gi...@apache.org> on 2023/05/25 09:36:32 UTC

[GitHub] [spark-docker] Yikun opened a new pull request, #43: [SPARK-43370] Switch spark user only when run driver and executor

Yikun opened a new pull request, #43:
URL: https://github.com/apache/spark-docker/pull/43

   ### What changes were proposed in this pull request?
   Switch spark user only when run driver and executor
   
   ### Why are the changes needed?
   Address doi comments: question 7 [1]
   
   [1] https://github.com/docker-library/official-images/pull/13089#issuecomment-1533540388 
   [2] https://github.com/docker-library/official-images/pull/13089#issuecomment-1561793792
   
   
   ### Does this PR introduce _any_ user-facing change?
   Yes
   
   ### How was this patch tested?
   1. test mannuly
   ```
   cd ~/spark-docker/3.4.0/scala2.12-java11-ubuntu
   $ docker build . -t spark-test
   
   $ docker run -ti spark-test bash
   spark@afa78af05cf8:/opt/spark/work-dir$
   
   $ docker run  --user root  -ti spark-test bash
   root@095e0d7651fd:/opt/spark/work-dir#
   ```
   2. ci passed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark-docker] Yikun closed pull request #43: [SPARK-43370] Switch spark user only when run driver and executor

Posted by "Yikun (via GitHub)" <gi...@apache.org>.

Yikun closed pull request #43: [SPARK-43370] Switch spark user only when run driver and executor
URL: https://github.com/apache/spark-docker/pull/43


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark-docker] Yikun commented on pull request #43: [SPARK-43370] Switch spark user only when run driver and executor

Posted by "Yikun (via GitHub)" <gi...@apache.org>.

Yikun commented on PR #43:
URL: https://github.com/apache/spark-docker/pull/43#issuecomment-1571445035

   @HyukjinKwon @yosifkit Thanks for review.
   
   Merged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark-docker] yosifkit commented on a diff in pull request #43: [SPARK-43370] Switch spark user only when run driver and executor

Posted by "yosifkit (via GitHub)" <gi...@apache.org>.

yosifkit commented on code in PR #43:
URL: https://github.com/apache/spark-docker/pull/43#discussion_r1212153792


##########
3.4.0/scala2.12-java11-ubuntu/entrypoint.sh:
##########
@@ -95,20 +104,13 @@ case "$1" in
       --resourceProfileId $SPARK_RESOURCE_PROFILE_ID
       --podName $SPARK_EXECUTOR_POD_NAME
     )
+    # Execute the container CMD under tini for better hygiene
+    exec $(switch_spark_if_root) /usr/bin/tini -s -- "${CMD[@]}"
     ;;
 
   *)
     # Non-spark-on-k8s command provided, proceeding in pass-through mode...
     CMD=("$@")
+    "${CMD[@]}"

Review Comment:
   This should have `exec` as well. Simplified, it could be `exec "$@"`. (Applies here or to #44)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark-docker] Yikun commented on pull request #43: [SPARK-43370] Switch spark user only when run driver and executor

Posted by "Yikun (via GitHub)" <gi...@apache.org>.

Yikun commented on PR #43:
URL: https://github.com/apache/spark-docker/pull/43#issuecomment-1563738824

1. `spark` as default USER
* Run directly: `spark` user will be used when enter docker image
```
$ docker run -ti spark-test bash
spark@afa78af05cf8:/opt/spark/work-dir$
```
* Base image: If users want to build image based on base image, users will have to switch `root` to install extra apt pkg and then switch back to `spark`.
* In previous docker (v3.3.0), `docker run -ti apache/spark:v3.3.0 bash`, `185` user will be used.
* K8s: we pin to `spark` user.

2. `root` as default USER
* Run directly: `root` user will be used when enter docker image
```
$ docker run -ti spark-test bash
root@afa78af05cf8:/opt/spark/work-dir$
```
* Base image: If users want to build image based on base image, they can install directly
* K8s: we pin to `spark` user.

So, looks like root user is more convenient (but bring some behavior change compare to previous `apache/spark` image). Wait for feedback from DOI.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark-docker] Yikun commented on pull request #43: [SPARK-43370] Switch spark user only when run driver and executor

Posted by "Yikun (via GitHub)" <gi...@apache.org>.

Yikun commented on PR #43:
URL: https://github.com/apache/spark-docker/pull/43#issuecomment-1571280119

   According the suggestion of https://github.com/docker-library/official-images/pull/13089#issuecomment-1570733215
   
   > I recommend that if the entrypoint needs to do setup work as root before stepping down then using the default USER root is appropriate (like mysql), otherwise USER configured to an app-specific user in the Dockerfile is better from a security perspective.
   
   For spark, root USER is not required by any setup work, so I think `spark` user is enough. So I will merge this PR, and close https://github.com/apache/spark-docker/pull/44 .


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org