You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "Yikun (via GitHub)" <gi...@apache.org> on 2023/06/01 08:17:25 UTC
[GitHub] [spark-docker] Yikun opened a new pull request, #45: [SPARK-43368] Use `libnss_wrapper` to fake passwd entry
Yikun opened a new pull request, #45:
URL: https://github.com/apache/spark-docker/pull/45
### What changes were proposed in this pull request?
Use `libnss_wrapper` to fake passwd entry instead of changing passwd to resolve random UID problem.
### Why are the changes needed?
In the past, we add the entry to `/etc/passwd` directly for current UID, it's mainly for [OpenShift anonymous random `uid` case](https://github.com/docker-library/official-images/pull/13089#issuecomment-1534706523) (See also in https://github.com/apache-spark-on-k8s/spark/pull/404), but this way bring the pontential security issue about widely permision of `/etc/passwd`.
According to DOI reviewer [suggestion](https://github.com/docker-library/official-images/pull/13089#issuecomment-1561793792), we'd better to resolve this problem by using [libnss_wrapper](https://cwrap.org/nss_wrapper.html). It's a library to help set a fake passwd entry by setting `LD_PRELOAD`, `NSS_WRAPPER_PASSWD`, `NSS_WRAPPER_GROUP`. Such as random UID is `1000`, the env will be:
```
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $LD_PRELOAD
/usr/lib/libnss_wrapper.so
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $NSS_WRAPPER_PASSWD
/tmp/tmp.r5x4SMX35B
spark@6f41b8e5be9b:/opt/spark/work-dir$ cat /tmp/tmp.r5x4SMX35B
spark:x:1000:1000:${SPARK_USER_NAME:-anonymous uid}:/opt/spark:/bin/false
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $NSS_WRAPPER_GROUP
/tmp/tmp.XcnnYuD68r
spark@6f41b8e5be9b:/opt/spark/work-dir$ cat /tmp/tmp.XcnnYuD68r
spark:x:1000:
```
### Does this PR introduce _any_ user-facing change?
Yes, setup fake ENV rather than changing `/etc/passwd`.
### How was this patch tested?
1. without `attempt_setup_fake_passwd_entry`, the user is `I have no name!`
```
# docker run -it --rm --user 1000:1000 spark-test bash
groups: cannot find name for group ID 1000
I have no name!@998110cd5a26:/opt/spark/work-dir$
```
2. stub the `attempt_setup_fake_passwd_entry` to see it works or not.
2.1 Apply a tmp change to cmd
```patch
diff --git a/entrypoint.sh.template b/entrypoint.sh.template
index 08fc925..77d5b04 100644
--- a/entrypoint.sh.template
+++ b/entrypoint.sh.template
@@ -118,6 +118,7 @@ case "$1" in
*)
# Non-spark-on-k8s command provided, proceeding in pass-through mode...
+ attempt_setup_fake_passwd_entry
exec "$@"
;;
esac
```
2.2 Build and run the image, specify a random UID/GID 1000
```bash
$ docker build . -t spark-test
$ docker run -it --rm --user 1000:1000 spark-test bash
# the user is set to spark rather than unknow user
spark@6f41b8e5be9b:/opt/spark/work-dir$
# NSS env is set right
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $LD_PRELOAD
/usr/lib/libnss_wrapper.so
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $NSS_WRAPPER_PASSWD
/tmp/tmp.r5x4SMX35B
spark@6f41b8e5be9b:/opt/spark/work-dir$ cat /tmp/tmp.r5x4SMX35B
spark:x:1000:1000:${SPARK_USER_NAME:-anonymous uid}:/opt/spark:/bin/false
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $NSS_WRAPPER_GROUP
/tmp/tmp.XcnnYuD68r
spark@6f41b8e5be9b:/opt/spark/work-dir$ cat /tmp/tmp.XcnnYuD68r
spark:x:1000:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-docker] Yikun commented on pull request #45: [SPARK-43368] Use `libnss_wrapper` to fake passwd entry
Posted by "Yikun (via GitHub)" <gi...@apache.org>.
Yikun commented on PR #45:
URL: https://github.com/apache/spark-docker/pull/45#issuecomment-1571612066
cc @yosifkit Thanks for the solution!
cc @pan3793 @dongjoon-hyun @erikerlandson (who involved spark on k8s user related work)
also cc @HyukjinKwon @zhengruifeng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-docker] Yikun commented on pull request #45: [SPARK-43368] Use `libnss_wrapper` to fake passwd entry
Posted by "Yikun (via GitHub)" <gi...@apache.org>.
Yikun commented on PR #45:
URL: https://github.com/apache/spark-docker/pull/45#issuecomment-1573047801
@HyukjinKwon @pan3793 Thanks, I will merge this soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-docker] Yikun closed pull request #45: [SPARK-43368] Use `libnss_wrapper` to fake passwd entry
Posted by "Yikun (via GitHub)" <gi...@apache.org>.
Yikun closed pull request #45: [SPARK-43368] Use `libnss_wrapper` to fake passwd entry
URL: https://github.com/apache/spark-docker/pull/45
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-docker] Yikun commented on pull request #45: [SPARK-43368] Use `libnss_wrapper` to fake passwd entry
Posted by "Yikun (via GitHub)" <gi...@apache.org>.
Yikun commented on PR #45:
URL: https://github.com/apache/spark-docker/pull/45#issuecomment-1573048404
Merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-43368] Use `libnss_wrapper` to fake passwd entry [spark-docker]
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on PR #45:
URL: https://github.com/apache/spark-docker/pull/45#issuecomment-3248061571
Hi @Yikun,
This change has also been adopted by the Spark operator. See https://github.com/kubeflow/spark-operator/pull/2312
I am an Openshift user myself and therefore ran into a regression. See: https://github.com/kubeflow/spark-operator/issues/2644
Have you also tested your commands for the home directory?
I tested the home directory with the following small Java class. I don't know how Spark evaluates this.
```
import javax.swing.filechooser.FileSystemView;
import java.io.File;
class Home {
public static void main(String... args) {
FileSystemView view = FileSystemView.getFileSystemView();
File file = view.getHomeDirectory();
String desktopPath = file.getPath();
System.out.println(desktopPath);
}
}
```
The output of the small program returns the following, which is obviously incorrect.
```
java Home
-anonymous uid}
```
Is there a reason to evaluate the GECOS field later?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-43368] Use `libnss_wrapper` to fake passwd entry [spark-docker]
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on PR #45:
URL: https://github.com/apache/spark-docker/pull/45#issuecomment-3248120664
Pull Request with a possible fix. https://github.com/apache/spark-docker/pull/45
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org