You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by jhereth <jh...@quda.de> on 2017/07/10 12:01:11 UTC

[Deploy] Should we make spark-daemon.sh busybox compatible?

Hi,

working on a dockerized Spark deployment I noticed the script sbin/spark-daemon.sh does not play nice with busybox.

The call "ps -p <pid>" [1] and the syntax "nohup -- <cmd>" [2] do not work with busybox.

Using an alpine based setup [1] requires installing the procps package (400KB installed size) and [2] the coreutils (5.5MB installed size). This is not much but adds to the image size and imposes more requirements for Spark to run.

These issues seem local to this script (I found no other occurences of the above syntax). Should we rewrite these lines?

Best regards,

Joachim


[1] https://github.com/apache/spark/blob/master/sbin/spark-daemon.sh#L136, https://github.com/apache/spark/blob/master/sbin/spark-daemon.sh#L144, ...

[2] https://github.com/apache/spark/blob/master/sbin/spark-daemon.sh#L128