You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2019/05/17 22:22:00 UTC
[jira] [Comment Edited] (HADOOP-16312) Remove dumb-init from
hadoop-runner image
[ https://issues.apache.org/jira/browse/HADOOP-16312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16842649#comment-16842649 ]
Eric Yang edited comment on HADOOP-16312 at 5/17/19 10:21 PM:
--------------------------------------------------------------
{quote}Do you suggest to keep dumb-init and use exec?{quote}
Dumb-init only works one way to push container into background execution, and the current output looks like this on my system:
{code}
hadoop 1 0.0 0.0 188 4 ? Ss 22:11 0:00 /usr/local/bin/dumb-init -- /opt/starter.sh /opt/hadoop/bin/ozone datanode PATH=/usr/local/sbin:/usr/
hadoop 6 0.0 0.0 11680 1500 ? Ss 22:11 0:00 bash /opt/starter.sh /opt/hadoop/bin/ozone datanode PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/us
hadoop 13 8.0 3.6 4685828 369912 ? Sl 22:11 0:11 \_ /usr/lib/jvm/jre//bin/java -Dproc_datanode -Djava.net.preferIPv4Stack=true -Dlog4j.configurationF
{code}
Bash between dumb-init and java will absorb all signal communications betweeen dumb-init and java. It is not working as intended. In addition, to improve debug experience in YARN, Kitematic and K8s, it is better to start the execution in foreground. This allows log aggregators to collect script output. If user really want to demonize the container, use docker -d option explicitly.
This means don't use dumb-init when bash -c set -e will cover what we try to accomplish with dumb-init.
was (Author: eyang):
{quote}Do you suggest to keep dumb-init and use exec?{quote}
Dumb-init only works one way to push container into background execution. To improve debug experience in YARN, Kitematic and K8s, it is better to start the execution in foreground. This allows log aggregators to collect script output. If user really want to demonize the container, use docker -d option explicitly.
> Remove dumb-init from hadoop-runner image
> -----------------------------------------
>
> Key: HADOOP-16312
> URL: https://issues.apache.org/jira/browse/HADOOP-16312
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Elek, Marton
> Priority: Major
>
> This issue is reported by [~eyang] in HDDS-1495.
> I think it's better to discuss under a separated issue as it's unrelated to HDDS-1495.
> The original problem description from [~eyang]
> {quote}Dumb-init is one way to always run contaized program in the background and respawn the program when program fails. This is poor man’s solution for keeping program alive.
> Cluster management software like Kubernetes or YARN have additional policy and logic to start the same docker container on a different node. Therefore, Dumb-init is not recommended for future Hadoop daemons instead allow cluster management software to make decision where to start the container. Dumb-init for demonize docker container will be removed, and change to use entrypoint.sh Docker provides -d flag to demonize foreground process. Most of the management system built on top of Docker, (ie. Kitematic, Apache YARN, and Kubernetes) integrates with Docker container at foreground to aggregate stdout and stderr output of the containerized program.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org