You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by ruiyun wan <wr...@gmail.com> on 2022/04/26 09:21:36 UTC
NO jobmanager.log for yarn-per-job submitted with sql-client.sh
Flink Version (1.13)
There is not jobmanager.log file when submit yarn-per-job with
sql-client.sh。(set execution.target = yarn-per-job;) There are not
-Dlog.file and -Dlog4j.configuration properties argument in the
auto-generated launch_container.sh on YARN cluster。e.g. the last line of
launch_container.sh is :
exec /bin/bash -c "$JAVA_HOME/bin/java -Xmx50331648 -Xms50331648 \
-XX:MaxMetaspaceSize=134217728 \
org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint .....
However,When use yarn-session.sh create a session cluster,the last line of
auto generated launch_container.sh is
exec /bin/bash -c "$JAVA_HOME/bin/java -Xmx1530082096 -Xms1530082096 \
-XX:MaxMetaspaceSize=268435456
-Dlog.file="/mnt/disk3/log/hadoop-yarn/containers/application_1640228329925_0008/container_e05_1640228329925_0008_01_000001/jobmanager.log"
\
-Dlog4j.configuration=file:log4j.properties \
-Dlog4j.configurationFile=file:log4j.properties \
org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint .....
which contained -Dlog.file and -Dlog4j.configuration
The question is :how to configure sql-client.sh to produce jobmanager.log
for yarn-per-job?
Re: NO jobmanager.log for yarn-per-job submitted with sql-client.sh
Posted by yu'an huang <h....@gmail.com>.
Hi ruiyun,
Is there any log4j configuration file or logback.xml in your Flink configuration directory? Yarn should check whether the file exists first and set relative log options.
> On 26 Apr 2022, at 5:21 PM, ruiyun wan <wr...@gmail.com> wrote:
>
> Flink Version (1.13)
> There is not jobmanager.log file when submit yarn-per-job with sql-client.sh。(set execution.target = yarn-per-job;) There are not -Dlog.file and -Dlog4j.configuration properties argument in the auto-generated launch_container.sh on YARN cluster。e.g. the last line of launch_container.sh is :
> exec /bin/bash -c "$JAVA_HOME/bin/java -Xmx50331648 -Xms50331648 \
> -XX:MaxMetaspaceSize=134217728 \
> org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint .....
>
> However,When use yarn-session.sh create a session cluster,the last line of auto generated launch_container.sh is
> exec /bin/bash -c "$JAVA_HOME/bin/java -Xmx1530082096 -Xms1530082096 \
> -XX:MaxMetaspaceSize=268435456
> -Dlog.file="/mnt/disk3/log/hadoop-yarn/containers/application_1640228329925_0008/container_e05_1640228329925_0008_01_000001/jobmanager.log" \
> -Dlog4j.configuration=file:log4j.properties \
> -Dlog4j.configurationFile=file:log4j.properties \
> org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint .....
> which contained -Dlog.file and -Dlog4j.configuration
>
> The question is :how to configure sql-client.sh to produce jobmanager.log for yarn-per-job?