You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Joshua Fan <jo...@gmail.com> on 2018/12/26 10:18:44 UTC
no log exists in JM and TM when updated to Flink 1.7
Hi,
It is very weird that there is no log file for JM and TM when run flink job
on yarn after updated flink to 1.7.on Flink 1.4.2, everything is OK. I
checked the log directory, there were jobmanager.error and jobmanager.out,
but without jobmanager.log, but the log message which should exist in
jobmanager.log now shows up in jobmanager.error. The taskmanager has the
same situation, no taskmanager.log but information exists in
taskmanager.error.
below is the container lauch shell, and it seems ok.
yarn 21784 0.0 0.0 109896 1480 ? Ss 15:13 0:00 /bin/bash
-c /home/yarn/software/java8/bin/java -Xmx424m
-Dlog.file=/data03/yarn/userlogs/application_1543893582405_1760130/container_e124_1543893582405_1760130_02_000001/jobmanager.log
-Dlogback.configurationFile=file:logback.xml
org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint 1>
/data03/yarn/userlogs/application_1543893582405_1760130/container_e124_1543893582405_1760130_02_000001/jobmanager.out
2>
/data03/yarn/userlogs/application_1543893582405_1760130/container_e124_1543893582405_1760130_02_000001/
any hints?Thanks a lot.
Yours
Joshua
Re: no log exists in JM and TM when updated to Flink 1.7
Posted by Joshua Fan <jo...@gmail.com>.
Hi Till
I found the root cause why log-not-show when use logback, because flink
does not include the logback-*.jar in the lib folder.
After I put the logback jar file in lib, everything is ok now.
I think flink should put the logback jar files into the lib directory, not
just the log4j jar file, because the both log configuration file exist in
conf directory.
Yours sincerely
Joshua
On Wed, Jan 2, 2019 at 10:08 PM Till Rohrmann <tr...@apache.org> wrote:
> Hi Joshua,
>
> could you check the content of the logback.xml. Maybe this file has
> changed between the versions.
>
> Cheers,
> Till
>
> On Wed, Dec 26, 2018 at 11:19 AM Joshua Fan <jo...@gmail.com>
> wrote:
>
>> Hi,
>>
>> It is very weird that there is no log file for JM and TM when run flink
>> job on yarn after updated flink to 1.7.on Flink 1.4.2, everything is OK. I
>> checked the log directory, there were jobmanager.error and jobmanager.out,
>> but without jobmanager.log, but the log message which should exist in
>> jobmanager.log now shows up in jobmanager.error. The taskmanager has the
>> same situation, no taskmanager.log but information exists in
>> taskmanager.error.
>>
>> below is the container lauch shell, and it seems ok.
>>
>> yarn 21784 0.0 0.0 109896 1480 ? Ss 15:13 0:00
>> /bin/bash -c /home/yarn/software/java8/bin/java -Xmx424m
>> -Dlog.file=/data03/yarn/userlogs/application_1543893582405_1760130/container_e124_1543893582405_1760130_02_000001/jobmanager.log
>> -Dlogback.configurationFile=file:logback.xml
>> org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint 1>
>> /data03/yarn/userlogs/application_1543893582405_1760130/container_e124_1543893582405_1760130_02_000001/jobmanager.out
>> 2>
>> /data03/yarn/userlogs/application_1543893582405_1760130/container_e124_1543893582405_1760130_02_000001/
>>
>> any hints?Thanks a lot.
>>
>> Yours
>> Joshua
>>
>
Re: no log exists in JM and TM when updated to Flink 1.7
Posted by Till Rohrmann <tr...@apache.org>.
Hi Joshua,
could you check the content of the logback.xml. Maybe this file has changed
between the versions.
Cheers,
Till
On Wed, Dec 26, 2018 at 11:19 AM Joshua Fan <jo...@gmail.com> wrote:
> Hi,
>
> It is very weird that there is no log file for JM and TM when run flink
> job on yarn after updated flink to 1.7.on Flink 1.4.2, everything is OK. I
> checked the log directory, there were jobmanager.error and jobmanager.out,
> but without jobmanager.log, but the log message which should exist in
> jobmanager.log now shows up in jobmanager.error. The taskmanager has the
> same situation, no taskmanager.log but information exists in
> taskmanager.error.
>
> below is the container lauch shell, and it seems ok.
>
> yarn 21784 0.0 0.0 109896 1480 ? Ss 15:13 0:00 /bin/bash
> -c /home/yarn/software/java8/bin/java -Xmx424m
> -Dlog.file=/data03/yarn/userlogs/application_1543893582405_1760130/container_e124_1543893582405_1760130_02_000001/jobmanager.log
> -Dlogback.configurationFile=file:logback.xml
> org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint 1>
> /data03/yarn/userlogs/application_1543893582405_1760130/container_e124_1543893582405_1760130_02_000001/jobmanager.out
> 2>
> /data03/yarn/userlogs/application_1543893582405_1760130/container_e124_1543893582405_1760130_02_000001/
>
> any hints?Thanks a lot.
>
> Yours
> Joshua
>