You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Averell <lv...@gmail.com> on 2019/03/15 03:21:29 UTC

Where does the logs in Flink GUI's Exception tab come from?

Hi everyone,

I am running Flink in EMR YARN cluster, and when the job failed and
restarted, I could see some logs in the Exception tab of Flink GUI.
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t1586/Screen_Shot_2019-03-15_at_12.png> 

I could not find this piece of logs on my cluster's hard-disk - not in TM or
JM logs.

Where can I find this?

Thanks.

Here below is my logback.xml. I'm not sure it has anything to do with my
question.

<configuration>
    <appender name="file" class="ch.qos.logback.core.FileAppender">
        <file>${log.file}</file>
        <append>false</append>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level
%logger{60} %X{sourceThread} - %msg%n</pattern>
        </encoder>
    </appender>
    
    <root level="WARN">
        <appender-ref ref="file"/>
    </root>
    <logger name="com.nbnco.csa.analysis.copper.sdc.flink.sink" level="INFO"
additivity="false">
        <appender-ref ref="file"/>
    </logger>
    <logger name="org.apache.flink.streaming.api.functions.source"
level="INFO" additivity="false">
        <appender-ref ref="file"/>
    </logger>
    <logger
name="org.apache.flink.shaded.akka.org.jboss.netty.channel.DefaultChannelPipeline"
level="ERROR">
        <appender-ref ref="file"/>
    </logger>
</configuration>
 



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Where does the logs in Flink GUI's Exception tab come from?

Posted by Averell <lv...@gmail.com>.
Hi Gary,

Thanks a lot for the explanation, and sorry for missing your earlier
message.
I am clear now.

Thanks and regards,
Averell



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Where does the logs in Flink GUI's Exception tab come from?

Posted by Gary Yao <ga...@ververica.com>.
Hi Averell,

I think I have answered your question previously [1]. The bottom line is
that
the error is logged on INFO level in the ExecutionGraph [2]. However, your
effective log level (of the root logger) is WARN. The log levels are ordered
as follows [3]:

    TRACE < DEBUG < INFO <  WARN < ERROR

It follows that, in your case, log requests to the root logger of level WARN
and ERROR only will appear in the log file – all other levels will be
discarded. It should be enough to set the root log level to INFO, or set the
log level for the org.apache.flink.runtime.executiongraph.ExecutionGraph
logger explicitly to INFO (or lower).

Best,
Gary

[1]
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/No-resource-available-error-while-testing-HA-tp25681p25835.html
[2]
https://github.com/apache/flink/blob/81acd0a490f3ac40cbb2736189796138ac109dd0/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionGraph.java#L1363
[3] https://logback.qos.ch/manual/architecture.html

On Fri, Mar 15, 2019 at 4:21 AM Averell <lv...@gmail.com> wrote:

> Hi everyone,
>
> I am running Flink in EMR YARN cluster, and when the job failed and
> restarted, I could see some logs in the Exception tab of Flink GUI.
> <
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t1586/Screen_Shot_2019-03-15_at_12.png>
>
>
> I could not find this piece of logs on my cluster's hard-disk - not in TM
> or
> JM logs.
>
> Where can I find this?
>
> Thanks.
>
> Here below is my logback.xml. I'm not sure it has anything to do with my
> question.
>
> <configuration>
>     <appender name="file" class="ch.qos.logback.core.FileAppender">
>         <file>${log.file}</file>
>         <append>false</append>
>         <encoder>
>             <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level
> %logger{60} %X{sourceThread} - %msg%n</pattern>
>         </encoder>
>     </appender>
>
>     <root level="WARN">
>         <appender-ref ref="file"/>
>     </root>
>     <logger name="com.nbnco.csa.analysis.copper.sdc.flink.sink"
> level="INFO"
> additivity="false">
>         <appender-ref ref="file"/>
>     </logger>
>     <logger name="org.apache.flink.streaming.api.functions.source"
> level="INFO" additivity="false">
>         <appender-ref ref="file"/>
>     </logger>
>     <logger
>
> name="org.apache.flink.shaded.akka.org.jboss.netty.channel.DefaultChannelPipeline"
> level="ERROR">
>         <appender-ref ref="file"/>
>     </logger>
> </configuration>
>
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>