[jira] [Commented] (FLINK-9335) Expose client logs via Flink UI

["Stephan Ewen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471675#comment-16471675 ] 

Stephan Ewen commented on FLINK-9335:
-------------------------------------

We are trying to make the Web UI less necessary by adding some features like

  - A 'library style' mode code container environments, where the main method runs in the container and then launches the Flink components from within the application.

  - A proper REST based API to submit job graphs (single HTTP POST to upload jars, start a job, ...) against existing Flink processes. In that case, the client logs would be on the machine that executes application main() method and POSTs the data flow graph to the REST endpoint.

With such a feature present, would you still rely on the Web UI to start jobs? We are thinking about discouraging that particular pattern, so would be interested to hear your opinion on that.

> Expose client logs via Flink UI
> -------------------------------
>
>                 Key: FLINK-9335
>                 URL: https://issues.apache.org/jira/browse/FLINK-9335
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Juho Autio
>            Priority: Major
>
> The logs logged by my Flink job jar _before_ *env.execute* can't be found in jobmanager log in Flink UI.
> In my case they seem to be going to /+home/hadoop/flink-1.5-SNAPSHOT/log/flink-hadoop-client-ip-10-0-10-29.log,+ for example.
> [~fhueske] said:
> {quote}It seems like good feature request to include the client logs.
> {quote}
> Implementing this may not be as trivial as just reading another log file though. As [~fhueske] commented:
> {quote}I assume that these logs are generated from a different process, i.e., the client process and not the JM or TM process.
>  Hence, they end up in a different log file and are not covered by the log collection of the UI.
>  The reason is that *this process might also be run on a machine outside of the cluster. So the client log file might not be accessible from the UI process*. 
> {quote}
> This was discussed on the user mailing list:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Application-logs-missing-from-jobmanager-log-tp19830p19882.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)