[jira] [Commented] (FLINK-12953) View logs from Job view in Web Dashboard

["vinoyang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16875916#comment-16875916 ] 

vinoyang commented on FLINK-12953:
----------------------------------

Hi [~chadrik], thanks for providing a clear gif example. However, I still have some questions:
 * Does the web UI you showed is not comes from Apache Beam? You said it comes from Google Dataflow which is a product of Google Cloud. So, does the web UI of beam support this feature in your gif? Sorry, I did not use Beam before. As you mentioned in your comment, it integrated with Stackdriver. IMO Flink should not support this feature, because it overflows the scope of one project. It's the requirement of a product which can integrate Flink and Elasticsearch to meet this requirement. But, if Apache beam itself can provide this feature, that would be interesting, Flink community may consider it.
 * Currently, Flink shows log based on JM/TM view, you need to view the log for one job from the job view. Do you consider collect those logs print by tasks in many TM into one place(e.g. one web page)?
 * Some people against our opinion(FLINK-11202) because of event correlation, their thought is event correlation is important for locating issues. Actually, it's true. So it's a balance. It seems a good way is that we stay the same, but integrate with some other systems like Elasticsearch to do these enhanced features you listed.

In addition, I think we still can move one step towards improving the user experience of viewing logs, such as listing the old log files no matter split business log or not. What do you think? cc [~rmetzger] [~aljoscha]

> View logs from Job view in Web Dashboard
> ----------------------------------------
>
>                 Key: FLINK-12953
>                 URL: https://issues.apache.org/jira/browse/FLINK-12953
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Web Frontend
>            Reporter: Chad Dombrova
>            Priority: Major
>         Attachments: dataflow-log-ux.gif
>
>
> As a (beam) developer I want to be able to print/log information from my custom transforms, and then monitor that output within the job view of the Web Dashboard, so that I don't have to go hunting through the combined log in the Task Manager view.   The Task Manager log has way too much in it, spanning all jobs, including output logged by both flink and user code.
> A good example of how this UX should work can be found in Google Dataflow:  
> - click on a job, and see the logged output for that job
> - click on a transform, and see the logged output for just that transform
> thanks!
> Edited:  changed Job Manager to Task Manager



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)