You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Saisai Shao (JIRA)" <ji...@apache.org> on 2016/01/22 02:29:39 UTC
[jira] [Comment Edited] (SPARK-12140) Support Streaming UI in
HistoryServer
[ https://issues.apache.org/jira/browse/SPARK-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111656#comment-15111656 ]
Saisai Shao edited comment on SPARK-12140 at 1/22/16 1:29 AM:
--------------------------------------------------------------
Hi guys, I though a bit on this feature, besides this one big concern [~tdas] mentioned above, I think to support this feature we should factor out some parts of streaming code:
1. Make streaming event be accepted in SparkListener, so that it could be written out to event log, this part is partially done in SPARK-12874.
2. Make {{StreamingJobProgressListener}} be independent of {{StreamingContext}}, so that it can be used to replay the event-log.
3. Make Streaming UI be served both for live and history UI, currently it can only be used as for live UI (binding to {{StreamingContext}}).
I'm not sure if I miss any other part. We could create a separate sub JIRA to track part 2 and 3, any comment is greatly appreciated. Thanks a lot.
was (Author: jerryshao):
Hi guys, I though a bit on this feature, besides this one big concern [~tdas] mentioned above, I think to support this feature we should factor out some parts of streaming code:
1. Make streaming event be accepted in SparkListener, so that it could be written out to event log, this part have been done in SPARK-12874.
2. Make {{StreamingJobProgressListener}} be independent of {{StreamingContext}}, so that it can be used to replay the event-log.
3. Make Streaming UI be served both for live and history UI, currently it can only be used as for live UI (binding to {{StreamingContext}}).
I'm not sure if I miss any other part. We could create a separate sub JIRA to track part 2 and 3, any comment is greatly appreciated. Thanks a lot.
> Support Streaming UI in HistoryServer
> -------------------------------------
>
> Key: SPARK-12140
> URL: https://issues.apache.org/jira/browse/SPARK-12140
> Project: Spark
> Issue Type: New Feature
> Components: Streaming
> Affects Versions: 1.6.0
> Reporter: Marcelo Vanzin
>
> SPARK-11206 added infrastructure that would allow the streaming UI to be shown in the History Server. We should add the necessary code to make that happen, although it requires some changes to how events and listeners are used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org