You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Daniel Imberman (Jira)" <ji...@apache.org> on 2020/03/30 00:37:00 UTC
[jira] [Updated] (AIRFLOW-849) Show whether spark job is "running"
or "waiting"
[ https://issues.apache.org/jira/browse/AIRFLOW-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Imberman updated AIRFLOW-849:
------------------------------------
Description:
As of now, Airflow shows that a task is "running" in Airflow when it could be listed as "WAITING" in the Spark UI (and can be listed as "WAITING" for hours when the Airflow UI might indicate it is running if I forget to check the Spark UI).
Would be awesome once ticket 802 for spark integration is complete to add this enhancement of more granular detail about the task.
Note (Daniel 3/29/20): We should add an "external" field to task state to store task-specific data like spark cluster state
was:
As of now, Airflow shows that a task is "running" in Airflow when it could be listed as "WAITING" in the Spark UI (and can be listed as "WAITING" for hours when the Airflow UI might indicate it is running if I forget to check the Spark UI).
Would be awesome once ticket 802 for spark integration is complete to add this enhancement of more granular detail about the task.
> Show whether spark job is "running" or "waiting"
> ------------------------------------------------
>
> Key: AIRFLOW-849
> URL: https://issues.apache.org/jira/browse/AIRFLOW-849
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ui
> Reporter: Daniel Blazevski
> Priority: Minor
>
> As of now, Airflow shows that a task is "running" in Airflow when it could be listed as "WAITING" in the Spark UI (and can be listed as "WAITING" for hours when the Airflow UI might indicate it is running if I forget to check the Spark UI).
> Would be awesome once ticket 802 for spark integration is complete to add this enhancement of more granular detail about the task.
> Note (Daniel 3/29/20): We should add an "external" field to task state to store task-specific data like spark cluster state
--
This message was sent by Atlassian Jira
(v8.3.4#803005)