You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Siddharth Anand (JIRA)" <ji...@apache.org> on 2016/10/10 22:43:21 UTC
[jira] [Updated] (AIRFLOW-562) Expose failed task_instances in the
Task_Instances view
[ https://issues.apache.org/jira/browse/AIRFLOW-562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Anand updated AIRFLOW-562:
------------------------------------
Description:
Currently (in master), the Task_instance page (Browse -> Task Instance) do not show tasks that were retried. In order to reveal retried tasks, you must look in the task_fail table as well.
Refer to the email thread below:
----------------------------------------
START OF EMAIL THREAD
----------------------------------------
Yes.
The task instance table only has one row per unique key : (dag_id, task_id, execution_date). To support the "cumulative duration" functionality, we had to write "failed task instance rows" to a new table called task_fail (a.k.a. TaskFail in models.py). That second table is not currently read to build the task_instances view (in views.py).
This should IMHO be a feature of the task_intances view - anyone want to submit a pull request for this feature : https://issues.apache.org/jira/browse/AIRFLOW-562?
-s
On Mon, Oct 10, 2016 at 1:23 PM, Colin Son <co...@thinknear.com> wrote:
Hello,
In the article that you sent me, it states this:
"As mentioned earlier, the current Task Duration chart is misleading. For
example, in the chart below, it appears as if each of our 3 tasks is
performing in a nearly consistent manner from run to run. However, what
happens if there are transient failures and retries? Unfortunately, the
chart below only displays the Task Duration for successful attempts and not
the cumulative time taken for that task to eventually succeed!"
Does this imply that the UI Page under Browse -> Task Instances, will also
only display successful attempts, when the task succeeds on subsequent
retries?
Thanks
On Mon, Oct 10, 2016 at 1:06 PM, siddharth anand <sa...@apache.org> wrote:
> Colin,
>
> Have a look at the link below:
> https://www.agari.com/a-summer-interns-journey-into-airflow-agari/
>
> We committed some changes to master over the Summer that provide a few new
> UI controls:
> * A new Task Tries view that plots Task Tries as a function of time for
> each task in a DAG
> * A Cumulative Duration checkbox to show the total time a task takes,
> including all retries, not just the time taken for the task's eventually
> successful run.
>
> -s
>
> On Mon, Oct 10, 2016 at 12:54 PM, Colin Son <co...@thinknear.com> wrote:
>
> > Hello,
> >
> > Whenever a Task Instance fails, retries, and succeeds, I noticed that the
> > WEB UI doesn't show you that it failed/retried. You would have to look in
> > the logs of that failed Task Instance and see if that Task Instance
> failed
> > or not. How come it doesn't show that the Task Instance's state initially
> > was failed/retried in the Task Instances page in the Web UI? I noticed
> that
> > it marks the state as "Success", even after it failed/retried/and
> > succeeded. Is there a way to change this?
> >
> > Thanks
> >
>
----------------------------------------
END OF EMAIL THREAD
----------------------------------------
was:
Currently (in master), the Task_instance page (Browse -> Task Instance) do not show tasks that were retried. In order to reveal retried tasks, you must look in the task_fail table as well.
Refer to the email thread below:
> Expose failed task_instances in the Task_Instances view
> --------------------------------------------------------
>
> Key: AIRFLOW-562
> URL: https://issues.apache.org/jira/browse/AIRFLOW-562
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Siddharth Anand
> Priority: Minor
>
> Currently (in master), the Task_instance page (Browse -> Task Instance) do not show tasks that were retried. In order to reveal retried tasks, you must look in the task_fail table as well.
> Refer to the email thread below:
> ----------------------------------------
> START OF EMAIL THREAD
> ----------------------------------------
> Yes.
> The task instance table only has one row per unique key : (dag_id, task_id, execution_date). To support the "cumulative duration" functionality, we had to write "failed task instance rows" to a new table called task_fail (a.k.a. TaskFail in models.py). That second table is not currently read to build the task_instances view (in views.py).
> This should IMHO be a feature of the task_intances view - anyone want to submit a pull request for this feature : https://issues.apache.org/jira/browse/AIRFLOW-562?
> -s
> On Mon, Oct 10, 2016 at 1:23 PM, Colin Son <co...@thinknear.com> wrote:
> Hello,
> In the article that you sent me, it states this:
> "As mentioned earlier, the current Task Duration chart is misleading. For
> example, in the chart below, it appears as if each of our 3 tasks is
> performing in a nearly consistent manner from run to run. However, what
> happens if there are transient failures and retries? Unfortunately, the
> chart below only displays the Task Duration for successful attempts and not
> the cumulative time taken for that task to eventually succeed!"
> Does this imply that the UI Page under Browse -> Task Instances, will also
> only display successful attempts, when the task succeeds on subsequent
> retries?
> Thanks
> On Mon, Oct 10, 2016 at 1:06 PM, siddharth anand <sa...@apache.org> wrote:
> > Colin,
> >
> > Have a look at the link below:
> > https://www.agari.com/a-summer-interns-journey-into-airflow-agari/
> >
> > We committed some changes to master over the Summer that provide a few new
> > UI controls:
> > * A new Task Tries view that plots Task Tries as a function of time for
> > each task in a DAG
> > * A Cumulative Duration checkbox to show the total time a task takes,
> > including all retries, not just the time taken for the task's eventually
> > successful run.
> >
> > -s
> >
> > On Mon, Oct 10, 2016 at 12:54 PM, Colin Son <co...@thinknear.com> wrote:
> >
> > > Hello,
> > >
> > > Whenever a Task Instance fails, retries, and succeeds, I noticed that the
> > > WEB UI doesn't show you that it failed/retried. You would have to look in
> > > the logs of that failed Task Instance and see if that Task Instance
> > failed
> > > or not. How come it doesn't show that the Task Instance's state initially
> > > was failed/retried in the Task Instances page in the Web UI? I noticed
> > that
> > > it marks the state as "Success", even after it failed/retried/and
> > > succeeded. Is there a way to change this?
> > >
> > > Thanks
> > >
> >
> ----------------------------------------
> END OF EMAIL THREAD
> ----------------------------------------
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)