You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Lance Norskog <la...@gmail.com> on 2016/05/06 21:59:29 UTC
'shutdown' state?
What is the 'shutdown' (blue rim) state? How does a task get to that state?
--
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA
Re: 'shutdown' state?
Posted by Chris Riccomini <cr...@apache.org>.
Thanks!
On Fri, May 6, 2016 at 4:23 PM, Lance Norskog <la...@gmail.com>
wrote:
> https://issues.apache.org/jira/browse/AIRFLOW-66
>
> It's very hard to get the experts to make a definitive description with
> diagrams. I used to be a Solr/Lucene consultant & worked with the
> committers, and they were willing to explain the same topic 100 times on
> the mailing list but would never make a page with diagrams that they could
> point to.
>
>
>
> On Fri, May 6, 2016 at 4:09 PM, Chris Riccomini <cr...@apache.org>
> wrote:
>
> > @Lance, mind opening a JIRA? Would be a good newb ticket.
> >
> > On Fri, May 6, 2016 at 4:07 PM, Lance Norskog <la...@gmail.com>
> > wrote:
> >
> > > Thanks!
> > >
> > > I ask here because the Search box on pythonhosted's Airflow docs has no
> > > hits for 'shutdown'.
> > >
> > > A sequence diagram and detailed explanation of each state would be
> useful
> > > documentation :)
> > >
> > > Cheers,
> > >
> > > Lance Norskog
> > >
> > >
> > > On Fri, May 6, 2016 at 3:40 PM, Maxime Beauchemin <
> > > maximebeauchemin@gmail.com> wrote:
> > >
> > > > It's a poison pill, basically the task instance polls at its own
> state
> > to
> > > > look for a shutdown state and kills itself if it sees it. It will
> then
> > > > proceed to send alert emails if any are set up, run the operator's
> > > > `on_kill` method and set its own state to failed.
> > > >
> > > > Max
> > > >
> > > > On Fri, May 6, 2016 at 3:01 PM, Jeremiah Lowin <jl...@apache.org>
> > > wrote:
> > > >
> > > > > Tasks (and jobs) are put in that state if you clear them
> > > > > (clear_task_instances() or “airflow clear”) while they’re running.
> > > > >
> > > > > Jeremiah
> > > > >
> > > > > On Fri, May 6, 2016 at 5:59 PM Lance Norskog <
> > lance.norskog@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > What is the 'shutdown' (blue rim) state? How does a task get to
> > that
> > > > > state?
> > > > > >
> > > > > > --
> > > > > > Lance Norskog
> > > > > > lance.norskog@gmail.com
> > > > > > Redwood City, CA
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Lance Norskog
> > > lance.norskog@gmail.com
> > > Redwood City, CA
> > >
> >
>
>
>
> --
> Lance Norskog
> lance.norskog@gmail.com
> Redwood City, CA
>
Re: 'shutdown' state?
Posted by Lance Norskog <la...@gmail.com>.
https://issues.apache.org/jira/browse/AIRFLOW-66
It's very hard to get the experts to make a definitive description with
diagrams. I used to be a Solr/Lucene consultant & worked with the
committers, and they were willing to explain the same topic 100 times on
the mailing list but would never make a page with diagrams that they could
point to.
On Fri, May 6, 2016 at 4:09 PM, Chris Riccomini <cr...@apache.org>
wrote:
> @Lance, mind opening a JIRA? Would be a good newb ticket.
>
> On Fri, May 6, 2016 at 4:07 PM, Lance Norskog <la...@gmail.com>
> wrote:
>
> > Thanks!
> >
> > I ask here because the Search box on pythonhosted's Airflow docs has no
> > hits for 'shutdown'.
> >
> > A sequence diagram and detailed explanation of each state would be useful
> > documentation :)
> >
> > Cheers,
> >
> > Lance Norskog
> >
> >
> > On Fri, May 6, 2016 at 3:40 PM, Maxime Beauchemin <
> > maximebeauchemin@gmail.com> wrote:
> >
> > > It's a poison pill, basically the task instance polls at its own state
> to
> > > look for a shutdown state and kills itself if it sees it. It will then
> > > proceed to send alert emails if any are set up, run the operator's
> > > `on_kill` method and set its own state to failed.
> > >
> > > Max
> > >
> > > On Fri, May 6, 2016 at 3:01 PM, Jeremiah Lowin <jl...@apache.org>
> > wrote:
> > >
> > > > Tasks (and jobs) are put in that state if you clear them
> > > > (clear_task_instances() or “airflow clear”) while they’re running.
> > > >
> > > > Jeremiah
> > > >
> > > > On Fri, May 6, 2016 at 5:59 PM Lance Norskog <
> lance.norskog@gmail.com>
> > > > wrote:
> > > >
> > > > > What is the 'shutdown' (blue rim) state? How does a task get to
> that
> > > > state?
> > > > >
> > > > > --
> > > > > Lance Norskog
> > > > > lance.norskog@gmail.com
> > > > > Redwood City, CA
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Lance Norskog
> > lance.norskog@gmail.com
> > Redwood City, CA
> >
>
--
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA
Re: 'shutdown' state?
Posted by Chris Riccomini <cr...@apache.org>.
@Lance, mind opening a JIRA? Would be a good newb ticket.
On Fri, May 6, 2016 at 4:07 PM, Lance Norskog <la...@gmail.com>
wrote:
> Thanks!
>
> I ask here because the Search box on pythonhosted's Airflow docs has no
> hits for 'shutdown'.
>
> A sequence diagram and detailed explanation of each state would be useful
> documentation :)
>
> Cheers,
>
> Lance Norskog
>
>
> On Fri, May 6, 2016 at 3:40 PM, Maxime Beauchemin <
> maximebeauchemin@gmail.com> wrote:
>
> > It's a poison pill, basically the task instance polls at its own state to
> > look for a shutdown state and kills itself if it sees it. It will then
> > proceed to send alert emails if any are set up, run the operator's
> > `on_kill` method and set its own state to failed.
> >
> > Max
> >
> > On Fri, May 6, 2016 at 3:01 PM, Jeremiah Lowin <jl...@apache.org>
> wrote:
> >
> > > Tasks (and jobs) are put in that state if you clear them
> > > (clear_task_instances() or “airflow clear”) while they’re running.
> > >
> > > Jeremiah
> > >
> > > On Fri, May 6, 2016 at 5:59 PM Lance Norskog <la...@gmail.com>
> > > wrote:
> > >
> > > > What is the 'shutdown' (blue rim) state? How does a task get to that
> > > state?
> > > >
> > > > --
> > > > Lance Norskog
> > > > lance.norskog@gmail.com
> > > > Redwood City, CA
> > > >
> > >
> >
>
>
>
> --
> Lance Norskog
> lance.norskog@gmail.com
> Redwood City, CA
>
Re: 'shutdown' state?
Posted by Lance Norskog <la...@gmail.com>.
Thanks!
I ask here because the Search box on pythonhosted's Airflow docs has no
hits for 'shutdown'.
A sequence diagram and detailed explanation of each state would be useful
documentation :)
Cheers,
Lance Norskog
On Fri, May 6, 2016 at 3:40 PM, Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:
> It's a poison pill, basically the task instance polls at its own state to
> look for a shutdown state and kills itself if it sees it. It will then
> proceed to send alert emails if any are set up, run the operator's
> `on_kill` method and set its own state to failed.
>
> Max
>
> On Fri, May 6, 2016 at 3:01 PM, Jeremiah Lowin <jl...@apache.org> wrote:
>
> > Tasks (and jobs) are put in that state if you clear them
> > (clear_task_instances() or “airflow clear”) while they’re running.
> >
> > Jeremiah
> >
> > On Fri, May 6, 2016 at 5:59 PM Lance Norskog <la...@gmail.com>
> > wrote:
> >
> > > What is the 'shutdown' (blue rim) state? How does a task get to that
> > state?
> > >
> > > --
> > > Lance Norskog
> > > lance.norskog@gmail.com
> > > Redwood City, CA
> > >
> >
>
--
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA
Re: 'shutdown' state?
Posted by Maxime Beauchemin <ma...@gmail.com>.
It's a poison pill, basically the task instance polls at its own state to
look for a shutdown state and kills itself if it sees it. It will then
proceed to send alert emails if any are set up, run the operator's
`on_kill` method and set its own state to failed.
Max
On Fri, May 6, 2016 at 3:01 PM, Jeremiah Lowin <jl...@apache.org> wrote:
> Tasks (and jobs) are put in that state if you clear them
> (clear_task_instances() or “airflow clear”) while they’re running.
>
> Jeremiah
>
> On Fri, May 6, 2016 at 5:59 PM Lance Norskog <la...@gmail.com>
> wrote:
>
> > What is the 'shutdown' (blue rim) state? How does a task get to that
> state?
> >
> > --
> > Lance Norskog
> > lance.norskog@gmail.com
> > Redwood City, CA
> >
>
Re: 'shutdown' state?
Posted by Jeremiah Lowin <jl...@apache.org>.
Tasks (and jobs) are put in that state if you clear them
(clear_task_instances() or “airflow clear”) while they’re running.
Jeremiah
On Fri, May 6, 2016 at 5:59 PM Lance Norskog <la...@gmail.com>
wrote:
> What is the 'shutdown' (blue rim) state? How does a task get to that state?
>
> --
> Lance Norskog
> lance.norskog@gmail.com
> Redwood City, CA
>