You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Lance Norskog <la...@gmail.com> on 2016/07/29 23:35:21 UTC

About Pools and Task Retries

When a Task in in a Pool, fails, and goes to sleep to retry, does it give
up its Pool membership while asleep?


-- 
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA

Re: About Pools and Task Retries

Posted by Lance Norskog <la...@gmail.com>.
The task copies a table from the application DB to Redshift for analytics
work, once a day. If the application DB is super-busy, I want the task to
back off for many minutes and retry later.

So, if I want a timeout of 1/2 an hour, I should have the error callback
leave the pool and the retry callback enter the pool again?

On Sat, Jul 30, 2016 at 3:31 AM, Bolke de Bruin <bd...@gmail.com> wrote:

> Yes it does.
>
> (Tasks are not really aware of pools / slots, I have some work in that
> direction but that will be ready after August)
>
> Sent from my iPhone
>
> > On 30 jul. 2016, at 01:35, Lance Norskog <la...@gmail.com>
> wrote:
> >
> > When a Task in in a Pool, fails, and goes to sleep to retry, does it give
> > up its Pool membership while asleep?
> >
> >
> > --
> > Lance Norskog
> > lance.norskog@gmail.com
> > Redwood City, CA
>



-- 
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA

Re: About Pools and Task Retries

Posted by Bolke de Bruin <bd...@gmail.com>.
Yes it does. 

(Tasks are not really aware of pools / slots, I have some work in that direction but that will be ready after August)

Sent from my iPhone

> On 30 jul. 2016, at 01:35, Lance Norskog <la...@gmail.com> wrote:
> 
> When a Task in in a Pool, fails, and goes to sleep to retry, does it give
> up its Pool membership while asleep?
> 
> 
> -- 
> Lance Norskog
> lance.norskog@gmail.com
> Redwood City, CA