Scheduler defunct Long Running Task with LocalExecutor

[Adinata <ma...@gmail.com>]

Hi all,

I think I have problem with long running task in LocalExecutor, that make
other scheduler process become defunct.
I'm using airflow 1.7.1.3, and running airflow scheduler -n 20. When it's
on the defunct state, it's unable to schedule any tasks. Do you have any
possible solution?

Attached the screenshot in this email.

--
*Adinata*
Engineer - UrbanIndo.com

Re: Scheduler defunct Long Running Task with LocalExecutor

[Adinata <ma...@gmail.com>]

Hi Nadeem,

I think my scheduler stuck because there is a remaining running task in the
executor. I guess that's why the scheduler `defunct`-ed. It won't do to
restart the scheduler because it will kill the running task. I'm not using
CeleryExecutor, and I'm not planning on trying it.

@all
If I wasn't wrong, in master we don't need to do `n numruns` of scheduler.
Does it safe to run master in production? Any guide on how to install the
master version?

If you don't recommend to run master in production, is there any trade off
in running multiple scheduler instance (e.g., under supervisor) with single
parallelism configuration?

Thanks,

--
*Adinata*
TOKI 2009
SMAN Plus Provinsi Riau 9th
13509022 - Informatika ITB 2009
Engineer - UrbanIndo.com

On Mon, Aug 15, 2016 at 11:23 AM, Nadeem Ahmed Nazeer <na...@neon-lab.com>
wrote:

> Hi Adinata,
>
> I had the same issue and have logged a JIRA for this (
> https://issues.apache.org/jira/browse/AIRFLOW-401).
>
> Bolke has commented on JIRA to have the issue verified against master
> branch and it is on my to-do list. Looks like there were a lot of scheduler
> fixes made since release. I haven't verified this on master yet, if you get
> a chance to verify, please let us know what you find.
>
> The only workaround I found is to set up a process that restarts the
> scheduler every X minutes although it might not be clean. Also, did you try
> with Celery Executor?
>
> Thanks,
> Nadeem
>
> On Sat, Aug 13, 2016 at 10:38 PM, Adinata <ma...@gmail.com> wrote:
>
> > Hi all,
> >
> > I think I have problem with long running task in LocalExecutor, that make
> > other scheduler process become defunct.
> > I'm using airflow 1.7.1.3, and running airflow scheduler -n 20. When it's
> > on the defunct state, it's unable to schedule any tasks. Do you have any
> > possible solution?
> >
> > Attached the screenshot in this email.
> >
> > --
> > *Adinata*
> > Engineer - UrbanIndo.com
> >
>

Re: Scheduler defunct Long Running Task with LocalExecutor

[Nadeem Ahmed Nazeer <na...@neon-lab.com>]

Hi Adinata,

I had the same issue and have logged a JIRA for this (
https://issues.apache.org/jira/browse/AIRFLOW-401).

Bolke has commented on JIRA to have the issue verified against master
branch and it is on my to-do list. Looks like there were a lot of scheduler
fixes made since release. I haven't verified this on master yet, if you get
a chance to verify, please let us know what you find.

The only workaround I found is to set up a process that restarts the
scheduler every X minutes although it might not be clean. Also, did you try
with Celery Executor?

Thanks,
Nadeem

On Sat, Aug 13, 2016 at 10:38 PM, Adinata <ma...@gmail.com> wrote:

> Hi all,
>
> I think I have problem with long running task in LocalExecutor, that make
> other scheduler process become defunct.
> I'm using airflow 1.7.1.3, and running airflow scheduler -n 20. When it's
> on the defunct state, it's unable to schedule any tasks. Do you have any
> possible solution?
>
> Attached the screenshot in this email.
>
> --
> *Adinata*
> Engineer - UrbanIndo.com
>