You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@airflow.apache.org by Scott Halgrim <sc...@zapier.com.INVALID> on 2018/12/20 20:12:27 UTC

timeout not working in SqlSensor?

Does the timeout param work the way I think it should in Airflow 1.8? My query just pokes at the poke interval indefinitely. I want it to timeout after a few hours and fail instead.

The code is basically

“””
task_name = SqlSensor(
   conn_id='redshift',
   sql=(
       """
         select * from this_table where date=‘2018-12-18'
     """
   ),
   task_id=’task_name',
   dag=dag,
   start_date=datetime(2016, 9, 25),
   poke_interval=20,
   time_out=45,
“”"
So after running the query a couple of times I would expect a failure, but that’s not what’s happening, it just keeps querying and querying every 20 seconds.

Thanks,

Scott

Re: timeout not working in SqlSensor?

Posted by Scott Halgrim <sc...@zapier.com.INVALID>.

Oh boy, you’re right about that. Thanks!

It still runs for a few hours before it times out, but at least it’s not running indefinitely anymore. I’ll take another look at this later.

Thanks again!

Scott
On Dec 20, 2018, 5:00 PM -0800, Maxime Beauchemin <ma...@gmail.com>, wrote:
> I think it's `timeout` not `time_out`.
> https://airflow.apache.org/code.html#basesensoroperator
>
> On Thu, Dec 20, 2018 at 12:12 PM Scott Halgrim
> <sc...@zapier.com.invalid> wrote:
>
> > Does the timeout param work the way I think it should in Airflow 1.8? My
> > query just pokes at the poke interval indefinitely. I want it to timeout
> > after a few hours and fail instead.
> >
> > The code is basically
> >
> > “””
> > task_name = SqlSensor(
> > conn_id='redshift',
> > sql=(
> > """
> > select * from this_table where date=‘2018-12-18'
> > """
> > ),
> > task_id=’task_name',
> > dag=dag,
> > start_date=datetime(2016, 9, 25),
> > poke_interval=20,
> > time_out=45,
> > “”"
> > So after running the query a couple of times I would expect a failure, but
> > that’s not what’s happening, it just keeps querying and querying every 20
> > seconds.
> >
> > Thanks,
> >
> > Scott
> >

Re: timeout not working in SqlSensor?

Posted by Maxime Beauchemin <ma...@gmail.com>.

I think it's `timeout` not `time_out`.
https://airflow.apache.org/code.html#basesensoroperator

On Thu, Dec 20, 2018 at 12:12 PM Scott Halgrim
<sc...@zapier.com.invalid> wrote:

> Does the timeout param work the way I think it should in Airflow 1.8? My
> query just pokes at the poke interval indefinitely. I want it to timeout
> after a few hours and fail instead.
>
> The code is basically
>
> “””
> task_name = SqlSensor(
>    conn_id='redshift',
>    sql=(
>        """
>          select * from this_table where date=‘2018-12-18'
>      """
>    ),
>    task_id=’task_name',
>    dag=dag,
>    start_date=datetime(2016, 9, 25),
>    poke_interval=20,
>    time_out=45,
> “”"
> So after running the query a couple of times I would expect a failure, but
> that’s not what’s happening, it just keeps querying and querying every 20
> seconds.
>
> Thanks,
>
> Scott
>