You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Andrew Maguire <an...@gmail.com> on 2017/07/28 16:32:31 UTC
Email on last failed try
Hey,
Just wondering if anyone knows if there might be a way to only send email
on the last failed try of a task?
Could I use a callable on failure only send the mail on the last failed
try.
We are using big query and getting lots of transient errors around limit of
concurrent queries that usually work on 2nd or 3rd try.
Else I might just use Gmail rules to filter emails accordingly.
Just wondering if anyone has done anything like this before or if there
would be an easy enough way to do this.
Cheers,
Andy
Re: Email on last failed try
Posted by Andrew Maguire <an...@gmail.com>.
Ah that must be what it is, have left that as default which I guess is
true.
Cheers
Andy
On Fri, 28 Jul 2017, 22:53 Maxime Beauchemin, <ma...@gmail.com>
wrote:
> Maybe you don't have have `email_on_retry=False`?
>
> On Fri, Jul 28, 2017 at 11:52 AM, Alex Guziel <
> alex.guziel@airbnb.com.invalid> wrote:
>
> > Sounds like unintended behavior. That should be what email_on_retry does.
> > If you can repro, file a ticket.
> >
> > On Fri, Jul 28, 2017 at 11:44 AM, Andrew Maguire <an...@gmail.com>
> > wrote:
> >
> > > Yeah - i have:
> > >
> > > 'email_on_failure': True
> > > 'retries': 4
> > >
> > > So i get emails on every try: e.g. Try 1 out of 5
> > >
> > > Really what i'm most worried about is the final failures then i have a
> > > problem, whereas if it fails 3 times and then succeeds i'm ok to be
> > unaware
> > > of that.
> > >
> > > Maybe email routing via gmail might be a good idea - so i can send the
> > > emails with "Try 5 out of 5" to a priority folder and just let the
> other
> > > ones come somewhere that i can check on every now and then.
> > >
> > > Cheers,
> > > Andy
> > >
> > > On Fri, Jul 28, 2017 at 5:42 PM Maxime Beauchemin <
> > > maximebeauchemin@gmail.com> wrote:
> > >
> > > > Wouldn't `email_on_failure=True` work for you?
> > > >
> > > > https://airflow.incubator.apache.org/code.html?
> > > highlight=email_on_failure#baseoperator
> > > >
> > > > On Fri, Jul 28, 2017 at 9:32 AM, Andrew Maguire <
> andrewm4894@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hey,
> > > > >
> > > > > Just wondering if anyone knows if there might be a way to only send
> > > email
> > > > > on the last failed try of a task?
> > > > >
> > > > > Could I use a callable on failure only send the mail on the last
> > failed
> > > > > try.
> > > > >
> > > > > We are using big query and getting lots of transient errors around
> > > limit
> > > > of
> > > > > concurrent queries that usually work on 2nd or 3rd try.
> > > > >
> > > > > Else I might just use Gmail rules to filter emails accordingly.
> > > > >
> > > > > Just wondering if anyone has done anything like this before or if
> > there
> > > > > would be an easy enough way to do this.
> > > > >
> > > > > Cheers,
> > > > > Andy
> > > > >
> > > >
> > >
> >
>
Re: Email on last failed try
Posted by Maxime Beauchemin <ma...@gmail.com>.
Maybe you don't have have `email_on_retry=False`?
On Fri, Jul 28, 2017 at 11:52 AM, Alex Guziel <
alex.guziel@airbnb.com.invalid> wrote:
> Sounds like unintended behavior. That should be what email_on_retry does.
> If you can repro, file a ticket.
>
> On Fri, Jul 28, 2017 at 11:44 AM, Andrew Maguire <an...@gmail.com>
> wrote:
>
> > Yeah - i have:
> >
> > 'email_on_failure': True
> > 'retries': 4
> >
> > So i get emails on every try: e.g. Try 1 out of 5
> >
> > Really what i'm most worried about is the final failures then i have a
> > problem, whereas if it fails 3 times and then succeeds i'm ok to be
> unaware
> > of that.
> >
> > Maybe email routing via gmail might be a good idea - so i can send the
> > emails with "Try 5 out of 5" to a priority folder and just let the other
> > ones come somewhere that i can check on every now and then.
> >
> > Cheers,
> > Andy
> >
> > On Fri, Jul 28, 2017 at 5:42 PM Maxime Beauchemin <
> > maximebeauchemin@gmail.com> wrote:
> >
> > > Wouldn't `email_on_failure=True` work for you?
> > >
> > > https://airflow.incubator.apache.org/code.html?
> > highlight=email_on_failure#baseoperator
> > >
> > > On Fri, Jul 28, 2017 at 9:32 AM, Andrew Maguire <andrewm4894@gmail.com
> >
> > > wrote:
> > >
> > > > Hey,
> > > >
> > > > Just wondering if anyone knows if there might be a way to only send
> > email
> > > > on the last failed try of a task?
> > > >
> > > > Could I use a callable on failure only send the mail on the last
> failed
> > > > try.
> > > >
> > > > We are using big query and getting lots of transient errors around
> > limit
> > > of
> > > > concurrent queries that usually work on 2nd or 3rd try.
> > > >
> > > > Else I might just use Gmail rules to filter emails accordingly.
> > > >
> > > > Just wondering if anyone has done anything like this before or if
> there
> > > > would be an easy enough way to do this.
> > > >
> > > > Cheers,
> > > > Andy
> > > >
> > >
> >
>
Re: Email on last failed try
Posted by Alex Guziel <al...@airbnb.com.INVALID>.
Sounds like unintended behavior. That should be what email_on_retry does.
If you can repro, file a ticket.
On Fri, Jul 28, 2017 at 11:44 AM, Andrew Maguire <an...@gmail.com>
wrote:
> Yeah - i have:
>
> 'email_on_failure': True
> 'retries': 4
>
> So i get emails on every try: e.g. Try 1 out of 5
>
> Really what i'm most worried about is the final failures then i have a
> problem, whereas if it fails 3 times and then succeeds i'm ok to be unaware
> of that.
>
> Maybe email routing via gmail might be a good idea - so i can send the
> emails with "Try 5 out of 5" to a priority folder and just let the other
> ones come somewhere that i can check on every now and then.
>
> Cheers,
> Andy
>
> On Fri, Jul 28, 2017 at 5:42 PM Maxime Beauchemin <
> maximebeauchemin@gmail.com> wrote:
>
> > Wouldn't `email_on_failure=True` work for you?
> >
> > https://airflow.incubator.apache.org/code.html?
> highlight=email_on_failure#baseoperator
> >
> > On Fri, Jul 28, 2017 at 9:32 AM, Andrew Maguire <an...@gmail.com>
> > wrote:
> >
> > > Hey,
> > >
> > > Just wondering if anyone knows if there might be a way to only send
> email
> > > on the last failed try of a task?
> > >
> > > Could I use a callable on failure only send the mail on the last failed
> > > try.
> > >
> > > We are using big query and getting lots of transient errors around
> limit
> > of
> > > concurrent queries that usually work on 2nd or 3rd try.
> > >
> > > Else I might just use Gmail rules to filter emails accordingly.
> > >
> > > Just wondering if anyone has done anything like this before or if there
> > > would be an easy enough way to do this.
> > >
> > > Cheers,
> > > Andy
> > >
> >
>
Re: Email on last failed try
Posted by Andrew Maguire <an...@gmail.com>.
Yeah - i have:
'email_on_failure': True
'retries': 4
So i get emails on every try: e.g. Try 1 out of 5
Really what i'm most worried about is the final failures then i have a
problem, whereas if it fails 3 times and then succeeds i'm ok to be unaware
of that.
Maybe email routing via gmail might be a good idea - so i can send the
emails with "Try 5 out of 5" to a priority folder and just let the other
ones come somewhere that i can check on every now and then.
Cheers,
Andy
On Fri, Jul 28, 2017 at 5:42 PM Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:
> Wouldn't `email_on_failure=True` work for you?
>
> https://airflow.incubator.apache.org/code.html?highlight=email_on_failure#baseoperator
>
> On Fri, Jul 28, 2017 at 9:32 AM, Andrew Maguire <an...@gmail.com>
> wrote:
>
> > Hey,
> >
> > Just wondering if anyone knows if there might be a way to only send email
> > on the last failed try of a task?
> >
> > Could I use a callable on failure only send the mail on the last failed
> > try.
> >
> > We are using big query and getting lots of transient errors around limit
> of
> > concurrent queries that usually work on 2nd or 3rd try.
> >
> > Else I might just use Gmail rules to filter emails accordingly.
> >
> > Just wondering if anyone has done anything like this before or if there
> > would be an easy enough way to do this.
> >
> > Cheers,
> > Andy
> >
>
Re: Email on last failed try
Posted by Maxime Beauchemin <ma...@gmail.com>.
Wouldn't `email_on_failure=True` work for you?
https://airflow.incubator.apache.org/code.html?highlight=email_on_failure#baseoperator
On Fri, Jul 28, 2017 at 9:32 AM, Andrew Maguire <an...@gmail.com>
wrote:
> Hey,
>
> Just wondering if anyone knows if there might be a way to only send email
> on the last failed try of a task?
>
> Could I use a callable on failure only send the mail on the last failed
> try.
>
> We are using big query and getting lots of transient errors around limit of
> concurrent queries that usually work on 2nd or 3rd try.
>
> Else I might just use Gmail rules to filter emails accordingly.
>
> Just wondering if anyone has done anything like this before or if there
> would be an easy enough way to do this.
>
> Cheers,
> Andy
>