You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Tamara Mendt <tm...@hellofresh.com> on 2016/10/20 14:38:42 UTC
Backfill for tasks to be scheduled @once
Hi,
I have tried to use the "@once" scheduling option for the first time for a
DAG that I only wish to run once. I would possibly want to run it again at
some random point in the future, but not on regular intervals.
I thought that if I set a start date to the DAG, it would be scheduled only
once, with the start date. However it seemed to be scheduled once with the
start date, and another time with the time point when I unpause the DAG.
Also, when I tried clearing all executions of the DAG and running a
backfill, nothing was scheduled.
Maybe I am confused as to how this type of one time scheduling works. Can
somebody please provide some insight?
Thank you,
Cheers,
Tamara.
--
[image: logo]
<http://www.facebook.com/hellofreshde> <http://twitter.com/HelloFreshde>
<http://instagram.com/hellofreshde/> <http://blog.hellofresh.de/>
<https://app.adjust.com/ayje08?campaign=Hellofresh&deep_link=hellofresh%3A%2F%2F&post_deep_link=https%3A%2F%2Fwww.hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_source%3Demail_signature&fallback=https%3A%2F%2Fwww.hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_source%3Demail_signature>
*HelloFresh App –Download Now!*
<https://app.adjust.com/ayje08?campaign=Hellofresh&deep_link=hellofresh%3A%2F%2F&post_deep_link=https%3A%2F%2Fwww.hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_source%3Demail_signature&fallback=https%3A%2F%2Fwww.hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_source%3Demail_signature>
*We're active in:*
US <https://www.hellofresh.com/?utm_medium=email&utm_source=email_signature>
| DE
<https://www.hellofresh.de/?utm_medium=email&utm_source=email_signature> |
UK
<https://www.hellofresh.co.uk/?utm_medium=email&utm_source=email_signature>
| NL
<https://www.hellofresh.nl/?utm_medium=email&utm_source=email_signature> |
AU
<https://www.hellofresh.com.au/?utm_medium=email&utm_source=email_signature>
| BE
<https://www.hellofresh.be/?utm_medium=email&utm_source=email_signature> |
AT <https://www.hellofresh.at/?utm_medium=email&utm_source=email_signature>
| CH
<https://www.hellofresh.ch/?utm_medium=email&utm_source=email_signature> |
CA <https://www.hellofresh.ca/?utm_medium=email&utm_source=email_signature>
www.HelloFreshGroup.com
<http://www.hellofreshgroup.com/?utm_medium=email&utm_source=email_signature>
We are hiring around the world – Click here to join us
<https://www.hellofresh.com/jobs/?utm_medium=email&utm_source=email_signature>
--
<https://www.hellofresh.com/jobs/?utm_medium=email&utm_source=email_signature>
HelloFresh AG, Berlin (Sitz der Gesellschaft) | Vorstände: Dominik S.
Richter (Vorsitzender), Thomas W. Griesel, Christian Gärtner | Vorsitzender
des Aufsichtsrats: Jeffrey Lieberman | Eingetragen beim Amtsgericht
Charlottenburg, HRB 171666 B | USt-Id Nr.: DE 302210417
*CONFIDENTIALITY NOTICE:* This message (including any attachments) is
confidential and may be privileged. It may be read, copied and used only by
the intended recipient. If you have received it in error please contact the
sender (by return e-mail) immediately and delete this message. Any
unauthorized use or dissemination of this message in whole or in parts is
strictly prohibited.
Re: Backfill for tasks to be scheduled @once
Posted by Boris Tyukin <bo...@boristyukin.com>.
hello Tamara,
I just created a quick example below and it worked like you would expect it
to work. It ran only once. Also when I cleared the task instance execution
via UI (or alternatively, deleting it and setting DagRun to running
status), it would rerun it again - this would be your on-demand scenario.
Make sure your start date is actually set in the past.
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
# Determine schedule:
dag_schedule_interval = "@once"
dag_start_date = datetime.now() - timedelta(seconds=60)
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'start_date': dag_start_date,
'email': ['airflow@airflow.com'],
'email_on_failure': False,
'email_on_retry': False,
}
# Change version number if schedule needs to be changed:
dag = DAG(
'once', default_args=default_args,
schedule_interval=dag_schedule_interval)
# t1, t2 and t3 are examples of tasks created by instantiating operators
t1 = BashOperator(
task_id='t1',
bash_command='echo execution ts {{ ts }} & echo 1',
# bash_command='exit 1',
dag=dag)
On Thu, Oct 20, 2016 at 10:38 AM, Tamara Mendt <tm...@hellofresh.com> wrote:
> Hi,
>
> I have tried to use the "@once" scheduling option for the first time for a
> DAG that I only wish to run once. I would possibly want to run it again at
> some random point in the future, but not on regular intervals.
>
> I thought that if I set a start date to the DAG, it would be scheduled only
> once, with the start date. However it seemed to be scheduled once with the
> start date, and another time with the time point when I unpause the DAG.
> Also, when I tried clearing all executions of the DAG and running a
> backfill, nothing was scheduled.
>
> Maybe I am confused as to how this type of one time scheduling works. Can
> somebody please provide some insight?
>
> Thank you,
>
> Cheers,
>
> Tamara.
>
> --
>
> [image: logo]
> <http://www.facebook.com/hellofreshde> <http://twitter.com/
> HelloFreshde>
> <http://instagram.com/hellofreshde/> <http://blog.hellofresh.de/>
> <https://app.adjust.com/ayje08?campaign=Hellofresh&
> deep_link=hellofresh%3A%2F%2F&post_deep_link=https%3A%2F%
> 2Fwww.hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_
> source%3Demail_signature&fallback=https%3A%2F%2Fwww.
> hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_source%
> 3Demail_signature>
>
> *HelloFresh App –Download Now!*
> <https://app.adjust.com/ayje08?campaign=Hellofresh&
> deep_link=hellofresh%3A%2F%2F&post_deep_link=https%3A%2F%
> 2Fwww.hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_
> source%3Demail_signature&fallback=https%3A%2F%2Fwww.
> hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_source%
> 3Demail_signature>
> *We're active in:*
> US <https://www.hellofresh.com/?utm_medium=email&utm_source=
> email_signature>
> | DE
> <https://www.hellofresh.de/?utm_medium=email&utm_source=email_signature> |
> UK
> <https://www.hellofresh.co.uk/?utm_medium=email&utm_source=email_signature
> >
> | NL
> <https://www.hellofresh.nl/?utm_medium=email&utm_source=email_signature> |
> AU
> <https://www.hellofresh.com.au/?utm_medium=email&utm_
> source=email_signature>
> | BE
> <https://www.hellofresh.be/?utm_medium=email&utm_source=email_signature> |
> AT <https://www.hellofresh.at/?utm_medium=email&utm_source=email_signature
> >
> | CH
> <https://www.hellofresh.ch/?utm_medium=email&utm_source=email_signature> |
> CA <https://www.hellofresh.ca/?utm_medium=email&utm_source=email_signature
> >
>
> www.HelloFreshGroup.com
> <http://www.hellofreshgroup.com/?utm_medium=email&utm_
> source=email_signature>
>
> We are hiring around the world – Click here to join us
> <https://www.hellofresh.com/jobs/?utm_medium=email&utm_
> source=email_signature>
>
> --
>
> <https://www.hellofresh.com/jobs/?utm_medium=email&utm_
> source=email_signature>
> HelloFresh AG, Berlin (Sitz der Gesellschaft) | Vorstände: Dominik S.
> Richter (Vorsitzender), Thomas W. Griesel, Christian Gärtner | Vorsitzender
> des Aufsichtsrats: Jeffrey Lieberman | Eingetragen beim Amtsgericht
> Charlottenburg, HRB 171666 B | USt-Id Nr.: DE 302210417
>
> *CONFIDENTIALITY NOTICE:* This message (including any attachments) is
> confidential and may be privileged. It may be read, copied and used only by
> the intended recipient. If you have received it in error please contact the
> sender (by return e-mail) immediately and delete this message. Any
> unauthorized use or dissemination of this message in whole or in parts is
> strictly prohibited.
>