You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Lance Norskog <la...@gmail.com> on 2016/05/25 05:39:57 UTC
Get the date into a dag?
My mad google skillz are failing me. Where is an example of getting the
execution date into the python code in a DAG?
I can't quite tell how to trigger the ninja template engine. Is there an
example somewhere of this?
--
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA
Re: Get the date into a dag?
Posted by Lance Norskog <la...@gmail.com>.
Thanks!
Also, I figured out a confusion of mine: there are two program runs. First,
the Python code in the Dag file runs runs when Airflow loads the Dag file.
Later, the operators run when Airflow runs the DAG. "execution date" does
not exist when loading the DAG, it only exists when running it.
On Thu, May 26, 2016 at 6:40 AM, Jeremiah Lowin <jl...@apache.org> wrote:
> Lance,
>
> If you set the "templated_fields" attribute (on mobile so that might not
> actually be correct) you can make your operator run any field through
> jinja. See how BashOperator handles bash_command for an example.
>
> Best,
> Jeremiah
>
> On Wed, May 25, 2016 at 6:40 PM Lance Norskog <la...@gmail.com>
> wrote:
>
> > Ah! There's the problem. I'm using a home-made Operator that does not
> know
> > how to use the templating engine, so this example does not count.
> >
> > It seems like I need to access the execution_date object.
> >
> >
> > On Tue, May 24, 2016 at 11:15 PM, הילה ויזן <hi...@gmail.com> wrote:
> >
> > > a working example: {{ts}} is the execution date
> > >
> > > general_templated_cmd = """
> > > <your command> {{ ts }} {{ task.task_id }} {{ dag.dag_id }}
> > > """
> > >
> > > t1 = BashOperator(
> > > task_id='morning_task',
> > > bash_command=general_templated_cmd,
> > > dag=dag)
> > >
> > > On Wed, May 25, 2016 at 8:39 AM, Lance Norskog <
> lance.norskog@gmail.com>
> > > wrote:
> > >
> > > > My mad google skillz are failing me. Where is an example of getting
> the
> > > > execution date into the python code in a DAG?
> > > >
> > > > I can't quite tell how to trigger the ninja template engine. Is there
> > an
> > > > example somewhere of this?
> > > >
> > > > --
> > > > Lance Norskog
> > > > lance.norskog@gmail.com
> > > > Redwood City, CA
> > > >
> > >
> >
> >
> >
> > --
> > Lance Norskog
> > lance.norskog@gmail.com
> > Redwood City, CA
> >
>
--
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA
Re: Get the date into a dag?
Posted by Jeremiah Lowin <jl...@apache.org>.
Lance,
If you set the "templated_fields" attribute (on mobile so that might not
actually be correct) you can make your operator run any field through
jinja. See how BashOperator handles bash_command for an example.
Best,
Jeremiah
On Wed, May 25, 2016 at 6:40 PM Lance Norskog <la...@gmail.com>
wrote:
> Ah! There's the problem. I'm using a home-made Operator that does not know
> how to use the templating engine, so this example does not count.
>
> It seems like I need to access the execution_date object.
>
>
> On Tue, May 24, 2016 at 11:15 PM, הילה ויזן <hi...@gmail.com> wrote:
>
> > a working example: {{ts}} is the execution date
> >
> > general_templated_cmd = """
> > <your command> {{ ts }} {{ task.task_id }} {{ dag.dag_id }}
> > """
> >
> > t1 = BashOperator(
> > task_id='morning_task',
> > bash_command=general_templated_cmd,
> > dag=dag)
> >
> > On Wed, May 25, 2016 at 8:39 AM, Lance Norskog <la...@gmail.com>
> > wrote:
> >
> > > My mad google skillz are failing me. Where is an example of getting the
> > > execution date into the python code in a DAG?
> > >
> > > I can't quite tell how to trigger the ninja template engine. Is there
> an
> > > example somewhere of this?
> > >
> > > --
> > > Lance Norskog
> > > lance.norskog@gmail.com
> > > Redwood City, CA
> > >
> >
>
>
>
> --
> Lance Norskog
> lance.norskog@gmail.com
> Redwood City, CA
>
Re: Get the date into a dag?
Posted by Lance Norskog <la...@gmail.com>.
Ah! There's the problem. I'm using a home-made Operator that does not know
how to use the templating engine, so this example does not count.
It seems like I need to access the execution_date object.
On Tue, May 24, 2016 at 11:15 PM, הילה ויזן <hi...@gmail.com> wrote:
> a working example: {{ts}} is the execution date
>
> general_templated_cmd = """
> <your command> {{ ts }} {{ task.task_id }} {{ dag.dag_id }}
> """
>
> t1 = BashOperator(
> task_id='morning_task',
> bash_command=general_templated_cmd,
> dag=dag)
>
> On Wed, May 25, 2016 at 8:39 AM, Lance Norskog <la...@gmail.com>
> wrote:
>
> > My mad google skillz are failing me. Where is an example of getting the
> > execution date into the python code in a DAG?
> >
> > I can't quite tell how to trigger the ninja template engine. Is there an
> > example somewhere of this?
> >
> > --
> > Lance Norskog
> > lance.norskog@gmail.com
> > Redwood City, CA
> >
>
--
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA
Re: Get the date into a dag?
Posted by הילה ויזן <hi...@gmail.com>.
a working example: {{ts}} is the execution date
general_templated_cmd = """
<your command> {{ ts }} {{ task.task_id }} {{ dag.dag_id }}
"""
t1 = BashOperator(
task_id='morning_task',
bash_command=general_templated_cmd,
dag=dag)
On Wed, May 25, 2016 at 8:39 AM, Lance Norskog <la...@gmail.com>
wrote:
> My mad google skillz are failing me. Where is an example of getting the
> execution date into the python code in a DAG?
>
> I can't quite tell how to trigger the ninja template engine. Is there an
> example somewhere of this?
>
> --
> Lance Norskog
> lance.norskog@gmail.com
> Redwood City, CA
>