You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Guru Devanla (Jira)" <ji...@apache.org> on 2019/09/23 12:40:00 UTC
[jira] [Updated] (AIRFLOW-5539) default_args dictionary is mutated
when passed as arugment to Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-5539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guru Devanla updated AIRFLOW-5539:
----------------------------------
Description:
The `default_args` argument is a dictionary that can be passed into the DAG constructor while constructing a dag. But, using the same variable to construct multiple DAGS breaks, since the `default_args` argument is mutated. Here is a minimal example:
{code:java}
// code placeholder
In [19]: from functools import partial
In [20]: default_args=dict(start_date=datetime.datetime(2019, 8, 1))
In [21]: default_dag=partial(DAG, dag_id='test', default_args=default_args)
In [23]: default_args['start_date'].tzinfo is None
Out[23]: True
In [24]: d1=default_dag() # construct the DAG.
# Now check the value in default_args.
In [25]: default_args['start_date'].tzinfo is None
Out[25]: False
In [26]:
{code}
My guess is that the deepcopy call to copy default_args is not working properly.
I am using Python 3.7.4.
was:
The `default_args` argument is a dictionary that can be passed into the DAG constructor while constructing a dag. But, using the same variable to construct multiple DAGS breaks, since the `default_args` argument is mutated. Here is a minimal example:
{code:java}
// code placeholder
In [19]: from functools import partial In [20]: default_args=dict(start_date=datetime.datetime(2019, 8, 1)) In [21]: default_dag=partial(DAG, dag_id='test', default_args=default_args)
In [23]: default_args['start_date'].tzinfo is None
Out[23]: True
In [24]: d1=default_dag() # construct the DAG.
# Now check the value in default_args. In [25]: default_args['start_date'].tzinfo is None
Out[25]: False
In [26]:
{code}
My guess is that the deepcopy call to copy default_args is not working properly.
I am using Python 3.7.4.
> default_args dictionary is mutated when passed as arugment to Airflow
> ---------------------------------------------------------------------
>
> Key: AIRFLOW-5539
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5539
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: 1.10.3
> Environment: Python 3.7.4, Linux
> Reporter: Guru Devanla
> Priority: Major
>
> The `default_args` argument is a dictionary that can be passed into the DAG constructor while constructing a dag. But, using the same variable to construct multiple DAGS breaks, since the `default_args` argument is mutated. Here is a minimal example:
>
> {code:java}
> // code placeholder
> In [19]: from functools import partial
> In [20]: default_args=dict(start_date=datetime.datetime(2019, 8, 1))
> In [21]: default_dag=partial(DAG, dag_id='test', default_args=default_args)
>
> In [23]: default_args['start_date'].tzinfo is None
> Out[23]: True
> In [24]: d1=default_dag() # construct the DAG.
> # Now check the value in default_args.
> In [25]: default_args['start_date'].tzinfo is None
> Out[25]: False
> In [26]:
> {code}
>
> My guess is that the deepcopy call to copy default_args is not working properly.
>
> I am using Python 3.7.4.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)