You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Ashika Umanga Umagiliya <um...@gmail.com> on 2017/07/18 05:22:24 UTC
Scheduler ignores the changed start_date ?
I used to have my start_date as '2017-03-14' for the DAG "cdna_daily_stg"
as follows :
default_args = {
'owner': 'cdna',
'depends_on_past': False,
'start_date': datetime(2017, 3, 14),
'email': ['some_email'],
'email_on_failure': True,
'email_on_retry': False,
'retries': 5,
'retry_delay': timedelta(minutes=5),
'on_failure_callback': on_failure_callback,
'on_success_call': on_success
}
dag = DAG(
dag_id='cdna_daily_stg',
default_args=default_args,
schedule_interval="0 2 * * *"
)
Due to some code refactoring in my DAG, I wanted to change my DAG name to
'cdna_daily_stg_v2' so I changed by start_date as well to '2017-07-14' as
follows:
default_args = {
'owner': 'cdna',
'depends_on_past': False,
'start_date': datetime(2017, 7, 14),
'email': ['some_email'],
'email_on_failure': True,
'email_on_retry': False,
'retries': 5,
'retry_delay': timedelta(minutes=5),
'on_failure_callback': on_failure_callback,
'on_success_call': on_success
}
dag = DAG(
dag_id='cdna_daily_stg_v2',
default_args=default_args,
schedule_interval="0 2 * * *"
)
But when I deploy my DAG with the new DAG , it keep starting DAG runs from
'2017-03-16' instead of from '2017-07-14'.
How to fix this ?
I still use same python file name, do I have to change the file names as
well as the DAG name ?
Re: Scheduler ignores the changed start_date ?
Posted by Ashika Umanga Umagiliya <um...@gmail.com>.
Any tips on this ?
I tried deleting unnecessary DAG info from mysql DB as well, didnt fix the
issue :
'delete from xcom where dag_id = "' *+* dag_input *+* '"',
'delete from task_instance where dag_id = "' + dag_input + '"',
'delete from sla_miss where dag_id = "' *+* dag_input *+* '"',
'delete from log where dag_id = "' + dag_input + '"',
'delete from job where dag_id = "' *+* dag_input *+* '"',
'delete from dag_run where dag_id = "' + dag_input + '"',
'delete from dag where dag_id = "' *+* dag_input *+* '"'
On Tue, Jul 18, 2017 at 2:22 PM, Ashika Umanga Umagiliya <
umanga.pdn@gmail.com> wrote:
>
> I used to have my start_date as '2017-03-14' for the DAG "cdna_daily_stg"
> as follows :
>
> default_args = {
> 'owner': 'cdna',
> 'depends_on_past': False,
> 'start_date': datetime(2017, 3, 14),
> 'email': ['some_email'],
> 'email_on_failure': True,
> 'email_on_retry': False,
> 'retries': 5,
> 'retry_delay': timedelta(minutes=5),
> 'on_failure_callback': on_failure_callback,
> 'on_success_call': on_success
> }
>
> dag = DAG(
> dag_id='cdna_daily_stg',
> default_args=default_args,
> schedule_interval="0 2 * * *"
> )
>
> Due to some code refactoring in my DAG, I wanted to change my DAG name to
> 'cdna_daily_stg_v2' so I changed by start_date as well to '2017-07-14' as
> follows:
>
> default_args = {
> 'owner': 'cdna',
> 'depends_on_past': False,
> 'start_date': datetime(2017, 7, 14),
> 'email': ['some_email'],
> 'email_on_failure': True,
> 'email_on_retry': False,
> 'retries': 5,
> 'retry_delay': timedelta(minutes=5),
> 'on_failure_callback': on_failure_callback,
> 'on_success_call': on_success
> }
>
> dag = DAG(
> dag_id='cdna_daily_stg_v2',
> default_args=default_args,
> schedule_interval="0 2 * * *"
> )
>
>
> But when I deploy my DAG with the new DAG , it keep starting DAG runs from
> '2017-03-16' instead of from '2017-07-14'.
>
> How to fix this ?
> I still use same python file name, do I have to change the file names as
> well as the DAG name ?
>
--
Umanga
http://jp.linkedin.com/in/umanga
http://umanga.ifreepages.com