[GitHub] [airflow] michaelgaotile commented on issue #22291: Airflow Tasks not starting at specified time, starts as normal after long delay

[GitBox <gi...@apache.org>]

michaelgaotile commented on issue #22291:
URL: https://github.com/apache/airflow/issues/22291#issuecomment-1068250368


   hi @pateash 
   
   Please see below for DAG.
   ```
   
   from airflow import DAG
   from airflow.providers.databricks.operators.databricks import DatabricksRunNowOperator
   from airflow.providers.amazon.aws.sensors.s3_key import S3KeySensor
   from airflow.providers.slack.operators.slack_webhook import SlackWebhookOperator
   from airflow.utils.dates import days_ago
   from airflow import macros
   from utils import failed_slack_alert, sla_miss_slack_alert
   from datetime import timedelta
   from textwrap import dedent
   
   # These arguments will be added to all the tasks in this file
   default_args = {
       'owner': 'XXXX',
       'depends_on_past': True,
       'start_date': days_ago(1), 
       'provide_context':True,
       'on_failure_callback':failed_slack_alert,
       'retries': 3,
       'retry_delay': macros.timedelta(minutes=20),
       'sla': timedelta(minutes=180)
   }
   
   dag = DAG('XXXX_daily_report', sla_miss_callback=sla_miss_slack_alert, schedule_interval='@daily', tags=['XXXX', 'marketing'], default_args=default_args)
   ds = "{{ ds }}"
   
   t_generate__report =  DatabricksRunNowOperator(
       task_id='generate_report',
       job_id=XXXX,
       databricks_conn_id = 'databricks',
       notebook_params= {'ds': ds, 'target_table_name':'dim_XXXX'},
       execution_timeout=timedelta(hours=2),
       dag=dag
       )
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org