[GitHub] [airflow] yuqian90 edited a comment on issue #7038: [AIRFLOW-4495] Allow externally triggered dags to run for future exec dates

[GitBox <gi...@apache.org>]

yuqian90 edited a comment on issue #7038: [AIRFLOW-4495] Allow externally triggered dags to run for future exec dates
URL: https://github.com/apache/airflow/pull/7038#issuecomment-577471036
 
 
   > Sorry for the comment spam but I thought it would be important for @yuqian90
   > 
   > We have the exact same problem you have, we solve it by setting the DAG with a start date of the timezones it should be scheduled in, e.g. Tokyo time. We then create a macro which is a modified version of this: https://stackoverflow.com/a/54922776/2958068
   > 
   > The macro above provides the execution date in the same time zone as the DAG. Further we modify it to provide the actual scheduled date e.g. if scheduled to run on 2019-01-01 08:00 that's the datetime it provides (which in the default Airlow variables is the 'next execution date').
   
   @notatallshaw  thanks for the comment. Did setting `start_date` to the timezone you want work? We definitely remember trying something like that. But the airflow scheduler still did not run tasks before UTC midnight. Looking at the original code before this PR, it's not hard to see why. No matter what timezone we set the `start_date` to, this line was always skipping tasks as long as the execution_date is greater than `timezone.utcnow()`. So if the timezone we need is in Asia, this does not work till UTC midnight.
   
   But this PR fixed / improved this. Now we no longer have to put tasks for date T on the DAG for execution_date T-1.
   
   ```
   airflow/jobs/scheduler_job.py
   ...
               # don't consider runs that are executed in the future
               if run.execution_date > timezone.utcnow():
                   self.log.error(
                       "Execution date is in future: %s",
                       run.execution_date
                   )
                   continue
   ```
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services