[jira] [Commented] (AIRFLOW-3581) Fix next_ds/prev_ds semantics for manual runs

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729816#comment-16729816 ] 

ASF GitHub Bot commented on AIRFLOW-3581:
-----------------------------------------

aoen commented on pull request #4385: [AIRFLOW-3581] Fix next_ds/prev_ds semantics for manual runs
URL: https://github.com/apache/incubator-airflow/pull/4385
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [X] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
     - https://issues.apache.org/jira/browse/AIRFLOW-3581
     - In case you are fixing a typo in the documentation you can prepend your commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
   
   ### Description
   
   - [X] Here are some details about my PR, including screenshots of any UI changes:
    
   This PR makes next_ds get set to execution_date for manually triggered runs instead of the next schedule-interval aligned execution date so that cron-like semantics are possible (i.e. having execution be aligned to the left side of the scheduling interval by using next_ds instead of execution_date in templates. Using the next schedule interval aligned execution date doesn't make sense for manually triggered runs.
   
   ### Tests
   
   - [X] My PR adds unit tests
   
   ### Commits
   
   - [X] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
     1. Subject is separated from body by a blank line
     1. Subject is limited to 50 characters (not including Jira issue reference)
     1. Subject does not end with a period
     1. Subject uses the imperative mood ("add", not "adding")
     1. Body wraps at 72 characters
     1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [X] In case of new functionality, my PR adds documentation that describes how to use it.
     - When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
     - All the public functions and the classes in the PR contain docstrings that explain what it does
   
   ### Code Quality
   
   - [X] Passes `flake8`
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Fix next_ds/prev_ds semantics for manual runs
> ---------------------------------------------
>
>                 Key: AIRFLOW-3581
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3581
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>            Reporter: Dan Davydov
>            Assignee: Dan Davydov
>            Priority: Major
>
> {color:#222222}next_ds is useful when you need cron-style scheduling, e.g. a task that runs for date "X" uses that date for its logic, e.g. send an email to users saying the run that was supposed to run for date "X" has completed. The problem is it doesn't behave as expected when it comes to manually triggered runs as illustrated by the diagrams below.{color}
>   
>  Using execution_date in a task
>  *Scheduled Run (works as expected)*
>  execution_date1           start_date1
>  \/                                  \/
>   *|-----------------------------|*
>  /\                                  /\
>  \ _________________/
>     scheduling_interval
>   
>  *Manual Run* *(works as expected)*
> triggered_date + execution_date + start_date
>  \/
>  *|*
>   
>  Using next_ds in a Task
>  *Scheduled Run (works as expected)*
>  next_ds1 + start_date1           next_ds2 + start_date2
>  \/                                                         \/
>   *|------------------------------------------------|*
>  /\                                                         /\
>   \____________________________/
>               scheduling_interval
>   
>  *Manual Run* *(next_ds1 is expected to match triggered_date as in the case for the manually triggered run that uses the regular execution_date above)*
>  triggered_date                                    next_ds1 + start_date
>  \/                                                         \/
>  *|-------------------------------------------------|*
>  /\                                                         /\
>   ____________________________/
>               0 to scheduling_interval (depending on when the next execution date is)
>   
>  What should happen is that next_ds is always set to execution_date for manually triggered runs instead of the next schedule-interval aligned execution date.
>   
>  This _might_ break backwards compatibility for some users but it can be argued that the current functionality is a bug. If it's really desired we can create new aliases that behave logically although I am against this.
>   
>  prev_ds should probably also be made consistent with this logic.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)