[GitHub] [airflow] dstandish opened a new pull request #5372: [AIRFLOW-3057] add prev_*_date_success to template context

[GitBox <gi...@apache.org>]

dstandish opened a new pull request #5372: [AIRFLOW-3057] add prev_*_date_success to template context
URL: https://github.com/apache/airflow/pull/5372
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   My PR addresses [AIRFLOW-3057] - add macros that see only successful dags
   
   ### Description
   
   Two new variables are added to template context: 
   * `prev_execution_date_success`
   * `prev_start_date_success`.
   
   These return the exec / start date for the same task in prior successful dag run.   
   
   Lazy evaluation is employed so that the query to look up prev_ti is not executed unless a templated field references the variable in question.
   
   To enable the above, I added `state` as a param to `DagRun.get_previous_dagrun` and `TaskInstance._get_previous_ti`, and added properties `previous_ti_success` and `previous_execution_date_success` and `previous_start_date_success` to `TaskInstance`.  
   
   #### Notes on decisions
   These new context variables e.g. `previous_execution_date_success` will return the execution date of the same TI in the prior successful dag run.  Alternatively, I could have had them return the dates for the prior successful TI.  
   
   I decided not to do that so that (1) no new indexes were required and (2) I could reuse the `get_previous_dagrun` method with only a small change.  The only time this would give a different result would be if a trigger rule allowed the dag run to be successful despite the task failing, or perhaps if a task was run manually without a dag run (not sure if this is possible?).  But these issues did not seem to justify the slightly more elaborate changes it would require.  In any case, I have made it clear in the docs that the execution date is derived from last successful dag run.
   
   ### Tests
   
   My PR adds test `TaskInstanceTest.test_previous_dates`.  
   
   Previously there was no test of `previous_ti`.  I have added some testing of this method.  There were setup synergies in testing `previous_ti`, `previous_ti_success`, `previous_execution_date_success` and `previous_start_date_success` together, so that's what I did. 
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
     1. Subject is separated from body by a blank line
     1. Subject is limited to 50 characters (not including Jira issue reference)
     1. Subject does not end with a period
     1. Subject uses the imperative mood ("add", not "adding")
     1. Body wraps at 72 characters
     1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes how to use it.
     - All the public functions and the classes in the PR contain docstrings that explain what it does
     - If you implement backwards incompatible changes, please leave a note in the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so we can assign it to a appropriate release
   
   ### Code Quality
   
   - [x] Passes `flake8`
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services