You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/10/21 00:17:02 UTC

[GitHub] [airflow] os1980 opened a new issue, #27172: Persistent DAG history

os1980 opened a new issue, #27172:
URL: https://github.com/apache/airflow/issues/27172

   ### Description
   
   DAGs are changing over time and the graph, grid and other views should be persistence to reflect the state of the DAG at that specific point in time.
   Currently the views are showing only the current definition of the DAG even for historical runs when the definition were different. 
   
   1. First definition with five tasks.
   
   <img width="1182" alt="dag_run_1" src="https://user-images.githubusercontent.com/10797855/197080950-3d3313bd-6556-4c09-ac15-dd1fc9447d03.png">
   
   2. Modification of the DAG and the removal of two tasks. 
   
   <img width="1131" alt="dag_run_2" src="https://user-images.githubusercontent.com/10797855/197081256-4ed73977-45bc-47ec-9597-26758b9eed07.png">
   
   3. Historical run view is also changed and reflects only the tasks of the current definition.
   
   <img width="1143" alt="dag_run_1_modified" src="https://user-images.githubusercontent.com/10797855/197081334-6d492650-a71e-4cb6-b81b-82d28aaf8123.png">
   
   Currently we lose the view of the history and the visibility of the other two tasks.
   
   
   ### Use case/motivation
   
   A nice feature will be that the views are persistence and showing the state of the DAGs at each specific point in time.
   The default view will be current definition, regardless of the running state instead of the last run.
   
   Using the screen shots above, the Graph view of 23:40 will show always the same five tasks even though the change of the DAG definition and the removal of two tasks.
   
   The run of 23:43 will show only the three tasks that are actually defined.
   A new drop down option is added to represent the current definition and state.
   <img width="1122" alt="Screenshot 2022-10-21 at 02 09 23" src="https://user-images.githubusercontent.com/10797855/197082527-6477326f-446c-4c14-922c-46f0d59441ba.png">
   
   Optionally in case of a running DAG it will show it progress by default.
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr closed issue #27172: Persistent DAG history

Posted by GitBox <gi...@apache.org>.
uranusjr closed issue #27172: Persistent DAG history
URL: https://github.com/apache/airflow/issues/27172


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr commented on issue #27172: Persistent DAG history

Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #27172:
URL: https://github.com/apache/airflow/issues/27172#issuecomment-1286401073

   See [AIP-36: DAG Versioning](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-36+DAG+Versioning). Feel free to contribute to the proposal.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #27172: Persistent DAG history

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #27172:
URL: https://github.com/apache/airflow/issues/27172#issuecomment-1286300651

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org