[I] Task and dagrun duration y-axis is incorrect if the last task has a different duration unit [airflow]

["tirkarthi (via GitHub)" <gi...@apache.org>]

tirkarthi opened a new issue, #38371:
URL: https://github.com/apache/airflow/issues/38371

   ### Apache Airflow version
   
   main (development)
   
   ### If "Other Airflow 2 version" selected, which one?
   
   _No response_
   
   ### What happened?
   
   During our internal testing it was found that the duration implementation we found that if the last taskinstance or if one of the taskinstance in the middle has a higher duration that leads to a different unit then the same unit applied to the previous durations and they are not scaled back. Below is an example where all taskinstances complete in seconds except for the last taskinstance which is more than a minute but all the durations are mentioned in minutes making it confusing. The same logic is present in dagrun duration as well. One solution would be to have everything as seconds and during chart construction we can scale them based on the max value. We have a fix for this tested internally and I will open a PR.
   
   ![image](https://github.com/apache/airflow/assets/3972343/eb5dd558-de69-44d0-a54a-4827ba797f18)
   
   
   ```python
   from datetime import datetime, timedelta
   
   from airflow import DAG
   from airflow.decorators import task
   from airflow.operators.empty import EmptyOperator
   
   from datetime import timedelta
   
   
   with DAG(
       dag_id="task_duration_average",
       start_date=datetime(2024, 1, 1),
       end_date=datetime(2024, 1, 15),
       catchup=True,
       schedule_interval="@daily",
   ) as dag:
   
       @task
       def sleeper(ds=None):
           import random, time, pendulum
   
           last_day = pendulum.datetime(2024, 1, 15)
           execution_date = pendulum.parse(ds)
   
           if execution_date == last_day:
               duration = 130
           else:
               duration = random.randint(1, 5)
   
           time.sleep(duration)
   
       sleeper()
   ```
   
   ### What you think should happen instead?
   
   _No response_
   
   ### How to reproduce
   
   1. Have a dag which takes few seconds and make last task instance take few minutes
   2. Look at the task duration and dagrun duration chart and the grid bars.
   
   ### Operating System
   
   Ubuntu
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other
   
   ### Deployment details
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Task and dagrun duration y-axis is incorrect if the last task has a different duration unit [airflow]

["bbovenzi (via GitHub)" <gi...@apache.org>]

bbovenzi closed issue #38371: Task and dagrun duration y-axis is incorrect if the last task has a different duration unit
URL: https://github.com/apache/airflow/issues/38371


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org