You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2023/01/03 03:46:48 UTC
[GitHub] [airflow] yxiao1996 opened a new pull request, #28686: Update gantt chart UI to display queued state of tasks
yxiao1996 opened a new pull request, #28686:
URL: https://github.com/apache/airflow/pull/28686
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
related: [#28341](https://github.com/apache/airflow/issues/28341)
### Description
This change updates the UI of gantt charts to display queued state of each tasks(represented as gray boxes). Also, tooltips are added to the queued state boxes in gantt charts to provide related info(start time, duration, etc.). Previously, the gantt chart only shows the tasks after they have start. By having queued state tracking in Gantt chart, it would be easier for airflow users to troubleshoot performance issue in DAG's.
### Testing
Tested this change in my local airflow dev environment using example DAG's.
Screenshot of updated gantt chart of a sample DAG run:
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi merged pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
bbovenzi merged PR #28686:
URL: https://github.com/apache/airflow/pull/28686
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yxiao1996 commented on pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
yxiao1996 commented on PR #28686:
URL: https://github.com/apache/airflow/pull/28686#issuecomment-1371162750
```
Unknown warnings generated:
/usr/local/lib/python3.7/site-packages/airflow/models/base.py:49:[31mDeprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. [32mTo prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". [36mSet environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable
SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message.[0m (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
ERROR! There were 1 warnings generated during the import
```
Seems unrelated to my change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yxiao1996 commented on pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
yxiao1996 commented on PR #28686:
URL: https://github.com/apache/airflow/pull/28686#issuecomment-1387510337
@bbovenzi Hi Brent, could you please help me take another look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi commented on a diff in pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
bbovenzi commented on code in PR #28686:
URL: https://github.com/apache/airflow/pull/28686#discussion_r1062423381
##########
airflow/www/static/js/gantt.js:
##########
@@ -212,12 +229,23 @@ d3.gantt = () => {
mapIndex: d.map_index,
});
})
- .attr('class', (d) => d.state || 'null')
+ .attr('class', (d) => `${d.state || 'null'} all-tasks`)
.attr('y', 0)
.attr('transform', rectTransform)
.attr('height', () => y.rangeBand())
.attr('width', (d) => d3.max([x(d.end_date.valueOf()) - x(d.start_date.valueOf()), 1]));
+ svg.selectAll('.chart')
Review Comment:
Could we add a comment above this svg and the other one to explain that its painting the instance run and the instance queued?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi commented on a diff in pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
bbovenzi commented on code in PR #28686:
URL: https://github.com/apache/airflow/pull/28686#discussion_r1062421784
##########
airflow/www/static/js/gantt.js:
##########
@@ -157,9 +164,19 @@ d3.gantt = () => {
if (!(a.end_date instanceof moment)) {
a.end_date = moment(a.end_date);
}
+ if (a.queued_dttm) {
Review Comment:
```suggestion
if (a.queued_dttm && !(a.queued_dttm instanceof moment)) {
```
You can combine the if statements
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on PR #28686:
URL: https://github.com/apache/airflow/pull/28686#issuecomment-1396304738
Awesome work, congrats on your first merged pull request!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yxiao1996 commented on a diff in pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
yxiao1996 commented on code in PR #28686:
URL: https://github.com/apache/airflow/pull/28686#discussion_r1062837601
##########
airflow/www/static/js/gantt.js:
##########
@@ -157,9 +164,19 @@ d3.gantt = () => {
if (!(a.end_date instanceof moment)) {
a.end_date = moment(a.end_date);
}
+ if (a.queued_dttm) {
Review Comment:
will do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yxiao1996 commented on a diff in pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
yxiao1996 commented on code in PR #28686:
URL: https://github.com/apache/airflow/pull/28686#discussion_r1062839814
##########
airflow/www/static/js/gantt.js:
##########
@@ -212,12 +229,23 @@ d3.gantt = () => {
mapIndex: d.map_index,
});
})
- .attr('class', (d) => d.state || 'null')
+ .attr('class', (d) => `${d.state || 'null'} all-tasks`)
.attr('y', 0)
.attr('transform', rectTransform)
.attr('height', () => y.rangeBand())
.attr('width', (d) => d3.max([x(d.end_date.valueOf()) - x(d.start_date.valueOf()), 1]));
+ svg.selectAll('.chart')
Review Comment:
sounds good, will add comment here to mark the line painting all instances vs. instance queued states. also will add similar comments in redraw method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yxiao1996 commented on pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
yxiao1996 commented on PR #28686:
URL: https://github.com/apache/airflow/pull/28686#issuecomment-1378090326
can anyone help me approve the workflow?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yxiao1996 commented on pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
yxiao1996 commented on PR #28686:
URL: https://github.com/apache/airflow/pull/28686#issuecomment-1370303473
Looks like static checks failed, will fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on PR #28686:
URL: https://github.com/apache/airflow/pull/28686#issuecomment-1369366171
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (flake8, mypy and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi commented on a diff in pull request #28686: Update gantt chart UI to display queued state of tasks
Posted by GitBox <gi...@apache.org>.
bbovenzi commented on code in PR #28686:
URL: https://github.com/apache/airflow/pull/28686#discussion_r1063471535
##########
airflow/www/static/js/gantt.js:
##########
@@ -212,12 +229,23 @@ d3.gantt = () => {
mapIndex: d.map_index,
});
})
- .attr('class', (d) => d.state || 'null')
+ .attr('class', (d) => `${d.state || 'null'} all-tasks`)
.attr('y', 0)
.attr('transform', rectTransform)
.attr('height', () => y.rangeBand())
.attr('width', (d) => d3.max([x(d.end_date.valueOf()) - x(d.start_date.valueOf()), 1]));
+ svg.selectAll('.chart')
Review Comment:
Just need to rebase too, to fix the CI issue. And I think this is looking good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org