You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/10/06 07:05:40 UTC
[GitHub] [airflow] flinz opened a new issue, #26905: Display selected task outputs (xcom) in task UI
flinz opened a new issue, #26905:
URL: https://github.com/apache/airflow/issues/26905
### Description
I often find myself checking the stats of a passed task, e.g. "inserted 3 new rows" or "discovered 4 new files" in the task logs. It would be very handy to see these on the UI directly, as part of the task details or elsewhere.
One idea would be to choose in the Task definition, which XCOM keys should be output in the task details, like so:
### Use case/motivation
As a developer, I want to better monitor the results of my tasks in terms of key metrics, so I can see the data processed by them. While for production, this can be achieved by forwarding/outputting metrics to other systems, like notification hooks, or ingesting them into e.g. grafana, I would like to do this already in AirFlow to a certain extent. This would certainly cut down on my clicks while running beta DAGs.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] flinz commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by GitBox <gi...@apache.org>.
flinz commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1269878752
@eladkal indeed, this would be similar to the display of the xcom, but selecting some of those to be exposed in the UI separately. My proposed solution above maybe would duplicate the xcom tab information so maybe that's not the route to go.
I was looking for a way to "expose a high level digest of task results in the UI" to make these available at a glance, instead of hiding them a few levels down in xcom.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Display selected task outputs (xcom) in task UI [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi closed issue #26905: Display selected task outputs (xcom) in task UI
URL: https://github.com/apache/airflow/issues/26905
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by GitBox <gi...@apache.org>.
bbovenzi commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1285822845
We're trying to bring more at-a-glance info into the grid view, like logs. This would be a good one too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1533105612
@parth166 Unassigned you. But are you on the Airflow Slack? Reach out to me there and I'll help you get started!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Display selected task outputs (xcom) in task UI [airflow]
Posted by "hduong-mwam (via GitHub)" <gi...@apache.org>.
hduong-mwam commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1817496200
Hi, I took a stab at this issue [PR #35719](https://github.com/apache/airflow/pull/35719). It's my first time PR-ing to airflow so would appreciate any comments!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by GitBox <gi...@apache.org>.
bbovenzi commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1353731570
If anyone wants to pick it up:
- we can add an "Xcoms" tab like the details/logs tabs we have
- we can also use the [list xcoms](https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/get_xcom_entries), and [get xcom entry](https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/get_xcom_entry) rest API endpoints
It should be relatively straightforward to follow how the rest of our API requests and react pages work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] parth166 commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by "parth166 (via GitHub)" <gi...@apache.org>.
parth166 commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1533040861
Hey @potiuk I haven't been able to work on this since I was trying to work on a different issue and I am only a beginner in airflow. Can you assign this to someone else and perhaps later I can follow the resolution of this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] parth166 commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by "parth166 (via GitHub)" <gi...@apache.org>.
parth166 commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1521054578
Hi!
I am new to airflow and would like to contribute. I just completed the setup of my environment. Was wondering if I can pick this up?
Thanks.
> If anyone wants to pick it up:
>
> * we can add an "Xcoms" tab like the details/logs tabs we have
> * we can also use the [list xcoms](https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/get_xcom_entries), and [get xcom entry](https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/get_xcom_entry) rest API endpoints
>
> It should be relatively straightforward to follow how the rest of our API requests and react pages work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1269612128
I'm not sure what is the request here?
There is `Xcom` tab.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pierrejeambrun commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by GitBox <gi...@apache.org>.
pierrejeambrun commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1319106530
Continuing our work to modernize the UI, I think adding a new xcom tab to task details to display xcom entries would be a good idea. I believe we already have an endpoint for retrieving XCOM entries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1269452331
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26905: Display selected task outputs (xcom) in task UI
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #26905:
URL: https://github.com/apache/airflow/issues/26905#issuecomment-1521892094
sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org