You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "asaff1 (via GitHub)" <gi...@apache.org> on 2024/10/29 12:51:11 UTC

[I] Adding tags to DAG runs [airflow]

asaff1 opened a new issue, #43472:
URL: https://github.com/apache/airflow/issues/43472

   ### Description
   
   Currently I see only option to tag DAGs. I don't see such an option for DAG runs.
   It will be quite useful for me, to add and filter runs by tags.
   
   ### Use case/motivation
   
   I'd like to filter runs based on tags. For example, I have different DAGs  (predict, train, etc.) that I could tag with tags like 'model_name=mymodel', then I can see all runs that used that model.
   This also gives the user a way to group dagruns based on keys like experiment=name, such management exists in other dag runners (e.g kubeflow)
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "asaff1 (via GitHub)" <gi...@apache.org>.
asaff1 commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2445356673

   @potiuk I might be able to contribute it. I can submit a PR. Any guidelines for that?
   
   As I see what needs to be done, is to add a new DB model `DagRunTag` with key, value fields (in models/dagrun.py),
    then adding a many to many relationship between `DagRun` and `DagRunTag`.
   Then the API needs to be changed, and for the UI I need to add a filtering / search by tags.
   Is that a good direction?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2445296724

   Yep. Would be a great feature to have. Maybe someone will pick it up and implement (but FYI. The fastest way to get something like tha timplemented, is to contribute it, second fastest is to find someone who will contribute it  - and for example pay them to do so - other than that someone will have to volunteer and implement it - and since this is an open-source project, it's on a discretion of whoever signs up and take that one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2445394498

   Yep. And it could be done in stages as well You could even add the list of things to do in the description of this issue with 
   
   - [ ] thing to do
   - [ ] another thing to do
   
   And check them one-by-one while doing it. 
   
   Also I think that one needs a say from @bbovenzi and @pierrejeambrun from the UI perspective, because if we implement something like that, it needs to someohow fit into navigation patterns in the new UI 
   
   (Note that this might be Airflow 3 - only change anyway) 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "asaff1 (via GitHub)" <gi...@apache.org>.
asaff1 commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2447402407

   @pierrejeambrun For my use case it is beneficial enough to set tags at creation time, or manually after creation (from the UI / API).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "nuncio8 (via GitHub)" <gi...@apache.org>.
nuncio8 commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2922616158

   Can i be assigned to this?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2931205400

   > I was thinking about using the Run ID for this. Would it work properlly? Where should the filter be added?
   
   I'am not sure I understand this statement can you elaborate on how this will help being able to set arbitrary tags to a DagRun @nuncio8?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2447510452

   > For my use case it is beneficial enough to set tags at creation time, or manually after creation (from the UI / API).
   
   `Set tags at creation time` what do you mean by that ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2940598962

   What do you mean by `changee the Run ID and then filter through it`? Where would dag run tags live?
   
   Feel free to open a PR. Yes you can request for my review, otherwise code owners of the relevant part of the code you are updating will be notified.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2447289071

   Interesting. I think we can make that work in the UI in terms of display and filtering in the DagRun table.
   
   The thing I am wondering is how and when will tags be added to a specific dagrun ? For dags it is a dag authoring time, which makes sense, for `dag_runs` it is less clear to me. Will a user need  to do that manually, or do the runs inherit the `DAG` tags by default, etc, etc


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "asaff1 (via GitHub)" <gi...@apache.org>.
asaff1 commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2447540086

   @pierrejeambrun I mean, setting tags when manually triggering DAGs via the UI, or by the REST API. (So I mean, at the creation time of dag runs.)
   
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2447437378

   First, before we add a new table. Let's see if we can use or improve existing fields to achieve this.
   
   - Filter the dags for which you want to see dag runs. 1 filter dags by tags, 2 show me only those dag runs
   - I wonder if we can filter by dag_run.conf values.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2444125925

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "nuncio8 (via GitHub)" <gi...@apache.org>.
nuncio8 commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2925425482

   I was thinking about using the Run ID for this. Would it work properlly? Where should the filter be added?
   
   ![Image](https://github.com/user-attachments/assets/6e7fd805-33f8-467e-aa37-88c5e94c2431)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Adding tags to DAG runs [airflow]

Posted by "nuncio8 (via GitHub)" <gi...@apache.org>.
nuncio8 commented on issue #43472:
URL: https://github.com/apache/airflow/issues/43472#issuecomment-2936756931

   > I'am not sure I understand this statement, can you elaborate on how this will help setting arbitrary tags on a DagRun [@nuncio8](https://github.com/nuncio8)?
   
   yes, i have further developed and thought it would be better to use a tag. but my ideia was to change the Run ID and then filter through it, instead of adding a new parameter. I am almost ready to make a pull request, I wonder if i can assign you to review it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org