You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "aru-trackunit (via GitHub)" <gi...@apache.org> on 2023/10/10 13:33:59 UTC

[I] Add custom labels to a kubernetes pod from dag [airflow]

aru-trackunit opened a new issue, #34848:
URL: https://github.com/apache/airflow/issues/34848

   ### Description
   
   Hi!
   
   It would be great if I could include custom kubernetes labels/annotation within kubernetes pod template coming from dag. 
   
   Meaning that if I add an argument or a list of arguments to a dag, then they should be displayed on a pod. 
   However airflow also supports different executors than Kubernetes and that's an unknown how to make a solution generic enough so it can mean something to other executors.
   
   ```
   metadata:
     annotations:
       dag_id: dag_numero_uno_task1
       run_id: scheduled__2023-10-09T00:00:00+00:00
       task_id: task1
       try_number: '1'
     labels:
       airflow-worker: '0'
       airflow_version: 2.7.1
       dag_id: asset-to-iso-feeds-scheduler
       kubernetes_executor: 'True'
       run_id: scheduled__2023-10-09T0000000000-d95390652
       task_id: create_stage_table_information
       try_number: '1'
       custom_label: value_from_dag
     name: dag_numero_uno_task1-id770eh4
     namespace: airflow
   ```
   
   Maybe there is already a way to do it and I just haven't encountered it?
   
   ### Use case/motivation
   
   We use airflow deployed on kubernetes cluster and looking into kubernetes integration tool that scrapes pods and describes them based on the custom annotations or labels. That feature can be used to assign a DAG to a specific team or order the criticality of issues etc.
   
   I hope I explained it well, let me know if there are some doubts
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Add custom labels to a kubernetes pod from dag [airflow]

Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on issue #34848:
URL: https://github.com/apache/airflow/issues/34848#issuecomment-1755591392

   I think you could try to use [executor_config](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/kubernetes.html#pod-override) Operator/Task Argument + [default_args](https://airflow.apache.org/docs/apache-airflow/stable/tutorial/fundamentals.html#default-arguments) DAG / TaskGroup argument


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Add custom labels to a kubernetes pod from dag [airflow]

Posted by "aru-trackunit (via GitHub)" <gi...@apache.org>.
aru-trackunit closed issue #34848: Add custom labels to a kubernetes pod from dag
URL: https://github.com/apache/airflow/issues/34848


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Add custom labels to a kubernetes pod from dag [airflow]

Posted by "aru-trackunit (via GitHub)" <gi...@apache.org>.
aru-trackunit commented on issue #34848:
URL: https://github.com/apache/airflow/issues/34848#issuecomment-1755679498

   Thank you @Taragolis for a suggestion it works as expected!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org