You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "rokos-angus (via GitHub)" <gi...@apache.org> on 2023/06/02 17:44:32 UTC

[GitHub] [airflow] rokos-angus opened a new issue, #31689: Unable to specify dns_config for kubernetes pod executor

rokos-angus opened a new issue, #31689:
URL: https://github.com/apache/airflow/issues/31689

   ### Apache Airflow version
   
   2.6.1
   
   ### What happened
   
   Seem to be unable to specify dns_config in KubernetesPodOperator, despite it being an argument.
   
   `Broken DAG: [/opt/airflow/dags/repo/bug-report-dag.py] Traceback (most recent call last):
     File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 429, in apply_defaults
       result = func(self, **kwargs, default_args=default_args)
     File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 789, in __init__
       f"Invalid arguments were passed to {self.__class__.__name__} (task_id: {task_id}). "
   airflow.exceptions.AirflowException: Invalid arguments were passed to KubernetesPodOperator (task_id: task). Invalid arguments were:
   **kwargs: {'dns_config': {'nameservers': None, 'options': None, 'searches': ['mydomain.com']}}`
   
   ### What you think should happen instead
   
   dnsConfig should be correctly added to the pod manifest
   
   ### How to reproduce
   
   `from kubernetes.client import models as k8s
   from airflow import DAG
   from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import KubernetesPodOperator
   from airflow.utils.dates import days_ago
   from airflow.hooks.base_hook import BaseHook
   from airflow.contrib.operators.slack_webhook_operator import SlackWebhookOperator
   
   default_args = {
       'owner': 'airflow'
   }
   
   with DAG(
       dag_id='k8s-example-0',
       default_args=default_args,
       schedule_interval="*/5 * * * *",
       start_date=days_ago(2),
       catchup=False,
       tags=['k8s-pod-operator','example', 'python'],
   ) as dag:
       k = KubernetesPodOperator(
           namespace='airflow',
           image="python",
           cmds=["python", "-c"],
           arguments=["print('hello ...');import time;time.sleep(120)"],
           name="k8s-pod",
           task_id="task",
           is_delete_operator_pod=True,
           hostnetwork=False,
           startup_timeout_seconds=1000,
           kubernetes_conn_id=None
       )
   `
   
   ### Operating System
   
   Debian GNU/Linux 11 (bullseye)
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   Official helm chart via flux
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] rokos-angus commented on issue #31689: Unable to specify dns_config for kubernetes pod executor

Posted by "rokos-angus (via GitHub)" <gi...@apache.org>.
rokos-angus commented on issue #31689:
URL: https://github.com/apache/airflow/issues/31689#issuecomment-1576858562

   Thanks @eladkal 
   Docs are a bit vague, but I was able to create a custom image which pip installed that package over the base airflow image and this works.
   Is this the recommended approach?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] eladkal closed issue #31689: Unable to specify dns_config for kubernetes pod executor

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #31689: Unable to specify dns_config for kubernetes pod executor
URL: https://github.com/apache/airflow/issues/31689


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] boring-cyborg[bot] commented on issue #31689: Unable to specify dns_config for kubernetes pod executor

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #31689:
URL: https://github.com/apache/airflow/issues/31689#issuecomment-1574096524

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] eladkal commented on issue #31689: Unable to specify dns_config for kubernetes pod executor

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #31689:
URL: https://github.com/apache/airflow/issues/31689#issuecomment-1574208779

   `dns_config` was added in https://github.com/apache/airflow/pull/30992
   Please update to `apache-airflow-providers-cncf-kubernetes==7.0.0`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org