You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "rokos-angus (via GitHub)" <gi...@apache.org> on 2023/06/02 17:44:32 UTC
[GitHub] [airflow] rokos-angus opened a new issue, #31689: Unable to specify dns_config for kubernetes pod executor
rokos-angus opened a new issue, #31689:
URL: https://github.com/apache/airflow/issues/31689
### Apache Airflow version
2.6.1
### What happened
Seem to be unable to specify dns_config in KubernetesPodOperator, despite it being an argument.
`Broken DAG: [/opt/airflow/dags/repo/bug-report-dag.py] Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 429, in apply_defaults
result = func(self, **kwargs, default_args=default_args)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 789, in __init__
f"Invalid arguments were passed to {self.__class__.__name__} (task_id: {task_id}). "
airflow.exceptions.AirflowException: Invalid arguments were passed to KubernetesPodOperator (task_id: task). Invalid arguments were:
**kwargs: {'dns_config': {'nameservers': None, 'options': None, 'searches': ['mydomain.com']}}`
### What you think should happen instead
dnsConfig should be correctly added to the pod manifest
### How to reproduce
`from kubernetes.client import models as k8s
from airflow import DAG
from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import KubernetesPodOperator
from airflow.utils.dates import days_ago
from airflow.hooks.base_hook import BaseHook
from airflow.contrib.operators.slack_webhook_operator import SlackWebhookOperator
default_args = {
'owner': 'airflow'
}
with DAG(
dag_id='k8s-example-0',
default_args=default_args,
schedule_interval="*/5 * * * *",
start_date=days_ago(2),
catchup=False,
tags=['k8s-pod-operator','example', 'python'],
) as dag:
k = KubernetesPodOperator(
namespace='airflow',
image="python",
cmds=["python", "-c"],
arguments=["print('hello ...');import time;time.sleep(120)"],
name="k8s-pod",
task_id="task",
is_delete_operator_pod=True,
hostnetwork=False,
startup_timeout_seconds=1000,
kubernetes_conn_id=None
)
`
### Operating System
Debian GNU/Linux 11 (bullseye)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Official helm chart via flux
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rokos-angus commented on issue #31689: Unable to specify dns_config for kubernetes pod executor
Posted by "rokos-angus (via GitHub)" <gi...@apache.org>.
rokos-angus commented on issue #31689:
URL: https://github.com/apache/airflow/issues/31689#issuecomment-1576858562
Thanks @eladkal
Docs are a bit vague, but I was able to create a custom image which pip installed that package over the base airflow image and this works.
Is this the recommended approach?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #31689: Unable to specify dns_config for kubernetes pod executor
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #31689: Unable to specify dns_config for kubernetes pod executor
URL: https://github.com/apache/airflow/issues/31689
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #31689: Unable to specify dns_config for kubernetes pod executor
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #31689:
URL: https://github.com/apache/airflow/issues/31689#issuecomment-1574096524
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #31689: Unable to specify dns_config for kubernetes pod executor
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #31689:
URL: https://github.com/apache/airflow/issues/31689#issuecomment-1574208779
`dns_config` was added in https://github.com/apache/airflow/pull/30992
Please update to `apache-airflow-providers-cncf-kubernetes==7.0.0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org