You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/03 08:25:45 UTC
[GitHub] [airflow] shivanitomarzeotap opened a new issue #20626: Not able to create k8s connection using airflow UI with kube config (json)
shivanitomarzeotap opened a new issue #20626:
URL: https://github.com/apache/airflow/issues/20626
### Apache Airflow version
2.1.4
### What happened
in_cluster configuration option is disabled, config path is empty and the kubeconfig(json) is provided to the airflow admin connection. As written [here](https://github.com/apache/airflow/blob/main/airflow/providers/cncf/kubernetes/hooks/kubernetes.py#L49-L56) because first two options are not selected, it should create the connection with provided kubeconfig json file.
But it fails saying that - _Invalid kube-config file_
### What you expected to happen
If in_cluster configuration and config path options are disabled, it should create the connection with provided kubeconfig json file.
### How to reproduce
Create airflow admin connection via airflow UI where in_cluster option is disabled and config path is empty.
<img width="877" alt="Screenshot 2022-01-03 at 1 42 34 PM" src="https://user-images.githubusercontent.com/52529737/147910225-5e1cac10-d854-42da-98cb-2c5fac6298a1.png">
and use this connection for [dag](https://github.com/apache/airflow/blob/main/airflow/providers/cncf/kubernetes/example_dags/example_spark_kubernetes.py)
### Operating System
-
### Versions of Apache Airflow Providers
_No response_
### Deployment
Composer
### Deployment details
_No response_
### Anything else
hypothesis - The connection form includes empty string for un-set values, and in code we only have a check for None and not for empty string.
Traceback - `[2022-01-03 07:54:31,547] {spark_kubernetes.py:68} INFO - Creating sparkApplication
[2022-01-03 07:54:31,586] {taskinstance.py:1465} ERROR - Task failed with exception
Traceback (most recent call last):
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1166, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1285, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1315, in _execute_task
result = task_copy.execute(context=context)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py", line 70, in execute
response = hook.create_custom_object(
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py", line 168, in create_custom_object
api = client.CustomObjectsApi(self.api_client)
File "/opt/python3.8/lib/python3.8/functools.py", line 967, in __get__
val = self.func(instance)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py", line 149, in api_client
return self.get_conn()
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py", line 127, in get_conn
config.load_kube_config(
File "/opt/python3.8/lib/python3.8/site-packages/kubernetes/config/kube_config.py", line 737, in load_kube_config
loader = _get_kube_config_loader_for_yaml_file(
File "/opt/python3.8/lib/python3.8/site-packages/kubernetes/config/kube_config.py", line 700, in _get_kube_config_loader_for_yaml_file
raise ConfigException(
kubernetes.config.config_exception.ConfigException: Invalid kube-config file. No configuration found.`
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] shivanitomarzeotap commented on issue #20626: Not able to create k8s connection using airflow UI with kube config (json)
Posted by GitBox <gi...@apache.org>.
shivanitomarzeotap commented on issue #20626:
URL: https://github.com/apache/airflow/issues/20626#issuecomment-1003935424
cc - @dstandish
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish closed issue #20626: Not able to create k8s connection using airflow UI with kube config (json)
Posted by GitBox <gi...@apache.org>.
dstandish closed issue #20626:
URL: https://github.com/apache/airflow/issues/20626
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on issue #20626: Not able to create k8s connection using airflow UI with kube config (json)
Posted by GitBox <gi...@apache.org>.
dstandish commented on issue #20626:
URL: https://github.com/apache/airflow/issues/20626#issuecomment-1040497575
i think this is resolved by https://github.com/apache/airflow/pull/19695 @shivanitomarzeotap
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish closed issue #20626: Not able to create k8s connection using airflow UI with kube config (json)
Posted by GitBox <gi...@apache.org>.
dstandish closed issue #20626:
URL: https://github.com/apache/airflow/issues/20626
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on issue #20626: Not able to create k8s connection using airflow UI with kube config (json)
Posted by GitBox <gi...@apache.org>.
dstandish commented on issue #20626:
URL: https://github.com/apache/airflow/issues/20626#issuecomment-1040497575
i think this is resolved by https://github.com/apache/airflow/pull/19695 @shivanitomarzeotap
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #20626: Not able to create k8s connection using airflow UI with kube config (json)
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #20626:
URL: https://github.com/apache/airflow/issues/20626#issuecomment-1003933982
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org