You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/08 10:08:53 UTC
[GitHub] [airflow] fboissier opened a new issue #21421: Unable to use EKSPodOperator with aws_conn_id parameter
fboissier opened a new issue #21421:
URL: https://github.com/apache/airflow/issues/21421
### Apache Airflow Provider(s)
amazon
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==2.4.0
### Apache Airflow version
2.2.2
### Operating System
AmazonLinux
### Deployment
MWAA
### Deployment details
Out of box deployment of MWAA with Airflow 2.2.2
### What happened
I try to launch a K8S pod using [EKSPodOperator](https://airflow.apache.org/docs/apache-airflow-providers-amazon/2.4.0/_api/airflow/providers/amazon/aws/operators/eks/index.html#airflow.providers.amazon.aws.operators.eks.EKSPodOperator) and the parameter **aws_conn_id** in order to authenticate to the EKS cluster through IAM / OIDC provider.
The pod is not starting and i've got the following error in my Airflow task log :
```
[2022-02-08, 10:55:30 CET] {{refresh_config.py:71}} ERROR - exec: process returned 1. Traceback (most recent call last):
File "/usr/lib64/python3.7/runpy.py", line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/lib64/python3.7/runpy.py", line 109, in _get_module_details
__import__(pkg_name)
File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 34, in <module>
from airflow import settings
File "/usr/local/lib/python3.7/site-packages/airflow/settings.py", line 35, in <module>
from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401
File "/usr/local/lib/python3.7/site-packages/airflow/configuration.py", line 1129, in <module>
conf.validate()
File "/usr/local/lib/python3.7/site-packages/airflow/configuration.py", line 226, in validate
self._validate_enums()
File "/usr/local/lib/python3.7/site-packages/airflow/configuration.py", line 253, in _validate_enums
+ f"{value!r}. Possible values: {', '.join(enum_options)}."
airflow.exceptions.AirflowConfigException: `[logging] logging_level` should not be 'fatal'. Possible values: CRITICAL, FATAL, ERROR, WARN, WARNING, INFO, DEBUG.
```
Indeed, the [EKSHook](https://github.com/apache/airflow/blob/providers-amazon/2.4.0/airflow/providers/amazon/aws/hooks/eks.py#L596) is setting **AIRFLOW__LOGGING__LOGGING_LEVEL** to **fatal** and [airflow core](https://github.com/apache/airflow/blob/2.2.2/airflow/configuration.py#L198) is checking that logging level is **FATAL**.
It seems we've got a case sensitive problem.
### What you expected to happen
I expected my POD starting using the IAM / OIDC provider auth on EKS.
### How to reproduce
```python
start_pod = EKSPodOperator(
task_id="run_pod",
cluster_name="<your_eks_cluster_name>",
pod_name="run_pod",
image="amazon/aws-cli:latest",
cmds=["sh", "-c", "ls"],
labels={"demo": "hello_world"},
get_logs=True,
# Delete the pod when it reaches its final state, or the execution is interrupted.
is_delete_operator_pod=True,
aws_conn_id="<your_aws_airflow_conn_id>"
)
```
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #21421: Unable to use EKSPodOperator with aws_conn_id parameter
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #21421:
URL: https://github.com/apache/airflow/issues/21421#issuecomment-1032431705
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #21421: Unable to use EKSPodOperator with aws_conn_id parameter
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #21421:
URL: https://github.com/apache/airflow/issues/21421
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org