You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "alixirs (via GitHub)" <gi...@apache.org> on 2026/06/01 13:10:04 UTC

[I] cncf.kubernetes: support kubernetes client 36.x (>=36.0.1) - 35.x no_proxy regression breaks KubernetesPodOperator/hooks behind a corporate proxy [airflow]

alixirs opened a new issue, #67845:
URL: https://github.com/apache/airflow/issues/67845

   ### Under which category would you file this issue?
   
   Providers
   
   ### Apache Airflow version
   
   3.2.1
   
   ### What happened and how to reproduce it?
   
   The provider pins `kubernetes>=35.0.0,<36.0.0` and `kubernetes_asyncio>=32.0.0,<36.0.0`. The only kubernetes release in [35.0.0, 36.0.0) is 35.0.0, which carries the upstream `no_proxy` regression `kubernetes-client/python#2520` - "no_proxy environment variable ignored in kubernetes-client Python library". The client then routes every request through HTTP(S)_PROXY, including in-cluster API-server traffic to KUBERNETES_SERVICE_HOST.
   
   For users behind a corporate proxy, this breaks every provider component that talks to the in-cluster API, most visibly KubernetesPodOperator (and KubernetesHook / AsyncKubernetesHook): the calls to create and monitor the target pod are sent to the proxy, which can't route to a cluster-internal IP, so they time out:
   
   ```
   ReadTimeoutError("HTTPSConnectionPool(host='10.x.x.x', port=443): Read timed out. (read timeout=30)")
   ```
   
   `nc -zv <KUBERNETES_SERVICE_HOST> 443` from the same pod connects fine, confirming it's proxy routing, not connectivity. The `no_proxy` fix landed only in kubernetes 36.0.0, which the provider bound excludes, so there is currently no proxy-respecting version installable within the supported range.
   
   Minimal (provider only, no executor):
   
   1. Airflow 3.2.1 + `apache-airflow-providers-cncf-kubernetes==10.17.1`, default `kubernetes==35.0.0`, running in-cluster.
   2. Set `HTTP_PROXY/HTTPS_PROXY` and `NO_PROXY` (incl. in-cluster ranges, `.svc`, `.svc.cluster.local`) in the pod env.
   3. Run a `KubernetesPodOperator` task (or just `KubernetesHook().core_v1_client.list_namespaced_pod(<ns>)`). The in-cluster API call is sent to the proxy and times out; `no_proxy` is ignored.
   4. Pin `kubernetes==36.0.1` - the proxy/auth issues resolve, but the provider then needs validation against v36's API changes.
   
   ### What you think should happen instead?
   
   cncf.kubernetes should support the `kubernetes` client 36.x, specifically ≥36.0.1, so proxied deployments have a working option. This needs validation, not just a bound bump:
   
   •  36.0.0 is not a viable floor. It shipped an in-cluster auth regression: `load_incluster_config()` writes `api_key['authorization']` but `auth_settings()` now expects `api_key['BearerToken']` ([#2582](https://github.com/kubernetes-client/python/issues/2582) / [#2584](https://github.com/kubernetes-client/python/issues/2584)) — so any provider code using in-cluster config (KPO, hooks) goes out as system:anonymous (401/403). Fixed in 36.0.1 ([#2585](https://github.com/kubernetes-client/python/issues/2585)).
   •  v36 carries breaking API changes from the OpenAPI generator 6.6.0 bump (see the `release-36.0` CHANGELOG, "Breaking Change from upgrading openapi generator to v6.6.0"; downstream fallout e.g. `ApiClient.__new__` changes, nolar/kopf#1307). KPO, the hooks, and the triggers call into the client directly and likely need adapting.
   •  `kubernetes_asyncio` has no 36.x release yet, so `kubernetes_asyncio<36.0.0` can't move until upstream publishes one. This gates `AsyncKubernetesHook` / deferrable KPO and probably means the sync bound has to move ahead of the async one.
   
   ### Operating System
   
   Astro Runtime Version: 3.2-4 PRETTY_NAME="Debian GNU/Linux 13 (trixie)" NAME="Debian GNU/Linux" VERSION_ID="13" VERSION="13 (trixie)" VERSION_CODENAME=trixie DEBIAN_VERSION_FULL=13.4
   
   ### Deployment
   
   Astronomer
   
   ### Apache Airflow Provider(s)
   
   cncf-kubernetes
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-cncf-kubernetes==10.17.1
   
   ### Official Helm Chart version
   
   Not Applicable
   
   ### Kubernetes Version
   
   1.33.12-gke.1059000
   
   ### Helm Chart configuration
   
   Not Applicable
   
   ### Docker Image customizations
   
   ```
   FROM astrocrpublic.azurecr.io/runtime:3.2-4
   
   ENV HTTP_PROXY="http://tinyproxy.proxy.svc.cluster.local:8888"
   ENV HTTPS_PROXY="http://tinyproxy.proxy.svc.cluster.local:8888"
   ENV NO_PROXY="localhost,127.0.0.0/8,19.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.ford.com,.appspot.com,.cloudfunctions.net,.cloudproxy.app,.datafusion.cloud.google.com,.datafusion.googleusercontent.com,.gcr.io,.internal,.googleadapis.com,.googleapis.com,.gstatic.com,.ltsapis.goog,.packages.cloud.google.com,.pkg.dev,.pki.goog,.run.app,.source.developers.google.com,.gcp.cloud.es.io,.smt-gce.susecloud.net"
   ENV http_proxy="http://tinyproxy.proxy.svc.cluster.local:8888"
   ENV https_proxy="http://tinyproxy.proxy.svc.cluster.local:8888"
   ENV no_proxy="localhost,127.0.0.0/8,19.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.ford.com,.appspot.com,.cloudfunctions.net,.cloudproxy.app,.datafusion.cloud.google.com,.datafusion.googleusercontent.com,.gcr.io,.internal,.googleadapis.com,.googleapis.com,.gstatic.com,.ltsapis.goog,.packages.cloud.google.com,.pkg.dev,.pki.goog,.run.app,.source.developers.google.com,.gcp.cloud.es.io,.smt-gce.susecloud.net"
   ```
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] cncf.kubernetes: support kubernetes client 36.x (>=36.0.1) - 35.x no_proxy regression breaks KubernetesPodOperator/hooks behind a corporate proxy [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on issue #67845:
URL: https://github.com/apache/airflow/issues/67845#issuecomment-4626325607

   `kubernetes_asyncio` 36.1 was released: https://github.com/tomplus/kubernetes_asyncio/issues/424#issuecomment-4625756834, thanks to the maintainer for a quick turnaround.
   
   https://github.com/apache/airflow/pull/68041 bumps the requirement


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] cncf.kubernetes: support kubernetes client 36.x (>=36.0.1) - 35.x no_proxy regression breaks KubernetesPodOperator/hooks behind a corporate proxy [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil closed issue #67845: cncf.kubernetes: support kubernetes client 36.x (>=36.0.1) - 35.x no_proxy regression breaks KubernetesPodOperator/hooks behind a corporate proxy
URL: https://github.com/apache/airflow/issues/67845


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] cncf.kubernetes: support kubernetes client 36.x (>=36.0.1) - 35.x no_proxy regression breaks KubernetesPodOperator/hooks behind a corporate proxy [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on issue #67845:
URL: https://github.com/apache/airflow/issues/67845#issuecomment-4596947203

   Looks like we will have a release supporting `36.x` for `kubernetes_asyncio` before next week:
   
   https://github.com/tomplus/kubernetes_asyncio


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] cncf.kubernetes: support kubernetes client 36.x (>=36.0.1) - 35.x no_proxy regression breaks KubernetesPodOperator/hooks behind a corporate proxy [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #67845:
URL: https://github.com/apache/airflow/issues/67845#issuecomment-4592884588

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org