You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "huynhsontung (via GitHub)" <gi...@apache.org> on 2026/05/09 01:42:29 UTC
[PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
huynhsontung opened a new pull request, #66616:
URL: https://github.com/apache/airflow/pull/66616
Agent-Logs-Url: https://github.com/huynhsontung/airflow/sessions/be394b63-9852-4a0f-a4b9-6d9e14c14f29
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
## Summary
Adds `ipFamilyPolicy` and `ipFamilies` fields to all Airflow Helm chart service templates to support IPv6 and dual-stack networking configurations.
## Changes
### Service Templates Updated
All 9 service templates now conditionally include `ipFamilyPolicy` and `ipFamilies` when set:
- `api-server-service.yaml`
- `flower-service.yaml`
- `redis-service.yaml`
- `pgbouncer-service.yaml`
- `statsd-service.yaml`
- `otel-collector-service.yaml`
- `worker-service.yaml` (headless)
- `triggerer-service.yaml` (headless)
- `scheduler-service.yaml` (headless)
### `values.yaml`
- Added `ipFamilyPolicy: ~` and `ipFamilies: []` defaults to all service sections
- Added new `service:` sections to `workers`, `scheduler`, and `triggerer` (previously had no configurable service options)
- Defaults are `null`/empty, preserving existing behavior unless explicitly set
### `values.schema.json`
- Added `ipFamilyPolicy` with enum validation (`SingleStack`, `PreferDualStack`, `RequireDualStack`, or `null`)
- Added `ipFamilies` with item enum validation (`IPv4`, `IPv6`)
- Added `service` property definitions for `workers`, `scheduler`, and `triggerer`
### Tests
Added tests to all relevant test files verifying:
- `ipFamilyPolicy` and `ipFamilies` are rendered when configured
- Neither field is present by default (no breaking change)
## Usage Example
```yaml
# Enable dual-stack on the API server
apiServer:
service:
ipFamilyPolicy: PreferDualStack
ipFamilies:
- IPv4
- IPv6
# Force IPv6-only on redis
redis:
service:
ipFamilyPolicy: SingleStack
ipFamilies:
- IPv6
```
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [x] Yes (please specify the tool below)
GitHub Copilot (Claude Sonnet 4.6)
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #66616:
URL: https://github.com/apache/airflow/pull/66616#discussion_r3367134518
##########
chart/tests/helm_tests/otel_collector/test_otel_collector.py:
##########
@@ -400,6 +400,24 @@ def test_service_no_annotations_block_when_unset(self):
docs = render_chart(values={"otelCollector": {"tracesEnabled": True}}, show_only=[SERVICE_TEMPLATE])
assert "annotations" not in jmespath.search("metadata", docs[0])
+ def test_ip_family_policy(self):
+ docs = render_chart(
+ values={
+ "ipFamilyPolicy": "PreferDualStack",
+ "ipFamilies": ["IPv4", "IPv6"],
+ },
+ show_only=[SERVICE_TEMPLATE],
+ )
Review Comment:
`otel-collector-service.yaml` is only rendered when `otelCollector.tracesEnabled` or `otelCollector.metricsEnabled` is true. This test does not enable either flag, so `render_chart(..., show_only=[SERVICE_TEMPLATE])` will return no docs and `docs[0]` will fail.
##########
chart/tests/helm_tests/otel_collector/test_otel_collector.py:
##########
@@ -400,6 +400,24 @@ def test_service_no_annotations_block_when_unset(self):
docs = render_chart(values={"otelCollector": {"tracesEnabled": True}}, show_only=[SERVICE_TEMPLATE])
assert "annotations" not in jmespath.search("metadata", docs[0])
+ def test_ip_family_policy(self):
+ docs = render_chart(
+ values={
+ "ipFamilyPolicy": "PreferDualStack",
+ "ipFamilies": ["IPv4", "IPv6"],
+ },
+ show_only=[SERVICE_TEMPLATE],
+ )
+
+ assert jmespath.search("spec.ipFamilyPolicy", docs[0]) == "PreferDualStack"
+ assert jmespath.search("spec.ipFamilies", docs[0]) == ["IPv4", "IPv6"]
+
+ def test_ip_family_policy_not_set_by_default(self):
+ docs = render_chart(show_only=[SERVICE_TEMPLATE])
+
+ assert jmespath.search("spec.ipFamilyPolicy", docs[0]) is None
Review Comment:
`otel-collector-service.yaml` is gated behind `otelCollector.tracesEnabled`/`metricsEnabled`, so rendering it with no values will likely produce `docs == []` and make `docs[0]` crash. Enable the service in the test while leaving `ipFamilyPolicy`/`ipFamilies` unset to assert the default rendering behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4411002396
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our [Contributors' Guide](https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:
- Pay attention to the quality of your code (ruff, mypy and type annotations). Our [prek-hooks]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-prek-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/airflow-core/docs/howto/custom-operator.rst) Consider adding an example Dag that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices).
- Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "huynhsontung (via GitHub)" <gi...@apache.org>.
huynhsontung commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4595034321
@jscheffl Having a global IPv6 switch would work for my use case. For some context, I originally wanted to extend the existing component service blocks, but then some components don't have a service block at all. I can see that having an IPv6 switch for each component can enable deployment scenarios with nodes on different network stacks or different network config for internal vs. external communication. For example, enabling dual-stack only on the public API server while using IPv4 only for internal communication between components. Although, this configuration may be less common. I have no issue with having a global network stack config switch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4642027626
Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl merged PR #66616:
URL: https://github.com/apache/airflow/pull/66616
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "huynhsontung (via GitHub)" <gi...@apache.org>.
huynhsontung commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4620387112
@jscheffl @bugraoz93 Agreed. How do you suggest the global switch should be structured? Should I move the two fields `ipFamilies` and `ipFamilyPolicy` to the top level? Should it be under a root `service` block? Or should it be a simple boolean like `enableIPv6`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "huynhsontung (via GitHub)" <gi...@apache.org>.
huynhsontung commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4513947371
Test deployed on a cluster. All services, including headless services, get dual-stack IP addresses when the `ipFamilyPolicy` is set to `PreferDualStack`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "huynhsontung (via GitHub)" <gi...@apache.org>.
huynhsontung commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4637071746
Updated to use top-level `ipFamilies` and `ipFamilyPolicy`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4595943580
I'd favor a single switch. Main reason is we have already too many config options and if it is a rare case then somebody can Kustomize around w/o need to embed complexity into the core chart. But as in the next 20 years or so the infrastrcture will slowly migrate away from IPv4 I think this optio is reasonable to have in the chart as a core feature.
How about other maintainers opinion? @Miretpl @potiuk @bugraoz93 @jedcunningham ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4582638382
Thanks for your contribution. Took a moment until I got back to it.
Sanity question to better understand - we have no IPv6 challenge in our env (yet) - does it make sense to set IPv6 on a per component base? This adds a lot of config switches. Would it not rather make sense having a global IPv4/IPv6 config switch that applies to all components of the deployment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4606366527
> I'd favor a single switch. Main reason is we have already too many config options and if it is a rare case then somebody can Kustomize around w/o need to embed complexity into the core chart. But as in the next 20 years or so the infrastrcture will slowly migrate away from IPv4 I think this optio is reasonable to have in the chart as a core feature.
>
> How about other maintainers opinion? @Miretpl @potiuk @bugraoz93 @jedcunningham ?
I would also lean toward a single switch for now. Just agreeing on the current state of config parameters already took days, which says a lot :sweat_smile: For anything more granular, Kustomize is the right tool, as Jens mentioned.
IPv6 support should live in the core, as it's a network-layer capability we need to support rather than a customisation or third-party tooling concern. It is clearly on the horizon, and some cloud providers are already moving in that direction, even if it is not mainstream yet.
After writing this, I realised I'm mostly echoing Jens' points, but yes, I fully agree with all of them 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add ipFamilyPolicy and ipFamilies support to all Helm service templates [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #66616:
URL: https://github.com/apache/airflow/pull/66616#issuecomment-4621703729
> @jscheffl @bugraoz93 Agreed. How do you suggest the global switch should be structured? Should I move the two fields `ipFamilies` and `ipFamilyPolicy` to the top level? Should it be under a root `service` block? Or should it be a simple boolean like `enableIPv6`?
We also have other top level properties like `networkPolicies`, `enableServiceLinks` and `imagePullSecrets` so I think top level is OK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org