You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "allenc97 (via GitHub)" <gi...@apache.org> on 2026/07/07 04:06:10 UTC

[PR] Databricks OIDC token federation for AWS IAM [airflow]

allenc97 opened a new pull request, #69509:
URL: https://github.com/apache/airflow/pull/69509

   <!-- SPDX-License-Identifier: Apache-2.0
         https://www.apache.org/licenses/LICENSE-2.0 -->
   
   Extends the Databricks provider hook with OIDC token federation for Airflow running with AWS credentials, parallel to the existing Kubernetes path (#61458). The hook mints an AWS-signed OIDC JWT via AWS STS `GetWebIdentityToken` and exchanges it for a Databricks service-principal token using the same RFC 8693 exchange as `federated_k8s` — only the JWT source differs. Like the Kubernetes path it stores no secrets in the connection; identity comes from the caller's assumed IAM role. Set `login=federated_aws` and provide `client_id` and `aws_conn_id` in the connection extra. See the Databricks [AWS IAM workload identity federation guide](https://docs.databricks.com/aws/en/dev-tools/auth/provider-aws-iam).
   
   * related: #54291
   
   ### Integration testing
   
   Verified end-to-end against a real workspace with this connection (no secrets stored) and DAG:
   
   ```json
   {
     "conn_type": "databricks",
     "host": "<workspace>.cloud.databricks.com",
     "login": "federated_aws",
     "extra": {
       "client_id": "<databricks-sp-client-id>",
       "aws_conn_id": "<aws_conn_id>"
     }
   }
   ```
   
   ```python
   from __future__ import annotations
   
   from airflow.providers.databricks.hooks.databricks import DatabricksHook
   from airflow.sdk import dag
   from airflow.sdk import task
   
   
   @dag(
       dag_id="test_databricks_federated_aws",
       schedule=None,
       catchup=False,
       description="Integration test: authenticate to Databricks via AWS IAM federation and confirm the SP identity.",
   )
   def test_databricks_federated_aws():
       @task
       def test_connection():
           hook = DatabricksHook(databricks_conn_id="databricks_federated_aws")
           print(hook.test_connection())
           print(hook._do_api_call(("GET", "/2.0/preview/scim/v2/Me")))
   
       test_connection()
   
   
   test_databricks_federated_aws()
   ```
   
   Requires outbound web identity federation enabled on the AWS account, the role granted `sts:GetWebIdentityToken`, and a Databricks service-principal federation policy whose `subject` is the role ARN and `audiences` is `["databricks"]`.
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (please specify the tool below)
   
   Generated-by: Claude Code (Opus 4.8) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   
   ---
   
   * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
   * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "allenc97 (via GitHub)" <gi...@apache.org>.
allenc97 commented on PR #69509:
URL: https://github.com/apache/airflow/pull/69509#issuecomment-4919554119

   I refactored this PR against https://github.com/apache/airflow/pull/69272 - given that the connection already supports other cloud based identity schemes I extended it to support AWS STS as well. However, if it's preferred based on the above PR's supplied OIDC callable changes that other Databricks supported OIDC sources remain user custom I am also happy to close this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "allenc97 (via GitHub)" <gi...@apache.org>.
allenc97 commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3575376934


##########
providers/databricks/src/airflow/providers/databricks/hooks/databricks_base.py:
##########
@@ -1167,6 +1233,9 @@ def _get_token(self, raise_error: bool = False) -> str | None:
         if self.databricks_conn.extra_dejson.get("federated_token_provider"):
             self.log.debug("Using OIDC token federation with a supplied token provider.")
             return self._get_federated_databricks_token(self._get_oidc_token_service_url())
+        if self._is_aws_federation():

Review Comment:
   Good catch - added a regression test to assert the precedence behavior when both are provided.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "moomindani (via GitHub)" <gi...@apache.org>.
moomindani commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3567557364


##########
providers/databricks/src/airflow/providers/databricks/hooks/databricks_base.py:
##########
@@ -917,13 +926,17 @@ def _get_federation_subject_token(self) -> tuple[str, str | None]:
         """
         Resolve the OIDC JWT to exchange for a Databricks token (RFC 8693 ``subject_token``).
 
-        Two subject-token sources are supported:
+        Three subject-token sources are supported:
 
         * ``federated_token_provider`` -- a dotted path to a ``Callable[[], str]`` that returns
           the JWT. The token is obtained in-process and never written to disk, so a control
           plane can vend a short-lived, per-workload identity token for the exchange. ``client_id``
           is optional here: supply it in the extra for a service principal federation policy, or
           omit it for an account-wide federation policy.
+        * AWS IAM (``federated_aws``) -- mint an AWS-signed OIDC JWT via AWS STS

Review Comment:
   Non-blocking: the sibling `_get_federated_databricks_token` docstring (below, outside this diff) still describes only two subject-token sources — "supplied either by a `federated_token_provider` callable or by the pod's Kubernetes service account". Worth updating it to mention the AWS path in the same way you updated this one.
   
   ---
   Drafted-by: Claude Code (Fable 5); reviewed by @moomindani before posting



##########
providers/databricks/src/airflow/providers/databricks/hooks/databricks_base.py:
##########
@@ -936,6 +949,8 @@ def _get_federation_subject_token(self) -> tuple[str, str | None]:
             return self._resolve_supplied_subject_token(provider), self.databricks_conn.extra_dejson.get(
                 "client_id"
             )
+        if self._is_aws_federation():

Review Comment:
   Non-blocking: if a connection sets both `login=federated_k8s` and `federated_aws: true` in extra, `_get_token` takes the Kubernetes branch and logs "Using Kubernetes OIDC token federation", but this method resolves the AWS token first — so the log and the actual subject-token source disagree. A one-line precedence note in the docstring (or an explicit guard rejecting the combination) would avoid confusion in misconfigured setups.
   
   ---
   Drafted-by: Claude Code (Fable 5); reviewed by @moomindani before posting



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "allenc97 (via GitHub)" <gi...@apache.org>.
allenc97 commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3572777923


##########
providers/databricks/src/airflow/providers/databricks/hooks/databricks_base.py:
##########
@@ -917,13 +926,17 @@ def _get_federation_subject_token(self) -> tuple[str, str | None]:
         """
         Resolve the OIDC JWT to exchange for a Databricks token (RFC 8693 ``subject_token``).
 
-        Two subject-token sources are supported:
+        Three subject-token sources are supported:
 
         * ``federated_token_provider`` -- a dotted path to a ``Callable[[], str]`` that returns
           the JWT. The token is obtained in-process and never written to disk, so a control
           plane can vend a short-lived, per-workload identity token for the exchange. ``client_id``
           is optional here: supply it in the extra for a service principal federation policy, or
           omit it for an account-wide federation policy.
+        * AWS IAM (``federated_aws``) -- mint an AWS-signed OIDC JWT via AWS STS

Review Comment:
   Updated the docstring for `_get_federated_databricks_token` to also include AWS STS.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "moomindani (via GitHub)" <gi...@apache.org>.
moomindani commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3575243998


##########
providers/databricks/src/airflow/providers/databricks/hooks/databricks_base.py:
##########
@@ -1167,6 +1233,9 @@ def _get_token(self, raise_error: bool = False) -> str | None:
         if self.databricks_conn.extra_dejson.get("federated_token_provider"):
             self.log.debug("Using OIDC token federation with a supplied token provider.")
             return self._get_federated_databricks_token(self._get_oidc_token_service_url())
+        if self._is_aws_federation():

Review Comment:
   Non-blocking: this fixup correctly reorders the dispatch so AWS wins over Kubernetes when both `federated_k8s` and `federated_aws` are configured — but there's no regression test for it. Since this is exactly the precedence bug from my earlier comment, a test asserting AWS takes precedence when both extras are set (e.g. `login="federated_k8s"` + `extra={"federated_aws": True}` → subject token comes from STS, not the pod) would guard against this silently regressing in a future refactor.
   
   ---
   Drafted-by: Claude Code (Sonnet 5); reviewed by @moomindani before posting



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "allenc97 (via GitHub)" <gi...@apache.org>.
allenc97 commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3591765950


##########
dev/breeze/tests/test_selective_checks.py:
##########
@@ -1083,7 +1083,7 @@ def assert_outputs_are_printed(expected_outputs: dict[str, str], stderr: str):
             ("providers/amazon/src/airflow/providers/amazon/provider.yaml",),
             {
                 "selected-providers-list-as-string": "amazon apache.hive cncf.kubernetes "
-                "common.compat common.messaging common.sql exasol ftp google http imap microsoft.azure "
+                "common.compat common.messaging common.sql databricks exasol ftp google http imap microsoft.azure "

Review Comment:
   @eladkal Just to make sure, are we ok to resolve + merge this? Sorry for the questions, this is my first time contributing and would just like to make sure I don't close this if the above doesn't address the concerns.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3597489754


##########
dev/breeze/tests/test_selective_checks.py:
##########
@@ -1083,7 +1083,7 @@ def assert_outputs_are_printed(expected_outputs: dict[str, str], stderr: str):
             ("providers/amazon/src/airflow/providers/amazon/provider.yaml",),
             {
                 "selected-providers-list-as-string": "amazon apache.hive cncf.kubernetes "
-                "common.compat common.messaging common.sql exasol ftp google http imap microsoft.azure "
+                "common.compat common.messaging common.sql databricks exasol ftp google http imap microsoft.azure "

Review Comment:
   This doesn't feel right also the description hint otherwise - there are no changes to Amazon provider.yaml



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "earnin-allenc97 (via GitHub)" <gi...@apache.org>.
earnin-allenc97 commented on PR #69509:
URL: https://github.com/apache/airflow/pull/69509#issuecomment-4919535501

   I refactored this PR against https://github.com/apache/airflow/pull/69272 - given that the connection already supports other cloud based identity schemes I extended it to support AWS STS as well. However, if it's preferred based on the above PR's supplied OIDC callable changes that other Databricks supported OIDC sources remain user custom I am also happy to close this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "allenc97 (via GitHub)" <gi...@apache.org>.
allenc97 commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3572773485


##########
providers/databricks/src/airflow/providers/databricks/hooks/databricks_base.py:
##########
@@ -936,6 +949,8 @@ def _get_federation_subject_token(self) -> tuple[str, str | None]:
             return self._resolve_supplied_subject_token(provider), self.databricks_conn.extra_dejson.get(
                 "client_id"
             )
+        if self._is_aws_federation():

Review Comment:
   Addressed this by updating order in code and also left a comment on the precedence order.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3597494704


##########
dev/breeze/tests/test_selective_checks.py:
##########
@@ -1083,7 +1083,7 @@ def assert_outputs_are_printed(expected_outputs: dict[str, str], stderr: str):
             ("providers/amazon/src/airflow/providers/amazon/provider.yaml",),
             {
                 "selected-providers-list-as-string": "amazon apache.hive cncf.kubernetes "
-                "common.compat common.messaging common.sql exasol ftp google http imap microsoft.azure "
+                "common.compat common.messaging common.sql databricks exasol ftp google http imap microsoft.azure "

Review Comment:
   but we can investigate later as this shouldn't block the merge. I'd appreciate if you can look into this as a followup just to make sure if it is right or not



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #69509:
URL: https://github.com/apache/airflow/pull/69509#issuecomment-4994671812

   Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal merged PR #69509:
URL: https://github.com/apache/airflow/pull/69509


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3583776493


##########
dev/breeze/tests/test_selective_checks.py:
##########
@@ -1083,7 +1083,7 @@ def assert_outputs_are_printed(expected_outputs: dict[str, str], stderr: str):
             ("providers/amazon/src/airflow/providers/amazon/provider.yaml",),
             {
                 "selected-providers-list-as-string": "amazon apache.hive cncf.kubernetes "
-                "common.compat common.messaging common.sql exasol ftp google http imap microsoft.azure "
+                "common.compat common.messaging common.sql databricks exasol ftp google http imap microsoft.azure "

Review Comment:
   Why databricks was added here? Not sure it is right



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #69509:
URL: https://github.com/apache/airflow/pull/69509#issuecomment-4900005196

   Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our [Contributors' Guide](https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (ruff, mypy and type annotations). Our [prek-hooks]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-prek-hooks) will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/airflow-core/docs/howto/custom-operator.rst) Consider adding an example Dag that shows how users should use it.
   - Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
   - Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
   - Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices).
   - Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
   Apache Airflow is a community-driven project and together we are making it better 🚀.
   In case of doubts contact the developers at:
   Mailing List: dev@airflow.apache.org
   Slack: https://s.apache.org/airflow-slack
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "moomindani (via GitHub)" <gi...@apache.org>.
moomindani commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3583643510


##########
providers/databricks/src/airflow/providers/databricks/hooks/databricks_base.py:
##########
@@ -1167,6 +1233,9 @@ def _get_token(self, raise_error: bool = False) -> str | None:
         if self.databricks_conn.extra_dejson.get("federated_token_provider"):
             self.log.debug("Using OIDC token federation with a supplied token provider.")
             return self._get_federated_databricks_token(self._get_oidc_token_service_url())
+        if self._is_aws_federation():

Review Comment:
   Confirmed — the new test correctly covers the precedence case (mocks `_get_k8s_jwt_token`, asserts it's never called, and verifies the subject token comes from AWS). Thanks for the quick turnaround!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "allenc97 (via GitHub)" <gi...@apache.org>.
allenc97 commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3583849819


##########
dev/breeze/tests/test_selective_checks.py:
##########
@@ -1083,7 +1083,7 @@ def assert_outputs_are_printed(expected_outputs: dict[str, str], stderr: str):
             ("providers/amazon/src/airflow/providers/amazon/provider.yaml",),
             {
                 "selected-providers-list-as-string": "amazon apache.hive cncf.kubernetes "
-                "common.compat common.messaging common.sql exasol ftp google http imap microsoft.azure "
+                "common.compat common.messaging common.sql databricks exasol ftp google http imap microsoft.azure "

Review Comment:
   This change addresses the failure in CI run https://github.com/apache/airflow/actions/runs/28840730236/job/85911957562 - since the current change introduces [amazon] as an extra the tests require that dependents are also verified.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #69509:
URL: https://github.com/apache/airflow/pull/69509#issuecomment-4950458001

   > I refactored this PR against #69272 - given that the connection already supports other cloud based identity schemes I extended it to support AWS STS as well. However, if it's preferred based on the above PR's supplied OIDC callable changes that other Databricks supported OIDC sources remain user custom I am also happy to close this PR.
   
   That is a question for Databricks experts cc @moomindani 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Databricks OIDC token federation for AWS IAM [airflow]

Posted by "allenc97 (via GitHub)" <gi...@apache.org>.
allenc97 commented on code in PR #69509:
URL: https://github.com/apache/airflow/pull/69509#discussion_r3597661373


##########
dev/breeze/tests/test_selective_checks.py:
##########
@@ -1083,7 +1083,7 @@ def assert_outputs_are_printed(expected_outputs: dict[str, str], stderr: str):
             ("providers/amazon/src/airflow/providers/amazon/provider.yaml",),
             {
                 "selected-providers-list-as-string": "amazon apache.hive cncf.kubernetes "
-                "common.compat common.messaging common.sql exasol ftp google http imap microsoft.azure "
+                "common.compat common.messaging common.sql databricks exasol ftp google http imap microsoft.azure "

Review Comment:
   I took a closer look at this - it seems as of https://github.com/apache/airflow/issues/44511 the intention is to no longer rely on provider.yaml for dependency management, e.g quoting `This means that dependencies should be moved from providrer.yaml files to pyproject.toml`. 
   
   This is perhaps also confusing because docs in various places are incorrect, e.g https://github.com/apache/airflow/blob/main/generated/README.md still says to modify provider.yaml, but no such section exists in say, https://github.com/apache/airflow/blob/providers-databricks/7.8.3/providers/amazon/provider.yaml.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org