You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "TimurRakhmatullin86 (via GitHub)" <gi...@apache.org> on 2026/07/07 03:15:06 UTC

[PR] Fix KafkaBaseHook.test_connection missing oauth_cb for managed Kafka [airflow]

TimurRakhmatullin86 opened a new pull request, #69507:
URL: https://github.com/apache/airflow/pull/69507

   <!--
    Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements. ...
   -->
   
   ### Problem
   
   `KafkaBaseHook.test_connection()` builds its `AdminClient` straight from the raw
   `extra_dejson`, bypassing `get_conn`, which is where the `oauth_cb` token callback is
   injected for managed Kafka clusters. As a result, **"Test Connection" in the UI fails for a
   Google Cloud Managed Service for Apache Kafka connection** even though the hook itself
   authenticates and works — the test path never gets the token callback.
   
   ```python
   def test_connection(self) -> tuple[bool, str]:
       try:
           config = self.get_connection(self.kafka_config_id).extra_dejson  # no oauth_cb
           t = AdminClient(config).list_topics(timeout=10)
   ```
   
   ### Fix
   
   The config-building logic (including the `oauth_cb` injection) is extracted into a
   `_get_config()` helper that **both** `get_conn` and `test_connection` use, so the UI test
   authenticates exactly like the real client.
   
   - The behavior of `get_conn` is unchanged — the config-building code is moved verbatim and
     `get_conn` is now `self._get_client(self._get_config())`.
   - As a small side benefit, `test_connection` now surfaces the clear
     `config['bootstrap.servers'] must be provided.` error for an empty config instead of an
     opaque client failure.
   - This also covers the Amazon MSK IAM `oauth_cb` path proposed in #69427 once that merges,
     since it goes through the same `_get_config()`.
   
   ### Tests
   
   - Added `test_test_connection_injects_managed_kafka_oauth`, which points the connection at a
     managed-Kafka bootstrap host and asserts the `oauth_cb` is present on the config passed to
     `AdminClient` (this path had no test coverage before).
   - The three existing `test_test_connection*` tests used a `MagicMock` for `bootstrap.servers`;
     they now use a plain `"localhost:9092"` so the connection test stays on the non-managed path
     and doesn't spuriously exercise the managed-Kafka branch through `_get_config`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix KafkaBaseHook.test_connection missing oauth_cb for managed Kafka [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #69507:
URL: https://github.com/apache/airflow/pull/69507


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org