You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "riccardoforzan (via GitHub)" <gi...@apache.org> on 2024/06/12 17:38:14 UTC

[I] Connection test problem with apache-airflow-providers-apache-kafka [airflow]

riccardoforzan opened a new issue, #40204:
URL: https://github.com/apache/airflow/issues/40204

   ### Apache Airflow Provider(s)
   
   apache-kafka
   
   ### Versions of Apache Airflow Providers
   
   1.4.1
   
   ### Apache Airflow version
   
   2.9.2
   
   ### Operating System
   
   Debian 11
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   _No response_
   
   ### What happened
   
   I am encountering an issue with Apache Airflow 2.9.2 when attempting to test a connection to Kafka. 
   I followed the integration guide as per the documentation [here](https://airflow.apache.org/docs/apache-airflow-providers-apache-kafka/stable/index.html). 
   
   After setting up the connection to Kafka, testing the connection with the command `airflow connections test <connection-name>` results in a failure.
   
   It appears that the test utilizes KafkaBaseHook, and it fails when reaching this specific statement: [base.py#L56](https://github.com/apache/airflow/blob/main/airflow/providers/apache/kafka/hooks/base.py#L56)
   
   Here is the configuration dictionary I am using
   
   ```json
   {
       "bootstrap.servers": "server",
       "security.protocol": "SASL_SSL",
       "sasl.mechanisms": "PLAIN",
       "sasl.username": "username",
       "sasl.password": "password"
   }
   ```
   
   ### What you think should happen instead
   
   I think it should use an instance of KafkaAdminClientHook instead of an instance of the generic class, which throws when invoking _get_client
   
   ### How to reproduce
   
   - Install the provider(https://airflow.apache.org/docs/apache-airflow-providers-apache-kafka/stable/index.html)
   - Declare a connection in the AirFlow UI using the Kafka Provider
   - In the terminal of the docker container issue the command `airflow connections test <connection-name>`
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Connection test problem with apache-airflow-providers-apache-kafka [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #40204:
URL: https://github.com/apache/airflow/issues/40204#issuecomment-2164425546

   Yes. There is a base Kafka connection and when "Test connection" is used, it instantiates Hook based on the type of connection - so it will instantiate the Base Kafka Hook and there _get_client is `Not Implemented` - easy fix will be to turn the `NotImplemented` into a client that can do a basic connection test.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Connection test problem with apache-airflow-providers-apache-kafka [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #40204: Connection test problem with apache-airflow-providers-apache-kafka
URL: https://github.com/apache/airflow/issues/40204


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Connection test problem with apache-airflow-providers-apache-kafka [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #40204:
URL: https://github.com/apache/airflow/issues/40204#issuecomment-2163580741

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org