[GitHub] [airflow] josh-fell opened a new issue #18124: AzureDataExplorerHook custom field mapped incorrectly

[GitBox <gi...@apache.org>]

josh-fell opened a new issue #18124:
URL: https://github.com/apache/airflow/issues/18124


   ### Apache Airflow Provider(s)
   
   microsoft-azure
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-microsoft-azure==1!3.1.1
   
   ### Apache Airflow version
   
   2.1.3 (latest released)
   
   ### Operating System
   
   Debian GNU/Linux
   
   ### Deployment
   
   Astronomer
   
   ### Deployment details
   
   _No response_
   
   ### What happened
   
   When attempting to connect to Azure Data Explorer, the following exception is thrown even though the corresponding "Authentication Method" field is populated in the Azure Data Explorer connection form:
   ```
   airflow.exceptions.AirflowException: Extra connection option is missing required parameter: `auth_method`
   ```
   
   Airflow Connection:
   ![image](https://user-images.githubusercontent.com/48934154/132747672-25cb79b2-4589-4560-bb7e-4508de7d8659.png)
   
   
   ### What you expected to happen
   
   Airflow tasks should be able to connect to Azure Data Explorer when properly populating the custom connection form.  Or, at the very least, the above exception should not be thrown when an Authentication Method is provided in the connection.
   
   ### How to reproduce
   
   1. Install the Microsoft Azure provider and create an Airflow Connection with the type `Azure Data Explorer`.
   2. Provide all values for "Auth Username", "Auth Password", "Tenant ID", and "Authentication Method".
   3. Finally execute a task which uses the `AzureDataExplorerHook`
   
   ### Anything else
   
   Looks like there are a few issues in the `AzureDataExplorerHook`:
   
   - The `get_required_param()` method is being passed a value of "auth_method" from the `Extras` field in the connection form.  The `Extras` field is no longer exposed in the connection form and would never be able to be provided.
   
   ```python
   def get_required_param(name: str) -> str:
               """Extract required parameter from extra JSON, raise exception if not found"""
               value = conn.extra_dejson.get(name)
               if not value:
                   raise AirflowException(f'Extra connection option is missing required parameter: `{name}`')
               return value
   
           auth_method = get_required_param('auth_method') or get_required_param(
               'extra__azure_data_explorer__auth_method'
           )
   ```
   - The custom fields mapping for "Tenant ID" and "Authentication Method" are switched so even if these values are provided in the connection form they will not be used properly in the hook.
   ```python
   @staticmethod
       def get_connection_form_widgets() -> Dict[str, Any]:
           """Returns connection widgets to add to connection form"""
           from flask_appbuilder.fieldwidgets import BS3PasswordFieldWidget, BS3TextFieldWidget
           from flask_babel import lazy_gettext
           from wtforms import PasswordField, StringField
   
           return {
               "extra__azure_data_explorer__auth_method": StringField(
                   lazy_gettext('Tenant ID'), widget=BS3TextFieldWidget()
               ),
               "extra__azure_data_explorer__tenant": StringField(
                   lazy_gettext('Authentication Method'), widget=BS3TextFieldWidget()
               ),
   ...
   ```
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #18124: Cannot retrieve Authentication Method in AzureDataExplorerHook using the custom connection fields

[GitBox <gi...@apache.org>]

potiuk closed issue #18124:
URL: https://github.com/apache/airflow/issues/18124


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org