[GitHub] [airflow] josh-fell opened a new issue #18442: Cannot retrieve Account URL in AzureBatchHook using the custom connection fields

[GitBox <gi...@apache.org>]

josh-fell opened a new issue #18442:
URL: https://github.com/apache/airflow/issues/18442


   ### Apache Airflow Provider(s)
   
   microsoft-azure
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-microsoft-azure==2.0.0
   
   ### Apache Airflow version
   
   2.1.0
   
   ### Operating System
   
   Debian GNU/Linux
   
   ### Deployment
   
   Astronomer
   
   ### Deployment details
   
   _No response_
   
   ### What happened
   
   Related to #18124.
   
   When attempting to connect to Azure Batch, the following exception is thrown even though the corresponding "Azure Batch Account URl" field is populated in the Azure Batch connection form:
   ```
   airflow.exceptions.AirflowException: Extra connection option is missing required parameter: `account_url`
   ```
   Airflow Connection:
   ![image](https://user-images.githubusercontent.com/48934154/134413597-260987e9-3035-4ced-a329-3af1129e4ae9.png)
   
   
   ### What you expected to happen
   
   Airflow tasks should be able to connect to Azure Batch when properly populating the custom connection form. Or, at the very least, the above exception should not be thrown when an Azure Batch Account URL is provided in the connection.
   
   ### How to reproduce
   
   1. Install the Microsoft Azure provider and create an Airflow Connection with the type Azure Batch
   2. Provide values for at least "Azure Batch Account URl"
   3. Finally execute a task which uses the AzureBatchHook
   
   ### Anything else
   
   The get_required_param() method is being passed a value of "auth_method" from the `Extra` field in the connection form. The `Extra` field is no longer exposed in the connection form and would never be able to be provided.
   ```python
   def _get_required_param(name):
           """Extract required parameter from extra JSON, raise exception if not found"""
           value = conn.extra_dejson.get(name)
           if not value:
              raise AirflowException(f'Extra connection option is missing required parameter: `{name}`')
           return value
   ...
   
   batch_account_url = _get_required_param('account_url') or _get_required_param(
       'extra__azure_batch__account_url'
   )
   ```
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal closed issue #18442: Cannot retrieve Account URL in AzureBatchHook using the custom connection fields

[GitBox <gi...@apache.org>]

eladkal closed issue #18442:
URL: https://github.com/apache/airflow/issues/18442


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org