[GitHub] [airflow] wolfier opened a new issue, #23140: Improve connection fetching from AWS Secrets Manager

[GitBox <gi...@apache.org>]

wolfier opened a new issue, #23140:
URL: https://github.com/apache/airflow/issues/23140

   ### Description
   
   Users need to make the values are URL safe in the when storing them in AWS Secrets Managers to experience the expected behaviour when fetching a connection and/or using a hook.
   
   ### Use case/motivation
   
   When a user adds the connection from the Airflow UI, the attributes of the connection, the individual fields, are stored in the metadata database as is. They are not made URL safe. This is why a user would expect saving the values as is would suffice when storing values in Secrets Manager. The difference here is how Airflow fetches the value from the [MetaStoreBackend](https://github.com/apache/airflow/blob/2.2.5/airflow/secrets/metastore.py) versus the [SecretsManagerBackend](https://github.com/apache/airflow/blob/2.2.5/airflow/providers/amazon/aws/secrets/secrets_manager.py).
   
   In the case of MetaStoreBackend, when Airflow pulls the row in the connection table from the metadata database, the data is already a Connection object as defined by the ORM.
   
   1. Query database for connection data ([source](https://github.com/apache/airflow/blob/2.2.5/airflow/secrets/metastore.py#L33-L38))
   2. Connection data is returned as an ORM object
   
   However, with SecretsManagerBackend, Airflow first converts it to an URI string then create a connection object from the string which expects the string to be url safe.
   
   1. Grab the plaintext value and convert it to a dictionary ([source](https://github.com/apache/airflow/blob/2.2.5/airflow/providers/amazon/aws/secrets/secrets_manager.py#L192-L204))
   2. Format a URI template with the dictionary ([source](https://github.com/apache/airflow/blob/2.2.5/airflow/secrets/base_secrets.py#L64))
   3. Create a connection object from the URI string ([source](https://github.com/apache/airflow/blob/2.2.5/airflow/secrets/base_secrets.py#L67))
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org