[GitHub] [airflow] Taragolis commented on pull request #23560: Add advanced secrets backend configurations

[GitBox <gi...@apache.org>]

Taragolis commented on PR #23560:
URL: https://github.com/apache/airflow/pull/23560#issuecomment-1174010308

   @potiuk @dstandish tl;dr: After tried to change something in secrets backends I found that might be better to try make this mechanism more transparent first at least for developers and after that we could go back to backend order/configurations.
   I don't think that is possible completely solve chicken-egg unfortunately.
   
   ---
   
   In each case if we change order or add ability to configure multiple backends we got some false positive warnings.
   
   **Variables expected that Metastore always last when created Variable in UI**
   
   https://github.com/apache/airflow/blob/e2fd41f7b14adef2c3a88dde14d088b5ef93b460/airflow/models/variable.py#L226-L237
   
   ---
   
   Some additional findings
   
   **Connections and Variables have methods for lookup values which might be better to move in some SecretsBackendsClass**
   
   https://github.com/apache/airflow/blob/e2fd41f7b14adef2c3a88dde14d088b5ef93b460/airflow/models/variable.py#L257-L275
   
   https://github.com/apache/airflow/blob/e2fd41f7b14adef2c3a88dde14d088b5ef93b460/airflow/models/connection.py#L410-L430
   
   **List variables/connections use different approach rather than get**
   
   Get variables/connections stored in metastore - use `airflow.secrets.metastore.MetastoreBackend`
   List variables/connections outside of `airflow.secrets.metastore.MetastoreBackend`
   
   I think most of (probably all) secrets backends which implemented in airflow or in community providers could implemented List operation. Which make possible to show this variables/connections in UI/CLI
   
   **No information for end-user where received variable/connection actually stored**
   
   Personally spend additional time last Friday just because I assume that we do not create specific connection in secrets backend.
   
   **Implicit ignore for end-users any json.Decoding errors in backend_kwargs**
   
   `¯\_(ツ)_/¯`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org