[GitHub] [airflow] kaxil edited a comment on issue #16952: Secrets Backend Search Path Ordering/Priority

[GitBox <gi...@apache.org>]

kaxil edited a comment on issue #16952:
URL: https://github.com/apache/airflow/issues/16952#issuecomment-880281931


   I think the priority is correct and will cause confusion if changed later on. And with customer Secrets Backend, you can mix and match however you like. We intentionally did this (same fox XCom Backend) so that companies can create one for their own needs as ONE SIZE DOES NOT FEEL ALL.
   
   That being said, something I had planned earlier, was to allow DAG Authors to pick a single backend to choose the variable or connections from  ( **Not getting configurations from Secrets Backend though** ).
   
   For Example the following will only check Environment Variables to get Airflow Variables.: 
   
   ```python
   Variable.get(
       key="example_key",
       secrets_backend="airflow.secrets.environment_variables.EnvironmentVariablesBackend"
   )
   
   ```
   
   Only changes required:
   
   ```diff
   diff --git a/airflow/models/variable.py b/airflow/models/variable.py
   index 7d4726966..05b266bd3 100644
   --- a/airflow/models/variable.py
   +++ b/airflow/models/variable.py
   @@ -124,6 +124,7 @@ class Variable(Base, LoggingMixin):
            key: str,
            default_var: Any = __NO_DEFAULT_SENTINEL,
            deserialize_json: bool = False,
   +        secrets_backend: Optional[str] = None,
        ) -> Any:
            """
            Gets a value for an Airflow Variable Key
   @@ -132,7 +133,7 @@ class Variable(Base, LoggingMixin):
            :param default_var: Default value of the Variable if the Variable doesn't exists
            :param deserialize_json: Deserialize the value to a Python dict
            """
   -        var_val = Variable.get_variable_from_secrets(key=key)
   +        var_val = Variable.get_variable_from_secrets(key, secrets_backend)
            if var_val is None:
                if default_var is not cls.__NO_DEFAULT_SENTINEL:
                    return default_var
   @@ -193,14 +194,35 @@ class Variable(Base, LoggingMixin):
                self._val = fernet.rotate(self._val.encode('utf-8')).decode()
   
        @staticmethod
   -    def get_variable_from_secrets(key: str) -> Optional[str]:
   +    def get_variable_from_secrets(
   +        key: str,
   +        secrets_backend: Optional[str] = None,
   +    ) -> Optional[str]:
            """
            Get Airflow Variable by iterating over all Secret Backends.
   
            :param key: Variable Key
            :return: Variable Value
            """
   -        for secrets_backend in ensure_secrets_loaded():
   +        secrets_backends = ensure_secrets_loaded()
   +        secrets_backends_classes = {
   +            f"{backend.__class__.__module__}.{backend.__class__.__name__}": backend
   +            for backend in secrets_backends
   +        }
   +
   +        if secrets_backend not in secrets_backends_classes:
   +            raise KeyError(
   +                f"Invalid secrets backend - '{secrets_backend}'. "
   +                f"Should be one of {', '.join(secrets_backends_classes.keys())}"
   +            )
   +
   +        if secrets_backend:
   +            var_val = secrets_backends_classes[secrets_backends].get_variable(key=key)
   +            if var_val is not None:
   +                return var_val
   +            return None
   +
   ```
   
   cc @fhoda 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org