[GitHub] [airflow] lidalei commented on issue #14592: Unreachable Secrets Backend Causes Web Server Crash

[GitBox <gi...@apache.org>]

lidalei commented on issue #14592:
URL: https://github.com/apache/airflow/issues/14592#issuecomment-852464627


   It is likely a desired (though not nice) behavior in Airflow 2.0.2. We observed a similar behavior when using GCP Secret Manger and the service account key file couldn't be found. Any Airflow command would fail because of it.
   ```
   Traceback (most recent call last): 
   File "/home/airflow/.local/bin/airflow", line 5, in <module> from airflow.__main__ import main 
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__init__.py", line 34, in <module> from airflow import settings 
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/settings.py", line 37, in <module> from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401 
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py", line 1099, in <module> secrets_backend_list = initialize_secrets_backends() 
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py", line 1020, in initialize_secrets_backends custom_secret_backend = get_custom_secret_backend() 
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py", line 1009, in get_custom_secret_backend return secrets_backend_cls(**alternative_secrets_config_dict) 
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/secrets/secret_manager.py", line 101, in __init__ self.credentials, self.project_id = get_credentials_and_project_id( 
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/utils/credentials_provider.py", line 309, in get_credentials_and_project_id return _CredentialProvider(*args, **kwargs).get_credentials_and_project() 
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/utils/credentials_provider.py", line 242, in get_credentials_and_project credentials, project_id = self._get_credentials_using_adc() 
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/utils/credentials_provider.py", line 295, in _get_credentials_using_adc credentials, project_id = google.auth.default(scopes=self.scopes) 
   File "/home/airflow/.local/lib/python3.8/site-packages/google/auth/_default.py", line 346, in default credentials, project_id = checker() 
   File "/home/airflow/.local/lib/python3.8/site-packages/google/auth/_default.py", line 189, in _get_explicit_environ_credentials credentials, project_id = load_credentials_from_file( 
   File "/home/airflow/.local/lib/python3.8/site-packages/google/auth/_default.py", line 100, in load_credentials_from_file raise exceptions.DefaultCredentialsError( google.auth.exceptions.DefaultCredentialsError: 
   File /secrets/xxx.json was not found.
   ```
   
   How about catching the exception and logging the error when a secrets backend cannot be initialized, instead of exiting?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org