[GitHub] [airflow] potiuk commented on issue #14592: Unreachable Secrets Backend Causes Web Server Crash

[GitBox <gi...@apache.org>]

potiuk commented on issue #14592:
URL: https://github.com/apache/airflow/issues/14592#issuecomment-852516068


   If I understand correctly - this is only when you cannot reach secret backend when it is configured ? Not about just a value missing in the backend?
   
   If my assumption is correct, then I have no doubts whatsoever that  his is desired, nice and "good" behavior for airflow to crash hard in this case. 
   
   It's far worse to continue running, when you rely on the backend with variables to be available and they cannot be read where you expec them (and the assumption is that when you configure the backend, you want it to be reachable). 
   
   This might lead to a number of problems - bad processing of data, wrong databases updated, wrong calculations, you name it. Totally unpredictable, because we have no idea what the retrieved value will be used for, and we have no idea whether a "fallback default' is good or not.
   
   It's NEVER a good idea to continue when you have unpredictable and unexpected configuration. Airflow as a system is not able to assess if the "default" value is "OK" to continue if the "expected" backend does not provide the right values (either because it is missing or wrongly configured). Actually it's even worse - Airflow is not even able to determine which kind of problem it is - is it wrongly configured? or just temporarily missing-in-action ?  It will look exactly the same from Airlfow's point of view and this means that by not-crashing we accept that sometimes Airflow will run with configuration A and sometimes (when the secret backend is not reachable) with configuration B. This is no-go.
   
   You either expect the secret backend to be available or not  - there is no middle ground.
   
   Crashing in this case is the only reasonable approach. 
   
   And I really like the output @lidalei  copied. It is very straightworward and easy to find what the problem is. Again - we have no  idea whether we can continue with default value or not. The only sensible approach is to crash hard.
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org