[GitHub] [airflow] kaxil commented on pull request #15795: [WIP] Ability to test connections from UI or API

[GitBox <gi...@apache.org>]

kaxil commented on pull request #15795:
URL: https://github.com/apache/airflow/pull/15795#issuecomment-839716702


   > Nice! One small (potential) issue - which might be worth thinking about here. I think often the Webserver is limited in connectivity. I know for a fact that MWAA and Composer have a much more restricted "environment" to run webserver on due to sensitivity of the webserver (it is exposed to outside world). I am not sure if they are limiting the outgoing connections from webserver, but I think they might @subashcanapathy ? or at least might want to limit it in the future.
   > 
   > Also there are are number of connections that might require the worker "environment" to work on (for example GOOGLE_APPLICATION_CREDENTIALS for gcp, or worker-identity configured for pod they are running on, or Kerberos configured for workers and only for workers (I've worked for a customer that had an environment where only the workers had the credentials that allowed them to make outgoing connections and authenticate using Kerberos).
   > 
   > This is not a blocker for that change, but just something to remember about - that it fhe "connection" test does not work via API/Webserver (the test will be executed in the webserver instance), it does not mean that the connection is not working from workers. It might be worth-while to add a "last successful connection from worker" - so basically store when last time the connection succeeded for actual worker running. This would give more complete information about the status of the connection.
   
   I consider this more of an "I have added a new connection now, I want to test if it is working or not" as a user can validate before they put or use the connection in their DAG. So not "when was this connection last successfully" as the connection to "test" like running a "select 1" query. We can extend this feature once this PR is merged to more connections and to "CLI".
   
   Actually "testing" will help us propagate the error message to UI, API (later CLI) unlike clubbing it to an exiting usage of connection is some Hook or Operator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org