[GitHub] [airflow] maxirus commented on issue #6643: [AIRFLOW-6040] Fix KubernetesJobWatcher Read time out error

[GitBox <gi...@apache.org>]

maxirus commented on issue #6643: [AIRFLOW-6040] Fix KubernetesJobWatcher Read time out error
URL: https://github.com/apache/airflow/pull/6643#issuecomment-558903525
 
 
   The `kube_client_request_args` config parameter is "global" to all of the client requests. Setting `timeout_seconds` here causes other methods, such as `create_namespaced_pod` to fail as this property is not recognized. If the ask here is to make this configurable, we'd need do so by adding another parameter to the `airflow.cfg` as to not break current functionality.
   
   My approach assumes that `_request_timeout` being set to 60 as the default in the config was deliberate so I wanted to be under this value by a considerable margin. That said, querying the API should realistically never take more than about 1s.
   
   If we didn't want to "hard-code" this value here, the other approaches I see are:
   
   1. Create a new config parameter, `watch_timeout_seconds`
   2. `if "_request_timeout" in kube_client_request_args then timeout_seconds = kube_client_request_args['_request_timeout'] - 1`
   3. Check if there are any events for the label/worker uuid first, before the watch. If so, then watch
   4. Leave it hard-coded
   5. Other ideas?
   
   P.S.: The Worker UUID seems to not persist and is created at runtime. If I follow correctly, this get generated each time the scheduler runs. How is this tracked across restarts?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services