[GitHub] [airflow] Frietziek edited a comment on issue #13916: ECONNRESET error in scheduler using KubernetesExecutor on AKS

[GitBox <gi...@apache.org>]

Frietziek edited a comment on issue #13916:
URL: https://github.com/apache/airflow/issues/13916#issuecomment-830191383


   Guys, I work with @alete89. Another solution for this, and specially if you are in older Airflow versions that still don't have the AIRFLOW__KUBERNETES__ENABLE_TCP_KEEPALIVE configuration key, is to execute at some moment at the start of airflow this in a python script:
   
   ```Python
   from urllib3.connection import HTTPConnection
   import socket
   
   
   HTTPConnection.default_socket_options = HTTPConnection.default_socket_options + [
       (socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1),
       (socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, 20),
       (socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, 5),
       (socket.IPPROTO_TCP, socket.TCP_KEEPCNT, 10)
   ]
   ```
   
   This worked for us apparently, and basically set on urllib3 (which is the library that airflow uses for connectivity under the hood) the same parametry as was mentioned in this issue and in other places on the internet.
   
   In our case, aparently, there were some tcp hangup that provoke the consumption of all available executor capacity of parallelism.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org