[GitHub] [airflow] potiuk commented on issue #7947: Fixes too high parallelism in CI

[GitBox <gi...@apache.org>]

potiuk commented on issue #7947: Fixes too high parallelism in CI
URL: https://github.com/apache/airflow/pull/7947#issuecomment-605416262
 
 
   Hello @ashb @Fokko @BasPH @kaxil @turbaszek @mik-laj @feluelle - I think we need to merge i quickly.  I believe I found and fixed the problem that caused the last two days constant failures of our CI jobs. 
   
   It is very likely due to less resources (memory) available for the CI workers on Travis and I believe it was caused by too high default parallelism. In the tests we run we had default parallelism set to 32.  This means that for local executor, 32 worker processes started. I think we simply did not have enough memory to run all the processes and some workers crashed with out-of-memory exceptions and they caused the Local executor to get the EOF error on the queue reading from those processes. 
   
   I fixed it by detecting on CI how many processes we have (2 on Travis) and setting parallelism to that value.
   
   I got consistently failing tests without it and it started to work after that. There is one problem I have to deal with now - there is another "concurrent" test that is failing with those settings so I think I will settle on 4 rather than NUM_PROC originally. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services