[GitHub] [airflow] potiuk edited a comment on pull request #13730: Run CI (Build Image and committer PRs) on self-hosted runner

[GitBox <gi...@apache.org>]

potiuk edited a comment on pull request #13730:
URL: https://github.com/apache/airflow/pull/13730#issuecomment-773678053


   @ashb  -> just to reiterate - those are very same failures I used to observe in many CI environments earlier. The biggest benefit of having full control over the CI machines is the ability to switch to in-memory-only operations.
   
   Switching to `tmpfs` solves the need of solving those kind of I/O failures in the cloud environment as well ups speeds up everything several times (sometimes even 10 times).
   
   I spent hourse  days on solving these kind of problems in the past and they are gone entirely with several times overall build time improvement when we switch to tmpfs for both the source code location (for any kind of node setup/precompiling/parsing/execution) and docker files (for actual image building/container execution etc.). 
   
   I could have saved days  of work last time i've done similar workarouds for spurious I/O failures - and then I stopped worrying about those kind of problems altogether when switched to tmpfs. This is like being in another world entirely. 
   
   Just saying.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org