You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/07/11 08:48:09 UTC
[GitHub] [airflow] potiuk opened a new pull request #16927: Disable cache for virtualenv for K8S tests
potiuk opened a new pull request #16927:
URL: https://github.com/apache/airflow/pull/16927
The cache seems to get broken easily due to use of symbolic links
and it is not as long to create the virtualenv when needed.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16927: Fix cache for K8S test virtualenv
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16927:
URL: https://github.com/apache/airflow/pull/16927#issuecomment-877771360
Yeah. Removing cache is best. We are also running different K8S for different python versions and this makes the attempts to cache the venv really bad idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16927: Fix cache for K8S test virtualenv
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16927:
URL: https://github.com/apache/airflow/pull/16927#issuecomment-877765265
The tests in main were failing with (https://github.com/apache/airflow/runs/3039240162?check_suite_focus=true#step:8:1585)
` /home/runner/actions-runner/_work/airflow/airflow/.build/.kubernetes_venv/airflow-python-3.9-v1.18.15_host_python_3.6_CeleryExecutor/bin/pip: 2: exec: /home/runner/actions-runner/_work/airflow/airflow/.build/.kubernetes_venv/airflow-python-3.9-v1.18.15_host_python_3.6_CeleryExecutor/bin/python3: not found`
Because the python3 binary was missing from the cache (I believe)
Let's see if it helps. Another hypothesis I had was that the python3 symbolic link is not properly stored in the cache of GitHub Actions. So if that one does not work we can simply disable the cache altogether (it does not take that long to build airflow's venv with [cncf.kubernetes, devel] extra only, and this is what the venv really is - it is needed to run all the k8s tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16927: Remove cache for kubernetes tests
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16927:
URL: https://github.com/apache/airflow/pull/16927#issuecomment-877780268
It works, but I forgot that the "upgrade" test had it's own cache. Removed it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #16927: Remove cache for kubernetes tests
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #16927:
URL: https://github.com/apache/airflow/pull/16927
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16927: Remove cache for kubernetes tests
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16927:
URL: https://github.com/apache/airflow/pull/16927#issuecomment-877798948
> Nice. I have a PR I have not merged because of failing helm tests!
Go ahead :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16927: Fix cache for K8S test virtualenv
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16927:
URL: https://github.com/apache/airflow/pull/16927#issuecomment-877764915
Hey @ephraimbuddy -> I think this is the final reason for K8S tests failing for 'main` repo. I think the venv cache has been irreparably broken by some early attempts of running the tests. I am rebuilding the cache from scratch with this PR and hopefully the "main" K8S tests will finally get green as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org