You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2021/08/07 20:51:43 UTC
[airflow] branch v2-1-test updated (562be8c -> 0948366)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch v2-1-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 562be8c Disable Helm tests when branch is not main
new 8bd1a7d Attempt to reduce flakiness of PythonVirtualeEnv test_airflow_context (#17486)
new 0948366 Install providers from sources in prod image only on main (#17458)
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/build-images.yml | 3 +++
scripts/in_container/entrypoint_ci.sh | 2 ++
tests/operators/test_python.py | 3 +++
3 files changed, 8 insertions(+)
[airflow] 01/02: Attempt to reduce flakiness of PythonVirtualeEnv
test_airflow_context (#17486)
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch v2-1-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 8bd1a7d3148104b9fa920ff59c70896cd0a7c5ef
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Sat Aug 7 22:47:15 2021 +0200
Attempt to reduce flakiness of PythonVirtualeEnv test_airflow_context (#17486)
We have a global limit (60 seconds) for individual test execution,
however 'test_airflow_context' of the Python Virtualenv test might take longer in
case they are run in parallel - because they are using dill serialization
including a lot of serializable data from the context of the task.
We give the test 120 seconds to complete now.
(cherry picked from commit 793afaade115dccda7b544cfe8af756b88350abb)
---
scripts/in_container/entrypoint_ci.sh | 2 ++
tests/operators/test_python.py | 3 +++
2 files changed, 5 insertions(+)
diff --git a/scripts/in_container/entrypoint_ci.sh b/scripts/in_container/entrypoint_ci.sh
index 6ba7f6e..0c00163 100755
--- a/scripts/in_container/entrypoint_ci.sh
+++ b/scripts/in_container/entrypoint_ci.sh
@@ -216,6 +216,8 @@ EXTRA_PYTEST_ARGS=(
"--pythonwarnings=ignore::PendingDeprecationWarning"
"--junitxml=${RESULT_LOG_FILE}"
# timeouts in seconds for individual tests
+ "--timeouts-order"
+ "moi"
"--setup-timeout=20"
"--execution-timeout=60"
"--teardown-timeout=20"
diff --git a/tests/operators/test_python.py b/tests/operators/test_python.py
index 3b1ad92..fd7f259 100644
--- a/tests/operators/test_python.py
+++ b/tests/operators/test_python.py
@@ -921,6 +921,9 @@ class TestPythonVirtualenvOperator(unittest.TestCase):
self._run_as_operator(f, templates_dict={'ds': '{{ ds }}'})
+ # This tests might take longer than default 60 seconds as it is serializing a lot of
+ # context using dill (which is slow apparently).
+ @pytest.mark.execution_timeout(120)
def test_airflow_context(self):
def f(
# basic
[airflow] 02/02: Install providers from sources in prod image only
on main (#17458)
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch v2-1-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 0948366ba47bc9bb529fb1c5c33f1a451426a923
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Sat Aug 7 22:47:48 2021 +0200
Install providers from sources in prod image only on main (#17458)
When we build production image during test in main, we install
providers from current sources, to make sure that all the tests
including Helm Chart/Kubernetes tests are using latest sources for
providers.
However, when we build the prod image during v* branches, we
want to build the production image using latest released providers
instead, because this will be the way it will be built shortly when
we release it. We do not run providers test not helm chart tests in
this branch so it is more important to build the image in the way it
will be built for releases - we run verification then and install
dependencies in the very same way it will be done during release.
(cherry picked from commit 5790f72c663596711a344e4c0e5f1036159d7190)
---
.github/workflows/build-images.yml | 3 +++
1 file changed, 3 insertions(+)
diff --git a/.github/workflows/build-images.yml b/.github/workflows/build-images.yml
index c2a9054..729d13f 100644
--- a/.github/workflows/build-images.yml
+++ b/.github/workflows/build-images.yml
@@ -74,6 +74,7 @@ jobs:
image-build: ${{ steps.dynamic-outputs.outputs.image-build }}
cacheDirective: ${{ steps.dynamic-outputs.outputs.cacheDirective }}
targetBranch: ${{ steps.dynamic-outputs.outputs.targetBranch }}
+ defaultBranch: ${{ steps.selective-checks.outputs.default-branch }}
steps:
# Retrieve it to be able to determine which files has changed in the incoming commit of the PR
# we checkout the target commit and it's parent to be able to compare them
@@ -231,6 +232,8 @@ jobs:
CHECK_IF_BASE_PYTHON_IMAGE_UPDATED: >
${{ github.event_name == 'pull_request_target' && 'false' || 'true' }}
VERSION_SUFFIX_FOR_PYPI: ".dev0"
+ INSTALL_PROVIDERS_FROM_SOURCES: >
+ ${{ needs.build-info.outputs.defaultBranch == 'main' && 'true' || 'false' }}
steps:
- name: Set envs
# Set pull image tag for CI image build, in order to pull the image pushed