You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ep...@apache.org on 2022/10/18 18:02:39 UTC
[airflow] branch v2-4-test updated (e271d16080 -> 6479853fb0)
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a change to branch v2-4-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
from e271d16080 Fix problem with building providers where constraints are conflicting (#26420)
new 0d95df0bb2 Do not restart breeze containers after restart (#26647)
new be935af91e Remove asset compilation for CI image builds (#26665)
new ae511653d0 Skip sdist providers installation for non-canary builds (#26671)
new c4dc56878f Make build-images workflow name dynamic (#26690)
new 9691512810 Update Airflow release docs (#26727)
new 6479853fb0 Update message with new breeze subcommand (#26817)
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/build-images.yml | 4 ++--
.github/workflows/ci.yml | 8 ++++----
dev/README_RELEASE_AIRFLOW.md | 8 ++++++--
dev/breeze/src/airflow_breeze/utils/reinstall.py | 4 ++--
scripts/ci/docker-compose/backend-mssql.yml | 2 +-
scripts/ci/docker-compose/backend-mysql.yml | 2 +-
scripts/ci/docker-compose/backend-postgres.yml | 2 +-
scripts/ci/docker-compose/integration-cassandra.yml | 2 +-
scripts/ci/docker-compose/integration-kerberos.yml | 2 +-
scripts/ci/docker-compose/integration-mongo.yml | 2 +-
scripts/ci/docker-compose/integration-openldap.yml | 2 +-
scripts/ci/docker-compose/integration-pinot.yml | 2 +-
scripts/ci/docker-compose/integration-rabbitmq.yml | 2 +-
scripts/ci/docker-compose/integration-redis.yml | 2 +-
scripts/ci/docker-compose/integration-trino.yml | 2 +-
15 files changed, 25 insertions(+), 21 deletions(-)
[airflow] 05/06: Update Airflow release docs (#26727)
Posted by ep...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-4-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 96915128107cbcbd5742c051f43f6416d8a44be2
Author: Jed Cunningham <66...@users.noreply.github.com>
AuthorDate: Tue Sep 27 14:54:00 2022 -0700
Update Airflow release docs (#26727)
Just some things I ran across when releasing 2.4.1rc1.
(cherry picked from commit ca480fc0e65ea92d3f5918c1b2047b9e12e36826)
---
dev/README_RELEASE_AIRFLOW.md | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/dev/README_RELEASE_AIRFLOW.md b/dev/README_RELEASE_AIRFLOW.md
index 5397cf5f28..16a09411ec 100644
--- a/dev/README_RELEASE_AIRFLOW.md
+++ b/dev/README_RELEASE_AIRFLOW.md
@@ -218,6 +218,7 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag
```shell script
git checkout v${VERSION_BRANCH}-test
+ git reset --hard origin/v${VERSION_BRANCH}-test
```
- Set your version in `setup.py` (without the RC tag)
@@ -241,12 +242,14 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag
```shell script
git checkout v${VERSION_BRANCH}-stable
+ git reset --hard origin/v${VERSION_BRANCH}-stable
```
- PR from the 'test' branch to the 'stable' branch, and manually merge it once approved. Here's how to manually merge the PR:
```shell script
git merge --ff-only v${VERSION_BRANCH}-test
+ git push origin v${VERSION_BRANCH}-stable
```
- Tag your release
@@ -276,7 +279,6 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag
- Tarball the repo
```shell script
- mkdir dist
git archive --format=tar.gz ${VERSION} \
--prefix=apache-airflow-${VERSION_WITHOUT_RC}/ \
-o dist/apache-airflow-${VERSION_WITHOUT_RC}-source.tar.gz
@@ -307,7 +309,7 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag
```shell script
git checkout origin/constraints-${VERSION_BRANCH}
git tag -s "constraints-${VERSION}" -m "Constraints for Apache Airflow ${VERSION}"
- git push origin "constraints-${VERSION}"
+ git push origin tag "constraints-${VERSION}"
```
- Push the artifacts to ASF dev dist repo
@@ -327,6 +329,8 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag
cd ${VERSION}
svn add *
svn commit -m "Add artifacts for Airflow ${VERSION}"
+ cd ${AIRFLOW_REPO_ROOT}
+ rm -rf asf-dist
```
## Prepare new release branches and cache - optional when first minor version is released
[airflow] 02/06: Remove asset compilation for CI image builds (#26665)
Posted by ep...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-4-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit be935af91e53eacbbc59536d2a2501ee733e3628
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Sep 26 10:31:39 2022 +0200
Remove asset compilation for CI image builds (#26665)
The asset compilation is not actually used in CI image so it is not
really needed to be run in CI.
(cherry picked from commit df3bfe3219da340c566afc9602278e2751889c70)
---
.github/workflows/build-images.yml | 2 --
.github/workflows/ci.yml | 3 ---
2 files changed, 5 deletions(-)
diff --git a/.github/workflows/build-images.yml b/.github/workflows/build-images.yml
index f8433ab92e..c497db74d9 100644
--- a/.github/workflows/build-images.yml
+++ b/.github/workflows/build-images.yml
@@ -227,8 +227,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
breeze static-checks --type update-providers-dependencies --all-files
--show-diff-on-failure --color always || true
if: needs.build-info.outputs.default-branch == 'main'
- - name: Compile www assets
- run: breeze compile-www-assets
- name: >-
Build & Push AMD64 CI images ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
${{ needs.build-info.outputs.all-python-versions-list-as-string }}
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index d8849e2c4f..25957c3a79 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -392,9 +392,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
if: >
needs.build-info.outputs.in-workflow-build == 'true' &&
needs.build-info.outputs.default-branch == 'main'
- - name: Compile www assets
- run: breeze compile-www-assets
- if: needs.build-info.outputs.in-workflow-build == 'true'
- name: >
Build & Push CI images ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
${{ needs.build-info.outputs.all-python-versions-list-as-string }}
[airflow] 06/06: Update message with new breeze subcommand (#26817)
Posted by ep...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-4-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 6479853fb04fce951b013a24385693f80d4b96db
Author: Ross Turk <ro...@rossturk.com>
AuthorDate: Sat Oct 1 09:38:31 2022 -0400
Update message with new breeze subcommand (#26817)
Signed-off-by: Ross Turk <ro...@rossturk.com>
Signed-off-by: Ross Turk <ro...@rossturk.com>
(cherry picked from commit 9a6fc73aba75a03b0dd6c700f0f8932f6a474ff7)
---
dev/breeze/src/airflow_breeze/utils/reinstall.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/dev/breeze/src/airflow_breeze/utils/reinstall.py b/dev/breeze/src/airflow_breeze/utils/reinstall.py
index 25a2fa909e..dc01332dab 100644
--- a/dev/breeze/src/airflow_breeze/utils/reinstall.py
+++ b/dev/breeze/src/airflow_breeze/utils/reinstall.py
@@ -47,7 +47,7 @@ def warn_non_editable():
"\n[error]Breeze is installed in a wrong way.[/]\n"
"\n[error]It should only be installed in editable mode[/]\n\n"
"[info]Please go to Airflow sources and run[/]\n\n"
- f" {NAME} self-upgrade --force --use-current-airflow-sources\n"
+ f" {NAME} setup self-upgrade --force --use-current-airflow-sources\n"
)
@@ -56,6 +56,6 @@ def warn_dependencies_changed():
f"\n[warning]Breeze dependencies changed since the installation![/]\n\n"
f"[warning]This might cause various problems!![/]\n\n"
f"If you experience problems - reinstall Breeze with:\n\n"
- f" {NAME} self-upgrade --force\n"
+ f" {NAME} setup self-upgrade --force\n"
"\nThis should usually take couple of seconds.\n"
)
[airflow] 01/06: Do not restart breeze containers after restart (#26647)
Posted by ep...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-4-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 0d95df0bb2d46c7396e4623ffdf75bcc1eb398eb
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Sun Sep 25 21:52:50 2022 +0200
Do not restart breeze containers after restart (#26647)
In order to avoid initialisation of database and other integrations
when you are entering/leaving breeze, those are started when breeze
starts but not stopped when you leave. Then stopping such running
auxiliary containers should be done with `breeze stop` after you
are done. However those who do not do it, and will restart their
machine will find that the containers get restarted.
This has been added as part of #13446 where health-checks are added.
However "always" was not a good choice. It should have been "on-failure"
(cherry picked from commit 337146dbff6933bd3819ad173f6878ddc934c419)
---
scripts/ci/docker-compose/backend-mssql.yml | 2 +-
scripts/ci/docker-compose/backend-mysql.yml | 2 +-
scripts/ci/docker-compose/backend-postgres.yml | 2 +-
scripts/ci/docker-compose/integration-cassandra.yml | 2 +-
scripts/ci/docker-compose/integration-kerberos.yml | 2 +-
scripts/ci/docker-compose/integration-mongo.yml | 2 +-
scripts/ci/docker-compose/integration-openldap.yml | 2 +-
scripts/ci/docker-compose/integration-pinot.yml | 2 +-
scripts/ci/docker-compose/integration-rabbitmq.yml | 2 +-
scripts/ci/docker-compose/integration-redis.yml | 2 +-
scripts/ci/docker-compose/integration-trino.yml | 2 +-
11 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/scripts/ci/docker-compose/backend-mssql.yml b/scripts/ci/docker-compose/backend-mssql.yml
index f5953a0bfe..877eae2ea9 100644
--- a/scripts/ci/docker-compose/backend-mssql.yml
+++ b/scripts/ci/docker-compose/backend-mssql.yml
@@ -40,7 +40,7 @@ services:
timeout: 10s
retries: 10
start_period: 2m
- restart: always
+ restart: "on-failure"
mssqlsetup:
image: mcr.microsoft.com/mssql/server:${MSSQL_VERSION}
depends_on:
diff --git a/scripts/ci/docker-compose/backend-mysql.yml b/scripts/ci/docker-compose/backend-mysql.yml
index 990633a557..34e9d34f9e 100644
--- a/scripts/ci/docker-compose/backend-mysql.yml
+++ b/scripts/ci/docker-compose/backend-mysql.yml
@@ -41,7 +41,7 @@ services:
interval: 10s
timeout: 10s
retries: 5
- restart: always
+ restart: "on-failure"
command: ['mysqld', '--character-set-server=utf8mb4',
'--collation-server=utf8mb4_unicode_ci']
volumes:
diff --git a/scripts/ci/docker-compose/backend-postgres.yml b/scripts/ci/docker-compose/backend-postgres.yml
index 8f9ee2dcce..afeb9472f2 100644
--- a/scripts/ci/docker-compose/backend-postgres.yml
+++ b/scripts/ci/docker-compose/backend-postgres.yml
@@ -42,6 +42,6 @@ services:
interval: 10s
timeout: 10s
retries: 5
- restart: always
+ restart: "on-failure"
volumes:
postgres-db-volume:
diff --git a/scripts/ci/docker-compose/integration-cassandra.yml b/scripts/ci/docker-compose/integration-cassandra.yml
index b0ad799dbd..953611c1a7 100644
--- a/scripts/ci/docker-compose/integration-cassandra.yml
+++ b/scripts/ci/docker-compose/integration-cassandra.yml
@@ -32,7 +32,7 @@ services:
interval: 5s
timeout: 30s
retries: 50
- restart: always
+ restart: "on-failure"
airflow:
environment:
diff --git a/scripts/ci/docker-compose/integration-kerberos.yml b/scripts/ci/docker-compose/integration-kerberos.yml
index 1df0a17f86..9edaa571d0 100644
--- a/scripts/ci/docker-compose/integration-kerberos.yml
+++ b/scripts/ci/docker-compose/integration-kerberos.yml
@@ -49,7 +49,7 @@ services:
interval: 5s
timeout: 30s
retries: 50
- restart: always
+ restart: "on-failure"
airflow:
hostname: krb5-machine-example-com
diff --git a/scripts/ci/docker-compose/integration-mongo.yml b/scripts/ci/docker-compose/integration-mongo.yml
index 44fcc7b1ec..0c41bb3485 100644
--- a/scripts/ci/docker-compose/integration-mongo.yml
+++ b/scripts/ci/docker-compose/integration-mongo.yml
@@ -27,7 +27,7 @@ services:
interval: 5s
timeout: 30s
retries: 50
- restart: always
+ restart: "on-failure"
airflow:
environment:
diff --git a/scripts/ci/docker-compose/integration-openldap.yml b/scripts/ci/docker-compose/integration-openldap.yml
index 9d12fc39f5..1afe1d21e0 100644
--- a/scripts/ci/docker-compose/integration-openldap.yml
+++ b/scripts/ci/docker-compose/integration-openldap.yml
@@ -33,7 +33,7 @@ services:
interval: 5s
timeout: 30s
retries: 50
- restart: always
+ restart: "on-failure"
airflow:
environment:
diff --git a/scripts/ci/docker-compose/integration-pinot.yml b/scripts/ci/docker-compose/integration-pinot.yml
index 70262e1ad7..730fd1a2c3 100644
--- a/scripts/ci/docker-compose/integration-pinot.yml
+++ b/scripts/ci/docker-compose/integration-pinot.yml
@@ -29,7 +29,7 @@ services:
interval: 5s
timeout: 30s
retries: 50
- restart: always
+ restart: "on-failure"
airflow:
environment:
diff --git a/scripts/ci/docker-compose/integration-rabbitmq.yml b/scripts/ci/docker-compose/integration-rabbitmq.yml
index 8610cf540d..bf04b4e84f 100644
--- a/scripts/ci/docker-compose/integration-rabbitmq.yml
+++ b/scripts/ci/docker-compose/integration-rabbitmq.yml
@@ -27,7 +27,7 @@ services:
interval: 5s
timeout: 30s
retries: 50
- restart: always
+ restart: "on-failure"
airflow:
environment:
diff --git a/scripts/ci/docker-compose/integration-redis.yml b/scripts/ci/docker-compose/integration-redis.yml
index 2b5f136260..abefa80ff3 100644
--- a/scripts/ci/docker-compose/integration-redis.yml
+++ b/scripts/ci/docker-compose/integration-redis.yml
@@ -29,7 +29,7 @@ services:
interval: 5s
timeout: 30s
retries: 50
- restart: always
+ restart: "on-failure"
airflow:
environment:
diff --git a/scripts/ci/docker-compose/integration-trino.yml b/scripts/ci/docker-compose/integration-trino.yml
index cabe24b4c1..89fe1eec9b 100644
--- a/scripts/ci/docker-compose/integration-trino.yml
+++ b/scripts/ci/docker-compose/integration-trino.yml
@@ -30,7 +30,7 @@ services:
interval: 5s
timeout: 30s
retries: 50
- restart: always
+ restart: "on-failure"
ports:
- "38080:8080"
[airflow] 03/06: Skip sdist providers installation for non-canary builds (#26671)
Posted by ep...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-4-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit ae511653d06f6a1c697a6480abaf5f1354d293ed
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Sep 26 11:22:14 2022 +0200
Skip sdist providers installation for non-canary builds (#26671)
The sdist providers installation takes a long time and it is unlikely
to fail if wheel installation succeeds. We should skip running the
providers builds for PRs to speed them up. In case there will be
any problem with sdist installation, we will catch it in our
"canary" builds anyway.
(cherry picked from commit e5c903c10e8556ce5bc34d4e8a5289e636750c5c)
---
.github/workflows/ci.yml | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 25957c3a79..26d8b71ce5 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -886,7 +886,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
env:
RUNS_ON: ${{ needs.build-info.outputs.runs-on }}
PYTHON_MAJOR_MINOR_VERSION: ${{ needs.build-info.outputs.default-python-version }}
- if: needs.build-info.outputs.image-build == 'true' && needs.build-info.outputs.default-branch == 'main'
+ if: >
+ needs.build-info.outputs.image-build == 'true' &&
+ needs.build-info.outputs.default-branch == 'main' &&
+ needs.build-info.outputs.canary-run == 'true'
steps:
- name: Cleanup repo
run: docker run -v "${GITHUB_WORKSPACE}:/workspace" -u 0:0 bash -c "rm -rf /workspace/*"
[airflow] 04/06: Make build-images workflow name dynamic (#26690)
Posted by ep...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-4-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit c4dc56878f377b312460eed1d0be2c080af8fc03
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Tue Sep 27 13:11:58 2022 +0200
Make build-images workflow name dynamic (#26690)
As of today you can dynamically name your build-images workflow.
This will be nice change to be able to distinguish the "tests"
from build images workflows.
(cherry picked from commit 2536c00111516a8373b183f5c9173e9771f08326)
---
.github/workflows/build-images.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/build-images.yml b/.github/workflows/build-images.yml
index c497db74d9..a2a28cbed1 100644
--- a/.github/workflows/build-images.yml
+++ b/.github/workflows/build-images.yml
@@ -17,6 +17,8 @@
#
---
name: "Build Images"
+run-name: >
+ Build images for ${{ github.event.pull_request.title }} ${{ github.event.pull_request._links.html.href }}
on: # yamllint disable-line rule:truthy
pull_request_target:
permissions: