You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2021/08/17 21:24:12 UTC
[airflow] branch v2-1-test updated (dff4950 -> 02d3703)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch v2-1-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
discard dff4950 Forces rebuilding the image for cache pushing
discard f5954bb Add Changelog updates for 2.1.3
new 53b3adb Forces rebuilding the image for cache pushing
new 02d3703 Add Changelog updates for 2.1.3
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (dff4950)
\
N -- N -- N refs/heads/v2-1-test (02d3703)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG.txt | 38 ++++++++++++++++++++++++++++----------
1 file changed, 28 insertions(+), 10 deletions(-)
[airflow] 01/02: Forces rebuilding the image for cache pushing
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v2-1-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 53b3adba13f9839eb5fb9285fea6383fd40fc4a4
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Aug 16 16:07:47 2021 +0200
Forces rebuilding the image for cache pushing
Fixes bug in pushing latest image to cache on "push/schedule".
When the build is successful and passes all tests vi either
`push' or 'schedule' events, we attempt to rebuild the image
with latest constraints just pushed and push it as a fresh
cache for Github Registry. This keeps the time to build image
small without manually refreshing the cache, it also automatically
checks if there is a new "python" base image available so that
we can use it in the new cache.
There was a bug that the image has not been FORCE_PULLED and
rebuilt in this case - just latest images were used.
This had so far no negative effects because due to test
instability, latest main images pretty much never succeeded in
all tests, so the images in `main` were refreshed manually
periodically anyway. However for v2-1-test the scope of tests
run is far smaller now (no Helm tests, no Provider tests)
and they succeed mostly when they should.
Also PROD image was built without ".dev0" suffix which
also failed.
This PR fixes it so that the images are built properly and pushed.
(cherry picked from commit dac71ef45fdbeba585ddb5b4211b31ef8643371f)
---
.github/workflows/ci.yml | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 0d65286..f8bc251 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -1145,10 +1145,18 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
env:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
+ # Rebuild images before push using the latest constraints (just pushed) without
+ # eager upgrade. Do not wait for images, but rebuild them, but always check if
+ # there is a new Python base image to pull and rebuild. This way, when latest python
+ # is not pushed, we will re-use the last cache to start from and when there is
+ # a new python image, we will rebuild it from scratch (same as during the "build-images.ci")
GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
GITHUB_REGISTRY_PUSH_IMAGE_TAG: "latest"
PUSH_PYTHON_BASE_IMAGE: "true"
+ FORCE_PULL_IMAGES: "true"
CHECK_IF_BASE_PYTHON_IMAGE_UPDATED: "true"
+ GITHUB_REGISTRY_WAIT_FOR_IMAGE: "false"
+ UPGRADE_TO_NEWER_DEPENDENCIES: "false"
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
@@ -1164,6 +1172,8 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
run: ./scripts/ci/images/ci_prepare_ci_image_on_ci.sh
- name: "Prepare PROD image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:latest"
run: ./scripts/ci/images/ci_prepare_prod_image_on_ci.sh
+ env:
+ VERSION_SUFFIX_FOR_PYPI: ".dev0"
- name: "Push CI image ${{ env.PYTHON_MAJOR_MINOR_VERSION }}:latest"
run: ./scripts/ci/images/ci_push_ci_images.sh
- name: "Push PROD images ${{ env.PYTHON_MAJOR_MINOR_VERSION }}:latest"
[airflow] 02/02: Add Changelog updates for 2.1.3
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v2-1-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 02d370312ea962999c545a40259ed0980856699a
Author: James Timmins <ja...@astronomer.io>
AuthorDate: Mon Aug 16 18:39:18 2021 -0700
Add Changelog updates for 2.1.3
---
CHANGELOG.txt | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 61 insertions(+)
diff --git a/CHANGELOG.txt b/CHANGELOG.txt
index 0ff1f3f..ab92577 100644
--- a/CHANGELOG.txt
+++ b/CHANGELOG.txt
@@ -1,3 +1,64 @@
+Airflow 2.1.3, 2021-08-23
+-------------------------
+
+Bug Fixes
+"""""""""
+
+- Fix task retries when they receive ``sigkill`` and have retries and properly handle ``sigterm`` (#16301)
+- Fix redacting secrets in context exceptions. (#17618)
+- Fix race condition with dagrun callbacks (#16741)
+- Add 'queued' to DagRunState (#16854)
+- Add 'queued' state to DagRun (#16401)
+- Fix external elasticsearch logs link (#16357)
+- Add proper warning message when recorded PID is different from current PID (#17411)
+- Fix running tasks with ``default_impersonation`` config (#17229)
+- Rescue if a DagRun's DAG was removed from db (#17544)
+- Fixed broken json_client (#17529)
+- Handle and log exceptions raised during task callback (#17347)
+- Fix CLI ``kubernetes cleanup-pods`` which fails on invalid label key (#17298)
+- Show serialization exceptions in DAG parsing log (#17277)
+- Fix: ``TaskInstance`` does not show ``queued_by_job_id`` & ``external_executor_id`` (#17179)
+- Adds more explanatory message when ``SecretsMasker`` is not configured (#17101)
+- Enable the use of ``__init_subclass__`` in subclasses of ``BaseOperator`` (#17027)
+- Fix task instance retrieval in XCom view (#16923)
+- Validate type of ``priority_weight`` during parsing (#16765)
+- Correctly handle custom ``deps` and ``task_group`` during DAG Serialization (#16734)
+- Fix slow (cleared) tasks being be adopted by Celery worker. (#16718)
+- Fix calculating duration in tree view (#16695)
+- Fix ``AttributeError``: ``datetime.timezone`` object has no attribute ``name`` (#16599)
+- Redact conn secrets in webserver logs (#16579)
+- Change graph focus to top of view instead of center (#16484)
+- Fail tasks in scheduler when executor reports they failed (#15929)
+- fix(smart_sensor): Unbound variable errors (#14774)
+- Add back missing permissions to ``UserModelView`` controls. (#17431)
+- Better diagnostics and self-healing of docker-compose (#17484)
+- Improve diagnostics message when users have ``secret_key`` mis-configured (#17410)
+- Stop checking ``execution_date`` in ``task_instance.refresh_from_db`` (#16809)
+
+Improvements
+""""""""""""
+
+- Run mini scheduler in ``LocalTaskJob`` during task exit (#16289)
+- Remove ``SQLAlchemy<1.4`` constraint (#16630)
+- Bump Jinja2 upper-bound from 2.12.0 to 4.0.0 (#16595)
+- Bump ``dnspython`` (#16698)
+- Updates to ``FlaskAppBuilder`` 3.3.2+ (#17208)
+- Add State types for tasks and DAGs (#15285)
+- Set Process title for Worker when using ``LocalExecutor`` (#16623)
+- Move ``DagFileProcessor`` and ``DagFileProcessorProcess`` out of ``scheduler_job.py`` (#16581)
+
+Doc only changes
+""""""""""""""""
+
+- Fix inconsistencies in configuration docs (#17317)
+- Fix docs link for using SQLite as Metadata DB (#17308)
+
+Misc
+""""
+
+- Switch back http provider after requests removes LGPL dependency (#16974)
+
+
Airflow 2.1.2, 2021-07-13
-------------------------