You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2020/12/02 15:29:52 UTC
[airflow] branch v1-10-test updated (653104a -> 2810fb0)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 653104a fixup! [AIRFLOW-2886] Generate random Flask SECRET_KEY in default config (#3738)
new 9b85931 Add Changelog for 1.10.14
new 2810fb0 Bump Airflow Version to 1.10.14
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG.txt | 29 ++++++++++++++++++++++++
IMAGES.rst | 18 +++++++--------
README.md | 12 +++++-----
airflow/version.py | 2 +-
docs/installation.rst | 8 +++----
docs/production-deployment.rst | 50 +++++++++++++++++++++---------------------
6 files changed, 74 insertions(+), 45 deletions(-)
[airflow] 01/02: Add Changelog for 1.10.14
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 9b85931219f2e25020d0a5d48fdd1cf803df3c51
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Wed Dec 2 15:25:02 2020 +0000
Add Changelog for 1.10.14
---
CHANGELOG.txt | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/CHANGELOG.txt b/CHANGELOG.txt
index b818fef..bc02a7f 100644
--- a/CHANGELOG.txt
+++ b/CHANGELOG.txt
@@ -1,3 +1,32 @@
+Airflow 1.10.14, 2020-12-05
+----------------------------
+
+Bug Fixes
+"""""""""
+
+- BugFix: Tasks with ``depends_on_past`` or ``task_concurrency`` are stuck (#12663)
+- Fix issue with empty Resources in executor_config (#12633)
+- Fix: Deprecated config ``force_log_out_after`` was not used (#12661)
+- Fix empty asctime field in JSON formatted logs (#10515)
+- [AIRFLOW-2809] Fix security issue regarding Flask SECRET_KEY (#3651)
+- [AIRFLOW-2884] Fix Flask SECRET_KEY security issue in www_rbac (#3729)
+- [AIRFLOW-2886] Generate random Flask SECRET_KEY in default config (#3738)
+
+
+Improvements
+""""""""""""
+
+- Update setup.py to get non-conflicting set of dependencies (#12636)
+- Rename ``[scheduler] max_threads`` to ``[scheduler] parsing_processes`` (#12605)
+- Add metric for scheduling delay between first run task & expected start time (#9544)
+
+
+Doc only changes
+""""""""""""""""
+
+- Clarified information about supported Databases
+
+
Airflow 1.10.13, 2020-11-24
----------------------------
[airflow] 02/02: Bump Airflow Version to 1.10.14
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 2810fb055bcd7b5d6724c3703c7ff9bb2404bfce
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Wed Dec 2 15:28:28 2020 +0000
Bump Airflow Version to 1.10.14
---
IMAGES.rst | 18 +++++++--------
README.md | 12 +++++-----
airflow/version.py | 2 +-
docs/installation.rst | 8 +++----
docs/production-deployment.rst | 50 +++++++++++++++++++++---------------------
5 files changed, 45 insertions(+), 45 deletions(-)
diff --git a/IMAGES.rst b/IMAGES.rst
index 724d73c..339969b 100644
--- a/IMAGES.rst
+++ b/IMAGES.rst
@@ -39,7 +39,7 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.13``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
the future ``2.*`` labels are build from git tags and they are "fixed" once built.
* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
@@ -115,15 +115,15 @@ parameter to Breeze:
.. code-block:: bash
./breeze build-image --python 3.7 --additional-extras=presto \
- --production-image --install-airflow-version=1.10.13
+ --production-image --install-airflow-version=1.10.14
This will build the image using command similar to:
.. code-block:: bash
pip install \
- apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv,presto]==1.10.13 \
- --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.13/constraints-3.6.txt"
+ apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv,presto]==1.10.14 \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt"
You can also build production images from specific Git version via providing ``--install-airflow-reference``
parameter to Breeze (this time constraints are taken from the ``constraints-master`` branch which is the
@@ -210,8 +210,8 @@ For example:
apache/airflow:master-python3.6 - production "latest" image from current master
apache/airflow:master-python3.6-ci - CI "latest" image from current master
apache/airflow:v1-10-test-python2.7-ci - CI "latest" image from current v1-10-test branch
- apache/airflow:1.10.13-python3.6 - production image for 1.10.13 release
- apache/airflow:1.10.13-1-python3.6 - production image for 1.10.13 with some patches applied
+ apache/airflow:1.10.14-python3.6 - production image for 1.10.14 release
+ apache/airflow:1.10.14-1-python3.6 - production image for 1.10.14 with some patches applied
You can see DockerHub images at `<https://hub.docker.com/repository/docker/apache/airflow>`_
@@ -292,7 +292,7 @@ additional apt dev and runtime dependencies.
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
@@ -308,7 +308,7 @@ the same image can be built using ``breeze`` (it supports auto-completion of the
.. code-block:: bash
./breeze build-image -f Dockerfile.ci \
- --production-image --python 3.7 --install-airflow-version=1.10.13 \
+ --production-image --python 3.7 --install-airflow-version=1.10.14 \
--additional-extras=jdbc --additional-python-deps="pandas" \
--additional-dev-apt-deps="gcc g++" --additional-runtime-apt-deps="default-jre-headless"
You can build the default production image with standard ``docker build`` command but they will only build
@@ -326,7 +326,7 @@ based on example in `this comment <https://github.com/apache/airflow/issues/8605
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
diff --git a/README.md b/README.md
index 5a5edd6..b72b175 100644
--- a/README.md
+++ b/README.md
@@ -76,7 +76,7 @@ Airflow is not a streaming solution, but it is often used to process real-time d
Apache Airflow is tested with:
-| | Master version (2.0.0dev) | Stable version (1.10.13) |
+| | Master version (2.0.0dev) | Stable version (1.10.14) |
| ------------ | ------------------------- | ------------------------ |
| Python | 3.6, 3.7, 3.8 | 2.7, 3.5, 3.6, 3.7, 3.8 |
| PostgreSQL | 9.6, 10, 11, 12, 13 | 9.6, 10, 11, 12, 13 |
@@ -109,7 +109,7 @@ if needed. This means that from time to time plain `pip install apache-airflow`
produce unusable Airflow installation.
In order to have repeatable installation, however, introduced in **Airflow 1.10.10** and updated in
-**Airflow 1.10.13** we also keep a set of "known-to-be-working" constraint files in the
+**Airflow 1.10.12** we also keep a set of "known-to-be-working" constraint files in the
orphan `constraints-master` and `constraints-1-10` branches. We keep those "known-to-be-working"
constraints files separately per major/minor python version.
You can use them as constraint files when installing Airflow from PyPI. Note that you have to specify
@@ -118,14 +118,14 @@ correct Airflow tag/version/branch and python versions in the URL.
1. Installing just Airflow:
```bash
-pip install apache-airflow==1.10.13 \
- --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.13/constraints-3.7.txt"
+pip install apache-airflow==1.10.14 \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.7.txt"
```
2. Installing with extras (for example postgres,gcp)
```bash
-pip install apache-airflow[postgres,gcp]==1.10.13 \
- --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.13/constraints-3.7.txt"
+pip install apache-airflow[postgres,gcp]==1.10.14 \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.7.txt"
```
For information on installing backport providers check https://airflow.readthedocs.io/en/latest/backport-providers.html.
diff --git a/airflow/version.py b/airflow/version.py
index 115c560..b3b5b30 100644
--- a/airflow/version.py
+++ b/airflow/version.py
@@ -18,4 +18,4 @@
# under the License.
#
-version = '1.10.13'
+version = '1.10.14'
diff --git a/docs/installation.rst b/docs/installation.rst
index 12ce19e..4a084e1 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -31,7 +31,7 @@ if needed. This means that from time to time plain ``pip install apache-airflow`
produce unusable Airflow installation.
In order to have repeatable installation, however, starting from **Airflow 1.10.10** and updated in
-**Airflow 1.10.13** we also keep a set of "known-to-be-working" constraint files in the
+**Airflow 1.10.12** we also keep a set of "known-to-be-working" constraint files in the
``constraints-master`` and ``constraints-1-10`` orphan branches.
Those "known-to-be-working" constraints are per major/minor python version. You can use them as constraint
files when installing Airflow from PyPI. Note that you have to specify correct Airflow version
@@ -51,18 +51,18 @@ and python versions in the URL.
.. code-block:: bash
- AIRFLOW_VERSION=1.10.13
+ AIRFLOW_VERSION=1.10.14
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
# For example: 3.6
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
- # For example: https://raw.githubusercontent.com/apache/airflow/constraints-1.10.13/constraints-3.6.txt
+ # For example: https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
2. Installing with extras (for example postgres, google)
.. code-block:: bash
- AIRFLOW_VERSION=1.10.13
+ AIRFLOW_VERSION=1.10.14
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
pip install "apache-airflow[postgres,google]==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
diff --git a/docs/production-deployment.rst b/docs/production-deployment.rst
index 3edddb8..ac6c76d 100644
--- a/docs/production-deployment.rst
+++ b/docs/production-deployment.rst
@@ -64,7 +64,7 @@ You should be aware, about a few things:
.. code-block:: dockerfile
- FROM: apache/airflow:1.10.13
+ FROM: apache/airflow:1.10.14
USER root
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
@@ -81,7 +81,7 @@ You should be aware, about a few things:
.. code-block:: dockerfile
- FROM: apache/airflow:1.10.13
+ FROM: apache/airflow:1.10.14
RUN pip install --no-cache-dir --user my-awesome-pip-dependency-to-add
@@ -92,7 +92,7 @@ You should be aware, about a few things:
.. code-block:: dockerfile
- FROM: apache/airflow:1.10.13
+ FROM: apache/airflow:1.10.14
USER root
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
@@ -125,7 +125,7 @@ in the `<#production-image-build-arguments>`_ chapter below.
Here just a few examples are presented which should give you general understanding of what you can customize.
-This builds the production image in version 3.7 with additional airflow extras from 1.10.13 PyPI package and
+This builds the production image in version 3.7 with additional airflow extras from 1.10.14 PyPI package and
additional apt dev and runtime dependencies.
.. code-block:: bash
@@ -134,7 +134,7 @@ additional apt dev and runtime dependencies.
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
@@ -150,7 +150,7 @@ the same image can be built using ``breeze`` (it supports auto-completion of the
.. code-block:: bash
./breeze build-image \
- --production-image --python 3.7 --install-airflow-version=1.10.13 \
+ --production-image --python 3.7 --install-airflow-version=1.10.14 \
--additional-extras=jdbc --additional-python-deps="pandas" \
--additional-dev-apt-deps="gcc g++" --additional-runtime-apt-deps="default-jre-headless"
@@ -166,7 +166,7 @@ based on example in `this comment <https://github.com/apache/airflow/issues/8605
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
@@ -225,7 +225,7 @@ Preparing the constraint files and wheel files:
pip download --dest docker-context-files \
--constraint docker-context-files/constraints-1-10.txt \
- apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv]==1.10.13
+ apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv]==1.10.14
Building the image (after copying the files downloaded to the "docker-context-files" directory:
@@ -233,7 +233,7 @@ Building the image (after copying the files downloaded to the "docker-context-fi
.. code-block:: bash
./breeze build-image \
- --production-image --python 3.7 --install-airflow-version=1.10.13 \
+ --production-image --python 3.7 --install-airflow-version=1.10.14 \
--disable-mysql-client-installation --disable-pip-cache --add-local-pip-wheels \
--constraints-location="/docker-context-files/constraints-1-10.txt"
@@ -245,7 +245,7 @@ or
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
@@ -392,7 +392,7 @@ The following build arguments (``--build-arg`` in docker build command) can be u
| | | ``constraints-master`` but can be |
| | | ``constraints-1-10`` for 1.10.* versions |
| | | or it could point to specific version |
-| | | for example ``constraints-1.10.13`` |
+| | | for example ``constraints-1.10.14`` |
+------------------------------------------+------------------------------------------+------------------------------------------+
| ``AIRFLOW_EXTRAS`` | (see Dockerfile) | Default extras with which airflow is |
| | | installed |
@@ -503,7 +503,7 @@ production image. There are three types of build:
| ``AIRFLOW_INSTALL_VERSION`` | Optional - might be used for |
| | package installation case to |
| | set Airflow version for example |
-| | "==1.10.13" |
+| | "==1.10.14" |
+-----------------------------------+-----------------------------------+
| ``AIRFLOW_CONSTRAINTS_REFERENCE`` | reference (branch or tag) from |
| | GitHub where constraints file |
@@ -512,7 +512,7 @@ production image. There are three types of build:
| | ``constraints-1-10`` for 1.10.* |
| | constraint or if you want to |
| | point to specific version |
-| | might be ``constraints-1.10.13`` |
+| | might be ``constraints-1.10.14`` |
+-----------------------------------+-----------------------------------+
| ``SLUGIFY_USES_TEXT_UNIDECODE`` | In case of of installing airflow |
| | 1.10.2 or 1.10.1 you need to |
@@ -546,7 +546,7 @@ of 2.0 currently):
docker build .
-This builds the production image in version 3.7 with default extras from 1.10.13 tag and
+This builds the production image in version 3.7 with default extras from 1.10.14 tag and
constraints taken from constraints-1-10-12 branch in GitHub.
.. code-block:: bash
@@ -554,14 +554,14 @@ constraints taken from constraints-1-10-12 branch in GitHub.
docker build . \
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
- --build-arg AIRFLOW_INSTALL_SOURCES="https://github.com/apache/airflow/archive/1.10.13.tar.gz#egg=apache-airflow" \
+ --build-arg AIRFLOW_INSTALL_SOURCES="https://github.com/apache/airflow/archive/1.10.14.tar.gz#egg=apache-airflow" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_BRANCH="v1-10-test" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty"
-This builds the production image in version 3.7 with default extras from 1.10.13 PyPI package and
-constraints taken from 1.10.13 tag in GitHub and pre-installed pip dependencies from the top
+This builds the production image in version 3.7 with default extras from 1.10.14 PyPI package and
+constraints taken from 1.10.14 tag in GitHub and pre-installed pip dependencies from the top
of v1-10-test branch.
.. code-block:: bash
@@ -570,14 +570,14 @@ of v1-10-test branch.
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
--build-arg AIRFLOW_BRANCH="v1-10-test" \
- --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.13" \
+ --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.14" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty"
-This builds the production image in version 3.7 with additional airflow extras from 1.10.13 PyPI package and
-additional python dependencies and pre-installed pip dependencies from 1.10.13 tagged constraints.
+This builds the production image in version 3.7 with additional airflow extras from 1.10.14 PyPI package and
+additional python dependencies and pre-installed pip dependencies from 1.10.14 tagged constraints.
.. code-block:: bash
@@ -585,15 +585,15 @@ additional python dependencies and pre-installed pip dependencies from 1.10.13 t
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
--build-arg AIRFLOW_BRANCH="v1-10-test" \
- --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.13" \
+ --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.14" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
--build-arg ADDITIONAL_AIRFLOW_EXTRAS="mssql,hdfs"
--build-arg ADDITIONAL_PYTHON_DEPS="sshtunnel oauth2client"
-This builds the production image in version 3.7 with additional airflow extras from 1.10.13 PyPI package and
+This builds the production image in version 3.7 with additional airflow extras from 1.10.14 PyPI package and
additional apt dev and runtime dependencies.
.. code-block:: bash
@@ -602,7 +602,7 @@ additional apt dev and runtime dependencies.
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \