You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2020/12/30 00:50:48 UTC
[airflow] branch 2.0.0-add-docs created (now 3af25a7)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch 2.0.0-add-docs
in repository https://gitbox.apache.org/repos/asf/airflow.git.
at 3af25a7 Use 2.0.0 in Airflow docs & Breeze
This branch includes the following new commits:
new 3af25a7 Use 2.0.0 in Airflow docs & Breeze
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[airflow] 01/01: Use 2.0.0 in Airflow docs & Breeze
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch 2.0.0-add-docs
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 3af25a761740a14311fa38e9d97994c7d155ab86
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Wed Dec 30 00:49:06 2020 +0000
Use 2.0.0 in Airflow docs & Breeze
This commit replaces mention of 1.10.14 to the latest available version
2.0.0
---
BREEZE.rst | 12 ++---
CONTRIBUTING.rst | 2 +-
README.md | 8 ++--
breeze | 2 +-
breeze-complete | 1 +
docs/apache-airflow/installation.rst | 12 ++---
docs/apache-airflow/production-deployment.rst | 68 +++++++++++++--------------
7 files changed, 53 insertions(+), 52 deletions(-)
diff --git a/BREEZE.rst b/BREEZE.rst
index ccd6889..ae02c46 100644
--- a/BREEZE.rst
+++ b/BREEZE.rst
@@ -801,7 +801,7 @@ Generating constraints
----------------------
Whenever setup.py gets modified, the CI master job will re-generate constraint files. Those constraint
-files are stored in separated orphan branches: ``constraints-master`` and ``constraint-1-10``.
+files are stored in separated orphan branches: ``constraints-master``, ``constraint-2-0`` and ``constraint-1-10``.
They are stored separately for each python version. Those are
constraint files as described in detail in the
`<CONTRIBUTING.rst#pinned-constraint-files>`_ contributing documentation.
@@ -1283,7 +1283,7 @@ This is the current syntax for `./breeze <./breeze>`_:
If specified, installs Airflow directly from PIP released version. This happens at
image building time in production image and at container entering time for CI image. One of:
- 1.10.14 1.10.12 1.10.11 1.10.10 1.10.9 none wheel sdist
+ 2.0.0 1.10.14 1.10.12 1.10.11 1.10.10 1.10.9 none wheel sdist
When 'none' is used, you can install airflow from local packages. When building image,
airflow package should be added to 'docker-context-files' and
@@ -1538,7 +1538,7 @@ This is the current syntax for `./breeze <./breeze>`_:
Generates pinned constraint files from setup.py. Those files are generated in files folder
- separate files for different python version. Those constraint files when pushed to orphan
- constraint-master and constraint-1-10 branches are used to generate repeatable
+ constraint-master, constraint-2-0 and constraint-1-10 branches are used to generate repeatable
CI builds as well as run repeatable production image builds. You can use those constraints
to predictably install released Airflow versions. This is mainly used to test the constraint
generation - constraints are pushed to the orphan branches by a successful scheduled
@@ -1692,7 +1692,7 @@ This is the current syntax for `./breeze <./breeze>`_:
wheel,sdist,both
- Default:
+ Default:
-v, --verbose
Show verbose information about executed docker, kind, kubectl, helm commands. Useful for
@@ -2119,7 +2119,7 @@ This is the current syntax for `./breeze <./breeze>`_:
wheel,sdist,both
- Default:
+ Default:
-S, --version-suffix-for-pypi SUFFIX
Adds optional suffix to the version in the generated backport package. It can be used
@@ -2389,7 +2389,7 @@ This is the current syntax for `./breeze <./breeze>`_:
If specified, installs Airflow directly from PIP released version. This happens at
image building time in production image and at container entering time for CI image. One of:
- 1.10.14 1.10.12 1.10.11 1.10.10 1.10.9 none wheel sdist
+ 2.0.0 1.10.14 1.10.12 1.10.11 1.10.10 1.10.9 none wheel sdist
When 'none' is used, you can install airflow from local packages. When building image,
airflow package should be added to 'docker-context-files' and
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 10ab03f..91ad56f 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -756,7 +756,7 @@ when installing ``apache-airflow``, you might need to provide additional constra
example ``pip install apache-airflow==1.10.2 Werkzeug<1.0.0``)
However we now have ``constraints-<PYTHON_MAJOR_MINOR_VERSION>.txt`` files generated
-automatically and committed to orphan ``constraints-master`` and ``constraint-1-10`` branches based on
+automatically and committed to orphan ``constraints-master``, ``constraint-2-0` and ``constraint-1-10`` branches based on
the set of all latest working and tested dependency versions. Those
``constraints-<PYTHON_MAJOR_MINOR_VERSION>.txt`` files can be used as
constraints file when installing Apache Airflow - either from the sources:
diff --git a/README.md b/README.md
index cf91c7a..080f18c 100644
--- a/README.md
+++ b/README.md
@@ -150,15 +150,15 @@ of extras. In order to install Airflow you need to either downgrade pip to versi
```bash
-pip install apache-airflow==1.10.14 \
- --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.7.txt"
+pip install apache-airflow==2.0.0 \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.0.0/constraints-3.7.txt"
```
2. Installing with extras (for example postgres,google)
```bash
-pip install apache-airflow[postgres,google]==1.10.14 \
- --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.7.txt"
+pip install apache-airflow[postgres,google]==2.0.0 \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.0.0/constraints-3.7.txt"
```
For information on installing backport providers check [/docs/backport-providers.rst][/docs/backport-providers.rst].
diff --git a/breeze b/breeze
index e090515..2780145 100755
--- a/breeze
+++ b/breeze
@@ -1707,7 +1707,7 @@ ${CMDNAME} generate-constraints [FLAGS]
Generates pinned constraint files from setup.py. Those files are generated in files folder
- separate files for different python version. Those constraint files when pushed to orphan
- constraint-master and constraint-1-10 branches are used to generate repeatable
+ constraint-master, constraint-2-0 and constraint-1-10 branches are used to generate repeatable
CI builds as well as run repeatable production image builds. You can use those constraints
to predictably install released Airflow versions. This is mainly used to test the constraint
generation - constraints are pushed to the orphan branches by a successful scheduled
diff --git a/breeze-complete b/breeze-complete
index 48b2aab..2eec59f 100644
--- a/breeze-complete
+++ b/breeze-complete
@@ -51,6 +51,7 @@ _breeze_allowed_package_formats="wheel sdist both"
}
_breeze_allowed_install_airflow_versions=$(cat <<-EOF
+2.0.0
1.10.14
1.10.12
1.10.11
diff --git a/docs/apache-airflow/installation.rst b/docs/apache-airflow/installation.rst
index 3299e62..093edff 100644
--- a/docs/apache-airflow/installation.rst
+++ b/docs/apache-airflow/installation.rst
@@ -61,14 +61,14 @@ and python versions in the URL.
.. code-block:: bash
- AIRFLOW_VERSION=1.10.14
+ AIRFLOW_VERSION=2.0.0
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
# For example: 3.6
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
- # For example: https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt
+ # For example: https://raw.githubusercontent.com/apache/airflow/constraints-2.0.0/constraints-3.6.txt
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
- Please note that with respect to Python 3 support, Airflow 1.10.14 has been
+ Please note that with respect to Python 3 support, Airflow 2.0.0 has been
tested with Python 3.6, 3.7, and 3.8, but does not yet support Python 3.9.
2. Installing with extras (for example postgres, google)
@@ -84,7 +84,7 @@ and python versions in the URL.
.. code-block:: bash
- AIRFLOW_VERSION=1.10.14
+ AIRFLOW_VERSION=2.0.0
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
pip install "apache-airflow[postgres,google]==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
@@ -102,8 +102,8 @@ As of Airflow 2.0 we agreed to certain rules we follow for Python support. They
release schedule of Python, nicely summarized in the
`Python Developer's Guide <https://devguide.python.org/#status-of-python-branches>`_
-1. We finish support for python versions when they reach EOL (For python 3.6 it means that we will remove it
- from being supported on 23.12.2021).
+1. We end support for Python versions when they reach EOL (For Python 3.6 it means that we will stop supporting it
+ on 23.12.2021).
2. The "oldest" supported version of Python is the default one. "Default" is only meaningful in terms of
"smoke tests" in CI PRs which are run using this default version.
diff --git a/docs/apache-airflow/production-deployment.rst b/docs/apache-airflow/production-deployment.rst
index 3ebd885..bef3dfa 100644
--- a/docs/apache-airflow/production-deployment.rst
+++ b/docs/apache-airflow/production-deployment.rst
@@ -151,7 +151,7 @@ You should be aware, about a few things:
.. code-block:: dockerfile
- FROM apache/airflow:1.10.14
+ FROM apache/airflow:2.0.0
USER root
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
@@ -168,7 +168,7 @@ You should be aware, about a few things:
.. code-block:: dockerfile
- FROM apache/airflow:1.10.14
+ FROM apache/airflow:2.0.0
RUN pip install --no-cache-dir --user my-awesome-pip-dependency-to-add
@@ -179,7 +179,7 @@ You should be aware, about a few things:
.. code-block:: dockerfile
- FROM apache/airflow:1.10.14
+ FROM apache/airflow:2.0.0
USER root
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
@@ -212,7 +212,7 @@ in the `<#production-image-build-arguments>`_ chapter below.
Here just a few examples are presented which should give you general understanding of what you can customize.
-This builds the production image in version 3.7 with additional airflow extras from 1.10.14 PyPI package and
+This builds the production image in version 3.7 with additional airflow extras from 2.0.0 PyPI package and
additional apt dev and runtime dependencies.
.. code-block:: bash
@@ -221,8 +221,8 @@ additional apt dev and runtime dependencies.
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALLATION_METHOD="apache-airflow" \
- --build-arg AIRFLOW_VERSION="1.10.14" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
+ --build-arg AIRFLOW_VERSION="2.0.0" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==2.0.0" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
@@ -238,7 +238,7 @@ the same image can be built using ``breeze`` (it supports auto-completion of the
.. code-block:: bash
./breeze build-image \
- --production-image --python 3.7 --install-airflow-version=1.10.14 \
+ --production-image --python 3.7 --install-airflow-version=2.0.0 \
--additional-extras=jdbc --additional-python-deps="pandas" \
--additional-dev-apt-deps="gcc g++" --additional-runtime-apt-deps="default-jre-headless"
@@ -254,8 +254,8 @@ based on example in `this comment <https://github.com/apache/airflow/issues/8605
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALLATION_METHOD="apache-airflow" \
- --build-arg AIRFLOW_VERSION="1.10.14" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
+ --build-arg AIRFLOW_VERSION="2.0.0" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==2.0.0" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
@@ -309,12 +309,12 @@ Preparing the constraint files and wheel files:
rm docker-context-files/*.whl docker-context-files/*.txt
- curl -Lo "docker-context-files/constraints-1-10.txt" \
- https://raw.githubusercontent.com/apache/airflow/constraints-1-10/constraints-3.7.txt
+ curl -Lo "docker-context-files/constraints-2-0.txt" \
+ https://raw.githubusercontent.com/apache/airflow/constraints-2-0/constraints-3.7.txt
pip download --dest docker-context-files \
- --constraint docker-context-files/constraints-1-10.txt \
- apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv]==1.10.14
+ --constraint docker-context-files/constraints-2-0.txt \
+ apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv]==2.0.0
Building the image (after copying the files downloaded to the "docker-context-files" directory:
@@ -322,9 +322,9 @@ Building the image (after copying the files downloaded to the "docker-context-fi
.. code-block:: bash
./breeze build-image \
- --production-image --python 3.7 --install-airflow-version=1.10.14 \
+ --production-image --python 3.7 --install-airflow-version=2.0.0 \
--disable-mysql-client-installation --disable-pip-cache --install-from-local-files-when-building \
- --constraints-location="/docker-context-files/constraints-1-10.txt"
+ --constraints-location="/docker-context-files/constraints-2-0.txt"
or
@@ -334,8 +334,8 @@ or
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALLATION_METHOD="apache-airflow" \
- --build-arg AIRFLOW_VERSION="1.10.14" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
+ --build-arg AIRFLOW_VERSION="2.0.0" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==2.0.0" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
@@ -624,13 +624,13 @@ production image. There are three types of build:
| | | Only used when ``INSTALL_FROM_PYPI`` is set to ``true``. |
+-----------------------------------+------------------------+-----------------------------------------------------------------------------------+
| ``AIRFLOW_INSTALL_VERSION`` | | Optional - might be used for package installation of different Airflow version |
-| | | for example"==1.10.14". For consistency, you should also set``AIRFLOW_VERSION`` |
+| | | for example"==2.0.0". For consistency, you should also set``AIRFLOW_VERSION`` |
| | | to the same value AIRFLOW_VERSION is embedded as label in the image created. |
+-----------------------------------+------------------------+-----------------------------------------------------------------------------------+
| ``AIRFLOW_CONSTRAINTS_REFERENCE`` | ``constraints-master`` | Reference (branch or tag) from GitHub where constraints file is taken from. |
| | | It can be ``constraints-master`` but also can be``constraints-1-10`` for |
| | | 1.10.* installations. In case of building specific version |
-| | | you want to point it to specific tag, for example ``constraints-1.10.14`` |
+| | | you want to point it to specific tag, for example ``constraints-2.0.0`` |
+-----------------------------------+------------------------+-----------------------------------------------------------------------------------+
| ``AIRFLOW_WWW`` | ``www`` | In case of Airflow 2.0 it should be "www", in case of Airflow 1.10 |
| | | series it should be "www_rbac". |
@@ -649,7 +649,7 @@ of 2.0 currently):
docker build .
-This builds the production image in version 3.7 with default extras from 1.10.14 tag and
+This builds the production image in version 3.7 with default extras from 2.0.0 tag and
constraints taken from constraints-1-10-12 branch in GitHub.
.. code-block:: bash
@@ -657,14 +657,14 @@ constraints taken from constraints-1-10-12 branch in GitHub.
docker build . \
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
- --build-arg AIRFLOW_INSTALLATION_METHOD="https://github.com/apache/airflow/archive/1.10.14.tar.gz#egg=apache-airflow" \
+ --build-arg AIRFLOW_INSTALLATION_METHOD="https://github.com/apache/airflow/archive/2.0.0.tar.gz#egg=apache-airflow" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_BRANCH="v1-10-test" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty"
-This builds the production image in version 3.7 with default extras from 1.10.14 PyPI package and
-constraints taken from 1.10.14 tag in GitHub and pre-installed pip dependencies from the top
+This builds the production image in version 3.7 with default extras from 2.0.0 PyPI package and
+constraints taken from 2.0.0 tag in GitHub and pre-installed pip dependencies from the top
of v1-10-test branch.
.. code-block:: bash
@@ -673,15 +673,15 @@ of v1-10-test branch.
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALLATION_METHOD="apache-airflow" \
- --build-arg AIRFLOW_VERSION="1.10.14" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
+ --build-arg AIRFLOW_VERSION="2.0.0" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==2.0.0" \
--build-arg AIRFLOW_BRANCH="v1-10-test" \
- --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.14" \
+ --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-2.0.0" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty"
-This builds the production image in version 3.7 with additional airflow extras from 1.10.14 PyPI package and
-additional python dependencies and pre-installed pip dependencies from 1.10.14 tagged constraints.
+This builds the production image in version 3.7 with additional airflow extras from 2.0.0 PyPI package and
+additional python dependencies and pre-installed pip dependencies from 2.0.0 tagged constraints.
.. code-block:: bash
@@ -689,16 +689,16 @@ additional python dependencies and pre-installed pip dependencies from 1.10.14 t
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALLATION_METHOD="apache-airflow" \
- --build-arg AIRFLOW_VERSION="1.10.14" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
+ --build-arg AIRFLOW_VERSION="2.0.0" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==2.0.0" \
--build-arg AIRFLOW_BRANCH="v1-10-test" \
- --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.14" \
+ --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-2.0.0" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \
--build-arg ADDITIONAL_AIRFLOW_EXTRAS="mssql,hdfs" \
--build-arg ADDITIONAL_PYTHON_DEPS="sshtunnel oauth2client"
-This builds the production image in version 3.7 with additional airflow extras from 1.10.14 PyPI package and
+This builds the production image in version 3.7 with additional airflow extras from 2.0.0 PyPI package and
additional apt dev and runtime dependencies.
.. code-block:: bash
@@ -707,8 +707,8 @@ additional apt dev and runtime dependencies.
--build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
--build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
--build-arg AIRFLOW_INSTALLATION_METHOD="apache-airflow" \
- --build-arg AIRFLOW_VERSION="1.10.14" \
- --build-arg AIRFLOW_INSTALL_VERSION="==1.10.14" \
+ --build-arg AIRFLOW_VERSION="2.0.0" \
+ --build-arg AIRFLOW_INSTALL_VERSION="==2.0.0" \
--build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
--build-arg AIRFLOW_SOURCES_FROM="empty" \
--build-arg AIRFLOW_SOURCES_TO="/empty" \