You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2020/12/02 23:57:41 UTC
[airflow] 04/05: Update documentation about PIP 20.3 incompatibility
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 4a3e890288bb74a1bcbfe183e19683938bd34c98
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Wed Dec 2 17:37:48 2020 +0100
Update documentation about PIP 20.3 incompatibility
---
CONTRIBUTING.rst | 26 ++++++++++++++++++++++++--
IMAGES.rst | 8 ++++++++
INSTALL | 26 +++++++++++++++++++++++---
LOCAL_VIRTUALENV.rst | 8 ++++++++
README.md | 9 +++++++++
docs/installation.rst | 16 ++++++++++++++++
docs/metrics.rst | 8 ++++++++
docs/security.rst | 23 +++++++++++++++++++++++
docs/start.rst | 9 +++++++++
9 files changed, 128 insertions(+), 5 deletions(-)
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 0c1c9c1..8c4bf35 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -541,6 +541,14 @@ extras can be specified after the usual pip install - for example
installs all development dependencies. There is also ``devel_ci`` that installs
all dependencies needed in the CI environment.
+.. note::
+ On 30th of November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver.
+ This resolver does not yet work with Apache Airflow and might leads to errors in installation -
+ depends on your choice of extras. In order to install Airflow you need to either downgrade
+ pip to version 20.2.4 ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
This is the full list of those extras:
.. START EXTRAS HERE
@@ -591,6 +599,14 @@ the other provider package you can install it adding [extra] after the
``pip install apache-airflow-backport-providers-google[amazon]`` in case you want to use GCP
transfer operators from Amazon ECS.
+.. note::
+ On 30th of November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver.
+ This resolver does not yet work with Apache Airflow and might leads to errors in installation -
+ depends on your choice of extras. In order to install Airflow you need to either downgrade
+ pip to version 20.2.4 ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
If you add a new dependency between different providers packages, it will be detected automatically during
pre-commit phase and pre-commit will fail - and add entry in dependencies.json so that the package extra
dependencies are properly added when package is installed.
@@ -671,6 +687,14 @@ install in case a direct or transitive dependency is released that breaks the in
when installing ``apache-airflow``, you might need to provide additional constraints (for
example ``pip install apache-airflow==1.10.2 Werkzeug<1.0.0``)
+.. note::
+ On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
+ does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice
+ of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
+ ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
However we now have ``constraints-<PYTHON_MAJOR_MINOR_VERSION>.txt`` files generated
automatically and committed to orphan ``constraints-master`` and ``constraint-1-10`` branches based on
the set of all latest working and tested dependency versions. Those
@@ -682,7 +706,6 @@ constraints file when installing Apache Airflow - either from the sources:
pip install -e . \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1-10/constraints-3.6.txt"
-
or from the pypi package:
.. code-block:: bash
@@ -690,7 +713,6 @@ or from the pypi package:
pip install apache-airflow \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1-10/constraints-3.6.txt"
-
This works also with extras - for example:
.. code-block:: bash
diff --git a/IMAGES.rst b/IMAGES.rst
index 339969b..6a04428 100644
--- a/IMAGES.rst
+++ b/IMAGES.rst
@@ -125,6 +125,14 @@ This will build the image using command similar to:
apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv,presto]==1.10.14 \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt"
+.. note::
+ On 30th of November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver.
+ This resolver does not yet work with Apache Airflow and might leads to errors in installation -
+ depends on your choice of extras. In order to install Airflow you need to either downgrade
+ pip to version 20.2.4 ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
You can also build production images from specific Git version via providing ``--install-airflow-reference``
parameter to Breeze (this time constraints are taken from the ``constraints-master`` branch which is the
HEAD of development for constraints):
diff --git a/INSTALL b/INSTALL
index 0e2f582..763ed20 100644
--- a/INSTALL
+++ b/INSTALL
@@ -31,16 +31,36 @@ source PATH_TO_YOUR_VENV/bin/activate
# [required] building and installing by pip (preferred)
pip install .
-# or directly
+NOTE!
+
+On 30th of November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver.
+This resolver does not yet work with Apache Airflow and might leads to errors in installation -
+depends on your choice of extras. In order to install Airflow you need to either downgrade
+pip to version 20.2.4 ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+``--use-deprecated legacy-resolver`` to your pip install command.
+
+
+# or you can install it directly via setup.py
python setup.py install
+
# You can also install recommended version of the dependencies by using
# constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case
# you have problems with installing the current requirements from PyPI.
-# There are different constraint files for different python versions. For example"
+# There are different constraint files for different python versions and you shopuld choose the
+# version of constraints specific for your version.
+# For example:
pip install . \
- --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-master/constraints-3.6.txt"
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt"
+
+
+.. note::
+ On 30th of November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver.
+ This resolver does not yet work with Apache Airflow and might leads to errors in installation -
+ depends on your choice of extras. In order to install Airflow you need to either downgrade
+ pip to version 20.2.4 ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
# You can also install Airflow with extras specified. The list of available extras:
# START EXTRAS HERE
diff --git a/LOCAL_VIRTUALENV.rst b/LOCAL_VIRTUALENV.rst
index 8a20c02..574366d 100644
--- a/LOCAL_VIRTUALENV.rst
+++ b/LOCAL_VIRTUALENV.rst
@@ -118,6 +118,14 @@ To create and initialize the local virtualenv:
pip install -U -e ".[devel,<OTHER EXTRAS>]" # for example: pip install -U -e ".[devel,gcp,postgres]"
+.. note::
+ On 30th of November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver.
+ This resolver does not yet work with Apache Airflow and might leads to errors in installation -
+ depends on your choice of extras. In order to install Airflow you need to either downgrade
+ pip to version 20.2.4 ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
In case you have problems with installing airflow because of some requirements are not installable, you can
try to install it with the set of working constraints (note that there are different constraint files
for different python versions:
diff --git a/README.md b/README.md
index b72b175..79cceed 100644
--- a/README.md
+++ b/README.md
@@ -122,6 +122,15 @@ pip install apache-airflow==1.10.14 \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.7.txt"
```
+**NOTE!!!**
+
+On 30th of November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver.
+This resolver does not yet work with Apache Airflow and might leads to errors in installation -
+depends on your choice of extras. In order to install Airflow you need to either downgrade
+pip to version 20.2.4 `pip upgrade --pip==20.2.4` or, in case you use Pip 20.3, you need to add option
+`--use-deprecated legacy-resolver` to your pip install command.
+
+
2. Installing with extras (for example postgres,gcp)
```bash
pip install apache-airflow[postgres,gcp]==1.10.14 \
diff --git a/docs/installation.rst b/docs/installation.rst
index 4a084e1..ed4f4a0 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -58,6 +58,14 @@ and python versions in the URL.
# For example: https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
+
+.. note::
+ On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
+ does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice
+ of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
+ ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
2. Installing with extras (for example postgres, google)
.. code-block:: bash
@@ -68,6 +76,14 @@ and python versions in the URL.
pip install "apache-airflow[postgres,google]==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
+.. note::
+ On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
+ does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice
+ of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
+ ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
You need certain system level requirements in order to install Airflow. Those are requirements that are known
to be needed for Linux system (Tested on Ubuntu Buster LTS) :
diff --git a/docs/metrics.rst b/docs/metrics.rst
index 1e6e06b..5f09b85 100644
--- a/docs/metrics.rst
+++ b/docs/metrics.rst
@@ -31,6 +31,14 @@ First you must install statsd requirement:
pip install 'apache-airflow[statsd]'
+.. note::
+ On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
+ does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice
+ of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
+ ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
Add the following lines to your configuration file e.g. ``airflow.cfg``
.. code-block:: ini
diff --git a/docs/security.rst b/docs/security.rst
index b22dfc0..5fdf23f 100644
--- a/docs/security.rst
+++ b/docs/security.rst
@@ -320,6 +320,13 @@ To use kerberos authentication, you must install Airflow with the ``kerberos`` e
pip install 'apache-airflow[kerberos]'
+.. note::
+ On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
+ does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice
+ of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
+ ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
OAuth Authentication
--------------------
@@ -359,6 +366,14 @@ To use GHE authentication, you must install Airflow with the ``github_enterprise
pip install 'apache-airflow[github_enterprise]'
+.. note::
+ On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
+ does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice
+ of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
+ ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
Setting up GHE Authentication
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -414,6 +429,14 @@ To use Google authentication, you must install Airflow with the ``google_auth``
pip install 'apache-airflow[google_auth]'
+.. note::
+ On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
+ does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice
+ of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
+ ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
Setting up Google Authentication
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/docs/start.rst b/docs/start.rst
index bff52ae..f2b4322 100644
--- a/docs/start.rst
+++ b/docs/start.rst
@@ -43,6 +43,15 @@ The installation is quick and straightforward.
# visit localhost:8080 in the browser and enable the example dag in the home page
+
+.. note::
+ On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
+ does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice
+ of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
+ ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+ ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
Upon running these commands, Airflow will create the ``$AIRFLOW_HOME`` folder
and lay an "airflow.cfg" file with defaults that get you going fast. You can
inspect the file either in ``$AIRFLOW_HOME/airflow.cfg``, or through the UI in