You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/14 02:47:47 UTC
[GitHub] [airflow] mik-laj opened a new pull request #13661: Structure and small improvements in installation.rst
mik-laj opened a new pull request #13661:
URL: https://github.com/apache/airflow/pull/13661
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #13661: Structure and small improvements in installation.rst
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #13661:
URL: https://github.com/apache/airflow/pull/13661#discussion_r557010506
##########
File path: docs/apache-airflow/installation.rst
##########
@@ -38,117 +45,70 @@ Airflow is tested with:
* Kubernetes: 1.16.9, 1.17.5, 1.18.6
**Note:** MySQL 5.x versions are unable to or have limitations with
-running multiple schedulers -- please see the "Scheduler" docs. MariaDB is not tested/recommended.
+running multiple schedulers -- please see: :doc:`/scheduler`. MariaDB is not tested/recommended.
**Note:** SQLite is used in Airflow tests. Do not use it in production. We recommend
using the latest stable version of SQLite for local development.
-Getting Airflow
-'''''''''''''''
-
-Airflow is published as ``apache-airflow`` package in PyPI. Installing it however might be sometimes tricky
-because Airflow is a bit of both a library and application. Libraries usually keep their dependencies open and
-applications usually pin them, but we should do neither and both at the same time. We decided to keep
-our dependencies as open as possible (in ``setup.cfg`` and ``setup.py``) so users can install different
-version of libraries if needed. This means that from time to time plain ``pip install apache-airflow`` will
-not work or will produce unusable Airflow installation.
+Please note that with respect to Python 3 support, Airflow 2.0.0 has been
+tested with Python 3.6, 3.7, and 3.8, but does not yet support Python 3.9.
-In order to have repeatable installation, however, starting from **Airflow 1.10.10** and updated in
-**Airflow 1.10.13** we also keep a set of "known-to-be-working" constraint files in the
-``constraints-master``, ``constraints-2-0`` and ``constraints-1-10`` orphan branches.
-Those "known-to-be-working" constraints are per major/minor python version. You can use them as constraint
-files when installing Airflow from PyPI. Note that you have to specify correct Airflow version
-and python versions in the URL.
+Installation tools
+''''''''''''''''''
The official way of installing Airflow is with the ``pip`` tool.
There was a recent (November 2020) change in resolver, so currently only 20.2.4 version is officially
supported, although you might have a success with 20.3.3+ version (to be confirmed if all initial
-issues from ``pip`` 20.3.0 release have been fixed in 20.3.3).
+issues from ``pip`` 20.3.0 release have been fixed in 20.3.3). In order to install Airflow you need to
+either downgrade pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+``--use-deprecated legacy-resolver`` to your pip install command.
While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, but they do not share the same workflow as
-``pip``- especially when it comes to constraint vs. requirements management.
+``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported. If you wish to install airflow
-using those tools you should use the constraint files described below and convert them to appropriate
+using those tools you should use the :ref:`constraint files <installation:constraints>` and convert them to appropriate
format and workflow that your tool requires.
- **Prerequisites**
-
- On Debian based Linux OS:
Review comment:
This information looks out of date. We have a section on system requirements below where there are some more libraries. See: Requirements (old docs), System dependencies (old docs)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #13661: Structure and small improvements in installation.rst
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #13661:
URL: https://github.com/apache/airflow/pull/13661#discussion_r557010506
##########
File path: docs/apache-airflow/installation.rst
##########
@@ -38,117 +45,70 @@ Airflow is tested with:
* Kubernetes: 1.16.9, 1.17.5, 1.18.6
**Note:** MySQL 5.x versions are unable to or have limitations with
-running multiple schedulers -- please see the "Scheduler" docs. MariaDB is not tested/recommended.
+running multiple schedulers -- please see: :doc:`/scheduler`. MariaDB is not tested/recommended.
**Note:** SQLite is used in Airflow tests. Do not use it in production. We recommend
using the latest stable version of SQLite for local development.
-Getting Airflow
-'''''''''''''''
-
-Airflow is published as ``apache-airflow`` package in PyPI. Installing it however might be sometimes tricky
-because Airflow is a bit of both a library and application. Libraries usually keep their dependencies open and
-applications usually pin them, but we should do neither and both at the same time. We decided to keep
-our dependencies as open as possible (in ``setup.cfg`` and ``setup.py``) so users can install different
-version of libraries if needed. This means that from time to time plain ``pip install apache-airflow`` will
-not work or will produce unusable Airflow installation.
+Please note that with respect to Python 3 support, Airflow 2.0.0 has been
+tested with Python 3.6, 3.7, and 3.8, but does not yet support Python 3.9.
-In order to have repeatable installation, however, starting from **Airflow 1.10.10** and updated in
-**Airflow 1.10.13** we also keep a set of "known-to-be-working" constraint files in the
-``constraints-master``, ``constraints-2-0`` and ``constraints-1-10`` orphan branches.
-Those "known-to-be-working" constraints are per major/minor python version. You can use them as constraint
-files when installing Airflow from PyPI. Note that you have to specify correct Airflow version
-and python versions in the URL.
+Installation tools
+''''''''''''''''''
The official way of installing Airflow is with the ``pip`` tool.
There was a recent (November 2020) change in resolver, so currently only 20.2.4 version is officially
supported, although you might have a success with 20.3.3+ version (to be confirmed if all initial
-issues from ``pip`` 20.3.0 release have been fixed in 20.3.3).
+issues from ``pip`` 20.3.0 release have been fixed in 20.3.3). In order to install Airflow you need to
+either downgrade pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+``--use-deprecated legacy-resolver`` to your pip install command.
While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, but they do not share the same workflow as
-``pip``- especially when it comes to constraint vs. requirements management.
+``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported. If you wish to install airflow
-using those tools you should use the constraint files described below and convert them to appropriate
+using those tools you should use the :ref:`constraint files <installation:constraints>` and convert them to appropriate
format and workflow that your tool requires.
- **Prerequisites**
-
- On Debian based Linux OS:
Review comment:
This information looks out of date. We have a section on system requirements below where there are some more libraries. See: Requirements (old docs), System dependencies (new olds)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #13661: Structure and small improvements in installation.rst
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #13661:
URL: https://github.com/apache/airflow/pull/13661#discussion_r557012061
##########
File path: docs/apache-airflow/installation.rst
##########
@@ -171,44 +131,83 @@ to be needed for Linux system (Tested on Ubuntu Buster LTS) :
You also need database client packages (Postgres or MySQL) if you want to use those databases.
-If the ``airflow`` command is not getting recognized (can happen on Windows when using WSL), then
Review comment:
I moved that to the "troubleshooting" section because it didn't fit me here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #13661: Structure and small improvements in installation.rst
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #13661:
URL: https://github.com/apache/airflow/pull/13661#discussion_r557010506
##########
File path: docs/apache-airflow/installation.rst
##########
@@ -38,117 +45,70 @@ Airflow is tested with:
* Kubernetes: 1.16.9, 1.17.5, 1.18.6
**Note:** MySQL 5.x versions are unable to or have limitations with
-running multiple schedulers -- please see the "Scheduler" docs. MariaDB is not tested/recommended.
+running multiple schedulers -- please see: :doc:`/scheduler`. MariaDB is not tested/recommended.
**Note:** SQLite is used in Airflow tests. Do not use it in production. We recommend
using the latest stable version of SQLite for local development.
-Getting Airflow
-'''''''''''''''
-
-Airflow is published as ``apache-airflow`` package in PyPI. Installing it however might be sometimes tricky
-because Airflow is a bit of both a library and application. Libraries usually keep their dependencies open and
-applications usually pin them, but we should do neither and both at the same time. We decided to keep
-our dependencies as open as possible (in ``setup.cfg`` and ``setup.py``) so users can install different
-version of libraries if needed. This means that from time to time plain ``pip install apache-airflow`` will
-not work or will produce unusable Airflow installation.
+Please note that with respect to Python 3 support, Airflow 2.0.0 has been
+tested with Python 3.6, 3.7, and 3.8, but does not yet support Python 3.9.
-In order to have repeatable installation, however, starting from **Airflow 1.10.10** and updated in
-**Airflow 1.10.13** we also keep a set of "known-to-be-working" constraint files in the
-``constraints-master``, ``constraints-2-0`` and ``constraints-1-10`` orphan branches.
-Those "known-to-be-working" constraints are per major/minor python version. You can use them as constraint
-files when installing Airflow from PyPI. Note that you have to specify correct Airflow version
-and python versions in the URL.
+Installation tools
+''''''''''''''''''
The official way of installing Airflow is with the ``pip`` tool.
There was a recent (November 2020) change in resolver, so currently only 20.2.4 version is officially
supported, although you might have a success with 20.3.3+ version (to be confirmed if all initial
-issues from ``pip`` 20.3.0 release have been fixed in 20.3.3).
+issues from ``pip`` 20.3.0 release have been fixed in 20.3.3). In order to install Airflow you need to
+either downgrade pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+``--use-deprecated legacy-resolver`` to your pip install command.
While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, but they do not share the same workflow as
-``pip``- especially when it comes to constraint vs. requirements management.
+``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported. If you wish to install airflow
-using those tools you should use the constraint files described below and convert them to appropriate
+using those tools you should use the :ref:`constraint files <installation:constraints>` and convert them to appropriate
format and workflow that your tool requires.
- **Prerequisites**
-
- On Debian based Linux OS:
Review comment:
This information looks out of date. We have a section on system requirements below where there are some more libraries.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj merged pull request #13661: Structure and small content improvements in installation.rst
Posted by GitBox <gi...@apache.org>.
mik-laj merged pull request #13661:
URL: https://github.com/apache/airflow/pull/13661
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #13661: Structure and small improvements in installation.rst
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #13661:
URL: https://github.com/apache/airflow/pull/13661#discussion_r557011223
##########
File path: docs/apache-airflow/installation.rst
##########
@@ -38,117 +45,70 @@ Airflow is tested with:
* Kubernetes: 1.16.9, 1.17.5, 1.18.6
**Note:** MySQL 5.x versions are unable to or have limitations with
-running multiple schedulers -- please see the "Scheduler" docs. MariaDB is not tested/recommended.
+running multiple schedulers -- please see: :doc:`/scheduler`. MariaDB is not tested/recommended.
**Note:** SQLite is used in Airflow tests. Do not use it in production. We recommend
using the latest stable version of SQLite for local development.
-Getting Airflow
-'''''''''''''''
-
-Airflow is published as ``apache-airflow`` package in PyPI. Installing it however might be sometimes tricky
-because Airflow is a bit of both a library and application. Libraries usually keep their dependencies open and
-applications usually pin them, but we should do neither and both at the same time. We decided to keep
-our dependencies as open as possible (in ``setup.cfg`` and ``setup.py``) so users can install different
-version of libraries if needed. This means that from time to time plain ``pip install apache-airflow`` will
-not work or will produce unusable Airflow installation.
+Please note that with respect to Python 3 support, Airflow 2.0.0 has been
+tested with Python 3.6, 3.7, and 3.8, but does not yet support Python 3.9.
-In order to have repeatable installation, however, starting from **Airflow 1.10.10** and updated in
-**Airflow 1.10.13** we also keep a set of "known-to-be-working" constraint files in the
-``constraints-master``, ``constraints-2-0`` and ``constraints-1-10`` orphan branches.
-Those "known-to-be-working" constraints are per major/minor python version. You can use them as constraint
-files when installing Airflow from PyPI. Note that you have to specify correct Airflow version
-and python versions in the URL.
+Installation tools
+''''''''''''''''''
The official way of installing Airflow is with the ``pip`` tool.
There was a recent (November 2020) change in resolver, so currently only 20.2.4 version is officially
supported, although you might have a success with 20.3.3+ version (to be confirmed if all initial
-issues from ``pip`` 20.3.0 release have been fixed in 20.3.3).
+issues from ``pip`` 20.3.0 release have been fixed in 20.3.3). In order to install Airflow you need to
+either downgrade pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+``--use-deprecated legacy-resolver`` to your pip install command.
While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, but they do not share the same workflow as
-``pip``- especially when it comes to constraint vs. requirements management.
+``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported. If you wish to install airflow
-using those tools you should use the constraint files described below and convert them to appropriate
+using those tools you should use the :ref:`constraint files <installation:constraints>` and convert them to appropriate
format and workflow that your tool requires.
- **Prerequisites**
-
- On Debian based Linux OS:
-
- .. code-block:: bash
-
- sudo apt-get update
- sudo apt-get install build-essential
-
-
-1. Installing just Airflow
-
-.. note::
-
- On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
- does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
- of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
- ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
- ``--use-deprecated legacy-resolver`` to your pip install command.
Review comment:
This text was repeated several times. I understand the purpose of this, but now we have a new "Installation tools" section so this seems redundant.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13661: Structure and small content improvements in installation.rst
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13661:
URL: https://github.com/apache/airflow/pull/13661#issuecomment-759930973
The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest master or amend the last commit of the PR, and push it with --force-with-lease.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org