You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "KesharwaniArpita (via GitHub)" <gi...@apache.org> on 2023/10/02 09:59:44 UTC
[PR] Created INSTALL.md and working to improve it [airflow]
KesharwaniArpita opened a new pull request, #34714:
URL: https://github.com/apache/airflow/pull/34714
I have created INSTALL.md derived from INSTALL which I found lacked the extension and preview of the markdown. But I still find it's not well written and I wish to improve it. But I need to understand what exactly does it cover and what exactly needs to be included. If possible, does anyone like to propose some ideas?
ALso, do we need to remove the original "INSTALL" file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1343346552
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
Review Comment:
We might want to use code block here
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
Review Comment:
This can be applied to all the commands below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1869834417
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1347379368
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
+- [Optional] Run Apache RAT (release audit tool) to validate license headers
+- RAT docs here:
+ ```https://creadur.apache.org/rat/.```
+- Requires Java and Apache Rat:
+
+ java -jar apache-rat.jar -E ./.rat-excludes -d
+
+**[optional] Airflow pulls in quite a lot of dependencies in order
+to connect to other services. You might want to test or run Airflow
+from a virtual env to make sure those dependencies are separated
+from your system wide versions**
+
+ python3 -m venv PATH_TO_YOUR_VENV
+ source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] Building and installing by pip (preferred)
+ pip install .
+
+#### Or directly
+ python setup.py install
Review Comment:
Don’t do this please. This is not equivalent to `pip install .` and can lead to very bad things in edge cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1344476869
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
Review Comment:
I'll fix that for sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1347909286
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
+- [Optional] Run Apache RAT (release audit tool) to validate license headers
+- RAT docs here:
+ ```https://creadur.apache.org/rat/.```
+- Requires Java and Apache Rat:
+
+ java -jar apache-rat.jar -E ./.rat-excludes -d
+
+**[optional] Airflow pulls in quite a lot of dependencies in order
+to connect to other services. You might want to test or run Airflow
+from a virtual env to make sure those dependencies are separated
+from your system wide versions**
+
+ python3 -m venv PATH_TO_YOUR_VENV
+ source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] Building and installing by pip (preferred)
+ pip install .
+
+#### Or directly
+ python setup.py install
Review Comment:
Yes just do that please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1745416495
SO while running the pre=commits in the repo, this is what I get:
"Lint JSON Schema files with JSON Schema............................................Failed
- hook id: lint-json-schema
- exit code: 1"
And, there are several of these failures, I think I need to work on these. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1746248617
I am sure why is "merge" being shown here but didn't do it (atleast on purpose) and I fixed the code block issues I was getting with two lines. Kindly review it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1347443124
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
Review Comment:
Would it be wise removing this part? The instructions seems to be more focused on setting up development environments rather than production setups.
What are your thoughts about that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1742922072
I would love to. I am new to airflow and I am looking for various ways to contribute and I would really appreciate if you can guide me through the pr. Coming to the PR, how exactly can I do it? What I can understand that I would need to install, configure and run pre-commit, along with addressing the en-route issues. Correct me if I am wrong.
I also wanted to ask where can I find the .yaml file for pre-commit configuration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1347422935
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
+- [Optional] Run Apache RAT (release audit tool) to validate license headers
+- RAT docs here:
+ ```https://creadur.apache.org/rat/.```
+- Requires Java and Apache Rat:
+
+ java -jar apache-rat.jar -E ./.rat-excludes -d
+
+**[optional] Airflow pulls in quite a lot of dependencies in order
+to connect to other services. You might want to test or run Airflow
+from a virtual env to make sure those dependencies are separated
+from your system wide versions**
+
+ python3 -m venv PATH_TO_YOUR_VENV
+ source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] Building and installing by pip (preferred)
+ pip install .
+
+#### Or directly
+ python setup.py install
Review Comment:
Can you provide with a better alternative? I'll surely improve the file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1744036171
If we're to change this one, we probably would want to change https://github.com/apache/airflow/blob/main/provider_packages/INSTALL and https://github.com/apache/airflow/blob/main/chart/INSTALL ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1770947528
Just to clarify - I think we should **not** have separate INSTALL.md at all. It duplicates a lot of the documentation from https://airflow.apache.org/docs/apache-airflow/stable/installation/index.html specifically one from https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html. So rather than duplicating it in INSTALL.md, we should make at most redirection to that page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1375439767
##########
INSTALL.md:
##########
@@ -0,0 +1,236 @@
+# INSTALL / BUILD instructions for Apache Airflow
Review Comment:
We should delete this one. It's not needed once you redirected users to documentation in the `INSTALL` file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1357874912
##########
INSTALL.md:
##########
@@ -0,0 +1,236 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Optional] Fetch the tarball and untar the source move into the directory that was untarred.
+- [Optional] Run Apache RAT (release audit tool) to validate license headers
+- RAT docs here:
+ ```https://creadur.apache.org/rat/.```
+- Requires Java and Apache Rat:
+
+ java -jar apache-rat.jar -E ./.rat-excludes -d
+- [Required] Instead of fetching and untarring the source tarball, you can use `pip` to directly install Apache Airflow with a specified version.
+
+ pip install apache-airflow==<version>
+
+**[optional] Airflow pulls in quite a lot of dependencies in order
+to connect to other services. You might want to test or run Airflow
+from a virtual env to make sure those dependencies are separated
+from your system wide versions**
+
+ python3 -m venv PATH_TO_YOUR_VENV
+ source PATH_TO_YOUR_VENV/bin/activate
+
+**[Required] Building and installing by pip (preferred)**
+
+ pip install .
+
+
+**You can also install recommended version of the dependencies by using
+constraint- ```python<PYTHON_MAJOR_MINOR_VERSION>.txt``` files as constraint file. This is needed in case
+you have problems with installing the current requirements from PyPI.
+There are different constraint files for different python versions. For example:**
+
+ pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+By default `pip install` in Airflow 2.0 installs only the provider packages that are needed by the extras and
+install them as packages from PyPI rather than from local sources:
+
+ pip install .[google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+You can upgrade just airflow, without paying attention to provider's dependencies by using 'constraints-no-providers'
+constraint files. This allows you to keep installed provider packages.
+
+ pip install . --upgrade \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.8.txt"
+
+
+You can also install airflow in "editable mode" (with `-e`) flag and then provider packages are
+available directly from the sources (and the provider packages installed from `PyPI` are UNINSTALLED in
+order to avoid having providers in two places. And `provider.yaml` files are used to discover capabilities
+of the providers which are part of the airflow source code.
+
+You can read more about `provider.yaml` and community-managed providers in
+`https://airflow.apache.org/docs/apache-airflow-providers/index.html` for developing custom providers
+and in ``CONTRIBUTING.rst`` for developing community maintained providers.
+
+This is useful if you want to develop providers:
+
+ pip install -e . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+You can also skip installing provider packages from `PyPI` by setting `INSTALL_PROVIDERS_FROM_SOURCE` to "true".
+In this case Airflow will be installed in non-editable mode with all providers installed from the sources.
+Additionally `provider.yaml` files will also be copied to providers folders which will make the providers
+discoverable by Airflow even if they are not installed from packages in this case.
+
+ INSTALL_PROVIDERS_FROM_SOURCES="true" pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+Airflow can be installed with extras to install some additional features (for example 'async' or 'doc' or
+to install automatically providers and all dependencies needed by that provider:
+
+ pip install .[async,google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+The list of available extras:
Review Comment:
Are extras still recommended for deployment? IIRC they are mostly kept only for compatibility, and for an actual production installation it is recommended to install the actual providers you want instead, which would pull in the dependencies themselves.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1347379368
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
+- [Optional] Run Apache RAT (release audit tool) to validate license headers
+- RAT docs here:
+ ```https://creadur.apache.org/rat/.```
+- Requires Java and Apache Rat:
+
+ java -jar apache-rat.jar -E ./.rat-excludes -d
+
+**[optional] Airflow pulls in quite a lot of dependencies in order
+to connect to other services. You might want to test or run Airflow
+from a virtual env to make sure those dependencies are separated
+from your system wide versions**
+
+ python3 -m venv PATH_TO_YOUR_VENV
+ source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] Building and installing by pip (preferred)
+ pip install .
+
+#### Or directly
+ python setup.py install
Review Comment:
Don’t do this please
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
+- [Optional] Run Apache RAT (release audit tool) to validate license headers
+- RAT docs here:
+ ```https://creadur.apache.org/rat/.```
+- Requires Java and Apache Rat:
+
+ java -jar apache-rat.jar -E ./.rat-excludes -d
+
+**[optional] Airflow pulls in quite a lot of dependencies in order
+to connect to other services. You might want to test or run Airflow
+from a virtual env to make sure those dependencies are separated
+from your system wide versions**
+
+ python3 -m venv PATH_TO_YOUR_VENV
+ source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] Building and installing by pip (preferred)
+ pip install .
+
+#### Or directly
+ python setup.py install
Review Comment:
Don’t do this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1347385103
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
Review Comment:
If you don’t want to run RAT, you can do `pip install apache-airflow==<version>` and avoid the tarball entirely. That makes the following instructions a bit complicated though.
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
Review Comment:
If you don’t want to run RAT, you can do `pip install apache-airflow==<version>` and avoid the tarball entirely. That makes the following instructions a bit complicated though.
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
Review Comment:
If you don’t want to run RAT, you can do `pip install apache-airflow==<version>` and avoid the tarball entirely. That makes all the next instructions a bit complicated though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "frankcash (via GitHub)" <gi...@apache.org>.
frankcash commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1749550086
Isn't this kind of defeated w/ the fact most people are running airflow w/ Docker and a lot of this is unnecessary w/ a docker pull?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1748933849
> It’s quite unclear to me what audience this document intends to target. Is it for people who want to deploy Airflow to production? Is it for people who want to build Airflow locally for some other purposes? (e.g. redistribution via a Docker image or something else) Something different entirely? The instruction seems to cover a hodgepodge of scenarios.
@uranusjr , Thank you for your feedback. I agree that the document currently covers a variety of scenarios, and it's not clear who the intended audience is. I'm actively working on improving the clarity and focus of this document. However, I must admit that I have limited experience in this area, so I would greatly appreciate the guidance and insights of the maintainers to ensure that the document serves its intended purpose effectively. Your expertise would be invaluable in helping me refine the content to better meet the needs of our users. Any suggestions or direction you could provide would be highly appreciated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1746122282
@KesharwaniArpita when you have the pre commit installed, and you try to commit it, the pre commit fixes most files and you should have it in your unstaged changes. Check using `git status`.
Some cannot be fixed by pre commit. To handle those, check logs here https://github.com/apache/airflow/actions/runs/6378645274/job/17309685834?pr=34714 and try to run the same commands locally. You will repro the issue and then work on a fix!
Ping here if you run into issues that you need help with :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1763453451
sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1770557024
I have added the recommended header in the doc, kindly review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #34714: Created INSTALL.md and working to improve it
URL: https://github.com/apache/airflow/pull/34714
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1742926579
As explained in the mentioned doc, first you need to activate the pre-commit:
```
pre-commit install
```
Then you can run
```
pre-commit run --all-files
```
to rerun the hooks on all the files including the committed changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1347422935
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
+- [Optional] Run Apache RAT (release audit tool) to validate license headers
+- RAT docs here:
+ ```https://creadur.apache.org/rat/.```
+- Requires Java and Apache Rat:
+
+ java -jar apache-rat.jar -E ./.rat-excludes -d
+
+**[optional] Airflow pulls in quite a lot of dependencies in order
+to connect to other services. You might want to test or run Airflow
+from a virtual env to make sure those dependencies are separated
+from your system wide versions**
+
+ python3 -m venv PATH_TO_YOUR_VENV
+ source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] Building and installing by pip (preferred)
+ pip install .
+
+#### Or directly
+ python setup.py install
Review Comment:
Can you provide with a better alternative? I'll surely improve the file.
Ummm, I think we can go with `pip install .`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1347381760
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
Review Comment:
Is this a guide to install Airflow for development? I don’t think we support installing Airflow for production on anywhere but Linux. And you should use Breeze for development. So this part does not make sense.
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
Review Comment:
Is this a guide to install Airflow for development? I don’t think we support installing Airflow for production on anywhere but Linux. And you should use Breeze for development. So this part does not make sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1748859252
It’s quite unclear to me what audience this document intends to target. Is it for people who want to deploy Airflow to production? Is it for people who want to build Airflow locally for some other purposes? (e.g. redistribution via a Docker image or something else) Something different entirely? The instruction seems to cover a hodgepodge of scenarios.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1347442907
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
+
+- [Required] Fetch the tarball and untar the source move into the directory that was untarred.
Review Comment:
I have made certain changes regarding this. Kindly Review.
##########
INSTALL.md:
##########
@@ -0,0 +1,233 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+ sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of ```mariadb_config not found```
+and mysql_config not found.
+
+Install MariaDB development headers:
+
+ sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+ brew install sqlite mysql postgresql
Review Comment:
Would it be wise removing this part?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #34714:
URL: https://github.com/apache/airflow/pull/34714#discussion_r1345182796
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
+python setup.py install
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
+python setup.py install
+
+### You can also install recommended version of the dependencies by using
+### constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case
+### you have problems with installing the current requirements from PyPI.
+### There are different constraint files for different python versions. For example"
Review Comment:
Need not be a heading. Only bold is enough
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
+python setup.py install
+
+### You can also install recommended version of the dependencies by using
+### constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case
+### you have problems with installing the current requirements from PyPI.
+### There are different constraint files for different python versions. For example"
+
+pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+By default `pip install` in Airflow 2.0 installs only the provider packages that are needed by the extras and
+install them as packages from PyPI rather than from local sources:
+
+pip install .[google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+You can upgrade just airflow, without paying attention to provider's dependencies by using 'constraints-no-providers'
+constraint files. This allows you to keep installed provider packages.
+
+pip install . --upgrade \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.8.txt"
+
+
+You can also install airflow in "editable mode" (with -e) flag and then provider packages are
+available directly from the sources (and the provider packages installed from PyPI are UNINSTALLED in
+order to avoid having providers in two places. And `provider.yaml` files are used to discover capabilities
+of the providers which are part of the airflow source code.
+
+You can read more about `provider.yaml` and community-managed providers in
+https://airflow.apache.org/docs/apache-airflow-providers/index.html for developing custom providers
+and in ``CONTRIBUTING.rst`` for developing community maintained providers.
+
+This is useful if you want to develop providers:
+
+pip install -e . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
Review Comment:
This need not be a heading. Just a bold text is fine
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
+python setup.py install
+
+### You can also install recommended version of the dependencies by using
+### constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case
+### you have problems with installing the current requirements from PyPI.
+### There are different constraint files for different python versions. For example"
+
+pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+By default `pip install` in Airflow 2.0 installs only the provider packages that are needed by the extras and
+install them as packages from PyPI rather than from local sources:
+
+pip install .[google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
+python setup.py install
+
+### You can also install recommended version of the dependencies by using
+### constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case
+### you have problems with installing the current requirements from PyPI.
+### There are different constraint files for different python versions. For example"
+
+pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
Review Comment:
These dont need to be headings
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
Review Comment:
Same comment as above
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
Review Comment:
+1 move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
+python setup.py install
+
+### You can also install recommended version of the dependencies by using
+### constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case
+### you have problems with installing the current requirements from PyPI.
+### There are different constraint files for different python versions. For example"
+
+pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+By default `pip install` in Airflow 2.0 installs only the provider packages that are needed by the extras and
+install them as packages from PyPI rather than from local sources:
+
+pip install .[google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+You can upgrade just airflow, without paying attention to provider's dependencies by using 'constraints-no-providers'
+constraint files. This allows you to keep installed provider packages.
+
+pip install . --upgrade \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.8.txt"
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
+python setup.py install
+
+### You can also install recommended version of the dependencies by using
+### constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case
+### you have problems with installing the current requirements from PyPI.
+### There are different constraint files for different python versions. For example"
+
+pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+By default `pip install` in Airflow 2.0 installs only the provider packages that are needed by the extras and
+install them as packages from PyPI rather than from local sources:
+
+pip install .[google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+You can upgrade just airflow, without paying attention to provider's dependencies by using 'constraints-no-providers'
+constraint files. This allows you to keep installed provider packages.
+
+pip install . --upgrade \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.8.txt"
+
+
+You can also install airflow in "editable mode" (with -e) flag and then provider packages are
+available directly from the sources (and the provider packages installed from PyPI are UNINSTALLED in
+order to avoid having providers in two places. And `provider.yaml` files are used to discover capabilities
+of the providers which are part of the airflow source code.
+
+You can read more about `provider.yaml` and community-managed providers in
+https://airflow.apache.org/docs/apache-airflow-providers/index.html for developing custom providers
+and in ``CONTRIBUTING.rst`` for developing community maintained providers.
+
+This is useful if you want to develop providers:
+
+pip install -e . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+You can also skip installing provider packages from PyPI by setting INSTALL_PROVIDERS_FROM_SOURCE to "true".
+In this case Airflow will be installed in non-editable mode with all providers installed from the sources.
+Additionally `provider.yaml` files will also be copied to providers folders which will make the providers
+discoverable by Airflow even if they are not installed from packages in this case.
+
+INSTALL_PROVIDERS_FROM_SOURCES="true" pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+Airflow can be installed with extras to install some additional features (for example 'async' or 'doc' or
+to install automatically providers and all dependencies needed by that provider:
+
+pip install .[async,google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+The list of available extras:
+
+### START EXTRAS HERE
+aiobotocore, airbyte, alibaba, all, all_dbs, amazon, apache.atlas, apache.beam, apache.cassandra,
+apache.drill, apache.druid, apache.flink, apache.hdfs, apache.hive, apache.impala, apache.kafka,
+apache.kylin, apache.livy, apache.pig, apache.pinot, apache.spark, apache.sqoop, apache.webhdfs,
+apprise, arangodb, asana, async, atlas, atlassian.jira, aws, azure, cassandra, celery, cgroups,
+cloudant, cncf.kubernetes, common.sql, crypto, dask, daskexecutor, databricks, datadog, dbt.cloud,
+deprecated_api, devel, devel_all, devel_ci, devel_hadoop, dingding, discord, doc, doc_gen, docker,
+druid, elasticsearch, exasol, facebook, ftp, gcp, gcp_api, github, github_enterprise, google,
+google_auth, grpc, hashicorp, hdfs, hive, http, imap, influxdb, jdbc, jenkins, kerberos, kubernetes,
+ldap, leveldb, microsoft.azure, microsoft.mssql, microsoft.psrp, microsoft.winrm, mongo, mssql,
+mysql, neo4j, odbc, openfaas, openlineage, opsgenie, oracle, otel, pagerduty, pandas, papermill,
+password, pinot, plexus, postgres, presto, rabbitmq, redis, s3, salesforce, samba, segment,
+sendgrid, sentry, sftp, singularity, slack, smtp, snowflake, spark, sqlite, ssh, statsd, tableau,
+tabular, telegram, trino, vertica, virtualenv, webhdfs, winrm, yandex, zendesk
+### END EXTRAS HERE
+
+### For installing Airflow in development environments - see CONTRIBUTING.rst
+
+### COMPILING FRONT-END ASSETS (in case you see "Please make sure to build the frontend in static/ directory and then restart the server")
+### Optional : Installing yarn - https://classic.yarnpkg.com/en/docs/install
+
+python setup.py compile_assets
Review Comment:
Move to a code block
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
+python setup.py install
+
+### You can also install recommended version of the dependencies by using
+### constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case
+### you have problems with installing the current requirements from PyPI.
+### There are different constraint files for different python versions. For example"
+
+pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+By default `pip install` in Airflow 2.0 installs only the provider packages that are needed by the extras and
+install them as packages from PyPI rather than from local sources:
+
+pip install .[google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+You can upgrade just airflow, without paying attention to provider's dependencies by using 'constraints-no-providers'
+constraint files. This allows you to keep installed provider packages.
+
+pip install . --upgrade \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.8.txt"
+
+
+You can also install airflow in "editable mode" (with -e) flag and then provider packages are
+available directly from the sources (and the provider packages installed from PyPI are UNINSTALLED in
+order to avoid having providers in two places. And `provider.yaml` files are used to discover capabilities
+of the providers which are part of the airflow source code.
+
+You can read more about `provider.yaml` and community-managed providers in
+https://airflow.apache.org/docs/apache-airflow-providers/index.html for developing custom providers
+and in ``CONTRIBUTING.rst`` for developing community maintained providers.
+
+This is useful if you want to develop providers:
+
+pip install -e . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+You can also skip installing provider packages from PyPI by setting INSTALL_PROVIDERS_FROM_SOURCE to "true".
+In this case Airflow will be installed in non-editable mode with all providers installed from the sources.
+Additionally `provider.yaml` files will also be copied to providers folders which will make the providers
+discoverable by Airflow even if they are not installed from packages in this case.
+
+INSTALL_PROVIDERS_FROM_SOURCES="true" pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+Airflow can be installed with extras to install some additional features (for example 'async' or 'doc' or
+to install automatically providers and all dependencies needed by that provider:
+
+pip install .[async,google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+The list of available extras:
+
+### START EXTRAS HERE
+aiobotocore, airbyte, alibaba, all, all_dbs, amazon, apache.atlas, apache.beam, apache.cassandra,
+apache.drill, apache.druid, apache.flink, apache.hdfs, apache.hive, apache.impala, apache.kafka,
+apache.kylin, apache.livy, apache.pig, apache.pinot, apache.spark, apache.sqoop, apache.webhdfs,
+apprise, arangodb, asana, async, atlas, atlassian.jira, aws, azure, cassandra, celery, cgroups,
+cloudant, cncf.kubernetes, common.sql, crypto, dask, daskexecutor, databricks, datadog, dbt.cloud,
+deprecated_api, devel, devel_all, devel_ci, devel_hadoop, dingding, discord, doc, doc_gen, docker,
+druid, elasticsearch, exasol, facebook, ftp, gcp, gcp_api, github, github_enterprise, google,
+google_auth, grpc, hashicorp, hdfs, hive, http, imap, influxdb, jdbc, jenkins, kerberos, kubernetes,
+ldap, leveldb, microsoft.azure, microsoft.mssql, microsoft.psrp, microsoft.winrm, mongo, mssql,
+mysql, neo4j, odbc, openfaas, openlineage, opsgenie, oracle, otel, pagerduty, pandas, papermill,
+password, pinot, plexus, postgres, presto, rabbitmq, redis, s3, salesforce, samba, segment,
+sendgrid, sentry, sftp, singularity, slack, smtp, snowflake, spark, sqlite, ssh, statsd, tableau,
+tabular, telegram, trino, vertica, virtualenv, webhdfs, winrm, yandex, zendesk
+### END EXTRAS HERE
+
+### For installing Airflow in development environments - see CONTRIBUTING.rst
Review Comment:
Need not be a heading. Just bold is enough
##########
INSTALL.md:
##########
@@ -0,0 +1,117 @@
+# INSTALL / BUILD instructions for Apache Airflow
+
+This is a generic installation method that requires a number of dependencies to be installed.
+
+Depending on your system you might need different prerequisites, but the following
+systems/prerequisites are known to work:
+
+Linux (Debian Bullseye and Linux Mint Debbie):
+
+sudo apt install build-essential python3-dev libsqlite3-dev openssl \
+ sqlite default-libmysqlclient-dev libmysqlclient-dev postgresql
+
+On Ubuntu 20.04 you may get an error of mariadb_config not found
+and mysql_config not found.
+
+Install MariaDB development headers:
+sudo apt-get install libmariadb-dev libmariadbclient-dev
+
+MacOS (Mojave/Catalina):
+
+brew install sqlite mysql postgresql
+
+### [required] fetch the tarball and untar the source move into the directory that was untarred.
+
+### [optional] run Apache RAT (release audit tool) to validate license headers
+### RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
+java -jar apache-rat.jar -E ./.rat-excludes -d .
+
+### [optional] Airflow pulls in quite a lot of dependencies in order
+### to connect to other services. You might want to test or run Airflow
+### from a virtual env to make sure those dependencies are separated
+### from your system wide versions
+
+python3 -m venv PATH_TO_YOUR_VENV
+source PATH_TO_YOUR_VENV/bin/activate
+
+### [required] building and installing by pip (preferred)
+pip install .
+
+### or directly
+python setup.py install
+
+### You can also install recommended version of the dependencies by using
+### constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case
+### you have problems with installing the current requirements from PyPI.
+### There are different constraint files for different python versions. For example"
+
+pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+By default `pip install` in Airflow 2.0 installs only the provider packages that are needed by the extras and
+install them as packages from PyPI rather than from local sources:
+
+pip install .[google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+
+You can upgrade just airflow, without paying attention to provider's dependencies by using 'constraints-no-providers'
+constraint files. This allows you to keep installed provider packages.
+
+pip install . --upgrade \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.8.txt"
+
+
+You can also install airflow in "editable mode" (with -e) flag and then provider packages are
+available directly from the sources (and the provider packages installed from PyPI are UNINSTALLED in
+order to avoid having providers in two places. And `provider.yaml` files are used to discover capabilities
+of the providers which are part of the airflow source code.
+
+You can read more about `provider.yaml` and community-managed providers in
+https://airflow.apache.org/docs/apache-airflow-providers/index.html for developing custom providers
+and in ``CONTRIBUTING.rst`` for developing community maintained providers.
+
+This is useful if you want to develop providers:
+
+pip install -e . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+You can also skip installing provider packages from PyPI by setting INSTALL_PROVIDERS_FROM_SOURCE to "true".
+In this case Airflow will be installed in non-editable mode with all providers installed from the sources.
+Additionally `provider.yaml` files will also be copied to providers folders which will make the providers
+discoverable by Airflow even if they are not installed from packages in this case.
+
+INSTALL_PROVIDERS_FROM_SOURCES="true" pip install . \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+Airflow can be installed with extras to install some additional features (for example 'async' or 'doc' or
+to install automatically providers and all dependencies needed by that provider:
+
+pip install .[async,google,amazon] \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.8.txt"
+
+The list of available extras:
+
+### START EXTRAS HERE
+aiobotocore, airbyte, alibaba, all, all_dbs, amazon, apache.atlas, apache.beam, apache.cassandra,
+apache.drill, apache.druid, apache.flink, apache.hdfs, apache.hive, apache.impala, apache.kafka,
+apache.kylin, apache.livy, apache.pig, apache.pinot, apache.spark, apache.sqoop, apache.webhdfs,
+apprise, arangodb, asana, async, atlas, atlassian.jira, aws, azure, cassandra, celery, cgroups,
+cloudant, cncf.kubernetes, common.sql, crypto, dask, daskexecutor, databricks, datadog, dbt.cloud,
+deprecated_api, devel, devel_all, devel_ci, devel_hadoop, dingding, discord, doc, doc_gen, docker,
+druid, elasticsearch, exasol, facebook, ftp, gcp, gcp_api, github, github_enterprise, google,
+google_auth, grpc, hashicorp, hdfs, hive, http, imap, influxdb, jdbc, jenkins, kerberos, kubernetes,
+ldap, leveldb, microsoft.azure, microsoft.mssql, microsoft.psrp, microsoft.winrm, mongo, mssql,
+mysql, neo4j, odbc, openfaas, openlineage, opsgenie, oracle, otel, pagerduty, pandas, papermill,
+password, pinot, plexus, postgres, presto, rabbitmq, redis, s3, salesforce, samba, segment,
+sendgrid, sentry, sftp, singularity, slack, smtp, snowflake, spark, sqlite, ssh, statsd, tableau,
+tabular, telegram, trino, vertica, virtualenv, webhdfs, winrm, yandex, zendesk
+### END EXTRAS HERE
+
+### For installing Airflow in development environments - see CONTRIBUTING.rst
+
+### COMPILING FRONT-END ASSETS (in case you see "Please make sure to build the frontend in static/ directory and then restart the server")
+### Optional : Installing yarn - https://classic.yarnpkg.com/en/docs/install
Review Comment:
Same comment as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Created INSTALL.md and working to improve it [airflow]
Posted by "KesharwaniArpita (via GitHub)" <gi...@apache.org>.
KesharwaniArpita commented on PR #34714:
URL: https://github.com/apache/airflow/pull/34714#issuecomment-1746252882
@amoghrajesh what do you mean by "You will repro the issue and then work on a fix!"? Sorry, I didn't understand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org