(airflow) branch main updated: Disable common.io as preinstalled provider before we cut-off 2.8 branch (#35313)

[po...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 905e2249a8 Disable common.io as preinstalled provider before we cut-off 2.8 branch (#35313)
905e2249a8 is described below

commit 905e2249a886d12dbe1252320d6d70a632908e5e
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Wed Nov 1 00:27:40 2023 +0100

    Disable common.io as preinstalled provider before we cut-off 2.8 branch (#35313)
    
    We need to do it in order to be able to generate PyPI constraints,
    because common.io has apache-airflow>=2.8.0 requirement that
    conflicts with the current 2.8.0dev0
---
 docs/apache-airflow/extra-packages-ref.rst  |  2 +-
 scripts/in_container/_in_container_utils.sh | 15 +++++----------
 setup.py                                    |  7 ++++---
 3 files changed, 10 insertions(+), 14 deletions(-)

diff --git a/docs/apache-airflow/extra-packages-ref.rst b/docs/apache-airflow/extra-packages-ref.rst
index b9637b1c14..dba750908d 100644
--- a/docs/apache-airflow/extra-packages-ref.rst
+++ b/docs/apache-airflow/extra-packages-ref.rst
@@ -285,7 +285,7 @@ These are extras that provide support for integration with external systems via
 +---------------------+-----------------------------------------------------+--------------------------------------+--------------+
 | extra               | install command                                     | enables                              | Preinstalled |
 +=====================+=====================================================+======================================+==============+
-| common.io           | ``pip install 'apache-airflow[common.io]'``         | Core IO Operators                    |      *       |
+| common.io           | ``pip install 'apache-airflow[common.io]'``         | Core IO Operators                    |              |
 +---------------------+-----------------------------------------------------+--------------------------------------+--------------+
 | common.sql          | ``pip install 'apache-airflow[common.sql]'``        | Core SQL Operators                   |      *       |
 +---------------------+-----------------------------------------------------+--------------------------------------+--------------+
diff --git a/scripts/in_container/_in_container_utils.sh b/scripts/in_container/_in_container_utils.sh
index 06ec060c90..2b1f44df9b 100644
--- a/scripts/in_container/_in_container_utils.sh
+++ b/scripts/in_container/_in_container_utils.sh
@@ -301,16 +301,11 @@ function install_all_providers_from_pypi_with_eager_upgrade() {
     local res
     for provider_package in ${ALL_PROVIDERS_PACKAGES}
     do
-        # Until we release "yandex" provider with protobuf support we need to remove it from the list of providers
-        # to install, because it is impossible to find common requirements for already released yandex provider
-        # and current airflow
-        if [[ ${provider_package} == "apache-airflow-providers-yandex" ]]; then
-            continue
-        fi
-        # Until we release latest `hive` provider with pure-sasl support, we need to remove it from the
-        # list of providers to install for Python 3.11 because we cannot build sasl it for Python 3.11
-        if [[ ${provider_package} == "apache-airflow-providers-apache-hive" \
-            && ${PYTHON_MAJOR_MINOR_VERSION} == "3.11" ]]; then
+        # Remove common.io provider in main branch until we cut-off v2-8-test branch and change
+        # version in main to 2.9.0 - otherwise we won't be able to generate PyPI constraints as
+        # released common-io provider has apache-airflow>2.8.0 as dependency and we cannot install
+        # the provider from PyPI
+        if [[ ${provider_package} == "apache-airflow-providers-common-io" ]]; then
             continue
         fi
         echo -n "Checking if ${provider_package} is available in PyPI: "
diff --git a/setup.py b/setup.py
index 8beea19f7d..1c1103f5e9 100644
--- a/setup.py
+++ b/setup.py
@@ -811,10 +811,11 @@ def sort_extras_dependencies() -> dict[str, list[str]]:
 EXTRAS_DEPENDENCIES = sort_extras_dependencies()
 
 # Those providers are pre-installed always when airflow is installed.
-# Those providers do not have dependency on airflow2.0 because that would lead to circular dependencies.
-# This is not a problem for PIP but some tools (pipdeptree) show those as a warning.
 PREINSTALLED_PROVIDERS = [
-    "common.io",
+    #   Until we cut-off the 2.8.0 branch and bump current airflow version to 2.9.0, we should
+    #   Keep common.io commented out in order ot be able to generate PyPI constraints because
+    #   The version from PyPI has requirement of apache-airflow>=2.8.0
+    #   "common.io",
     "common.sql",
     "ftp",
     "http",