You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2025/07/11 17:15:47 UTC
(airflow) branch v3-0-test updated (b57ccf0f829 -> 61f39160b62)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit b57ccf0f829 [v3-0-test] Fix reading huge (XCom) resposne in TaskSDK task process (#53186) (#53194)
omit 62c83d11daf [v3-0-test] Dont use builtin len as variable (#51992) (#53193)
omit 8ccc2f93ac5 [v3-0-test] Move --no-binary flag to uv command line from pyproject.toml (#53179) (#53185)
omit 220fb39d603 [v3-0-test] Unify connection not found exceptions between AF2 and AF3 (#52968) (#53093)
omit 293a383c909 Run Task failure callbacks on DAG Processor when task is externally killed (#53058) (#53143)
omit 08223283e24 [v3-0-test] docs: update public interface doc to reflect airflow.sdk and AIP-72 (#52197) (#53117)
omit 19b81f61a87 [v3-0-test] Remove remnants of ~= used in requires-python. (#52985) (#52987)
omit 5fd84d9531e Remove upper-binding for "python-requires" (#52980) (#52984)
omit dcae57d6fe9 [v3-0-test] Make the update_installer_and_pre_commit script more resilient to re matches (#52960) (#52961)
omit 3323f43d1a0 [v3-0-test] Workaround the need to use latest version on Lucas pre-commit's (#52931) (#52942)
omit 088410e0c23 [v3-0-test] Advanced auto classification for provider documentation (#52902) (#52925)
omit 369ad9bcad0 [Backport v3-0-test] Add back dag parsing pre-import optimization (#50371) (#52698)
new 5b81b39b12a [v3-0-test] Dont use builtin len as variable (#51992) (#53193)
new d159feef80d [v3-0-test] Fix reading huge (XCom) resposne in TaskSDK task process (#53186) (#53194)
new 125e2e5f1dc [Backport v3-0-test] Add back dag parsing pre-import optimization (#50371) (#52698)
new aea7296ae2f [v3-0-test] Advanced auto classification for provider documentation (#52902) (#52925)
new 1edbd2dd3d4 [v3-0-test] Workaround the need to use latest version on Lucas pre-commit's (#52931) (#52942)
new b494abdc875 [v3-0-test] Make the update_installer_and_pre_commit script more resilient to re matches (#52960) (#52961)
new 1b87286f24a Remove upper-binding for "python-requires" (#52980) (#52984)
new 6e394358cd9 [v3-0-test] Remove remnants of ~= used in requires-python. (#52985) (#52987)
new 2af039b4f8c [v3-0-test] docs: update public interface doc to reflect airflow.sdk and AIP-72 (#52197) (#53117)
new 9359b88bc9c Run Task failure callbacks on DAG Processor when task is externally killed (#53058) (#53143)
new 33cf4223a09 [v3-0-test] Unify connection not found exceptions between AF2 and AF3 (#52968) (#53093)
new 61f39160b62 [v3-0-test] Move --no-binary flag to uv command line from pyproject.toml (#53179) (#53185)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b57ccf0f829)
\
N -- N -- N refs/heads/v3-0-test (61f39160b62)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 12 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
(airflow) 09/12: [v3-0-test] docs: update public interface doc to reflect airflow.sdk and AIP-72 (#52197) (#53117)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 2af039b4f8ca384b3d96c26d2c22088c4a5364ce
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Thu Jul 10 16:27:55 2025 +0200
[v3-0-test] docs: update public interface doc to reflect airflow.sdk and AIP-72 (#52197) (#53117)
* docs: update public interface doc to reflect airflow.sdk and AIP-72
- Added a note under "Using Airflow Public Interfaces" to recommend using `airflow.sdk` as the official interface from Airflow 3.0.
- Referenced AIP-72 and linked related documentation.
- Encouraged users to prefer REST API and Python Client for integrations.
* Update airflow-core/docs/public-airflow-interface.rst
Great
---------
(cherry picked from commit e142ab96a0ecbb953e9fb8ab6d17b4c2a7624aba)
Co-authored-by: N R Navaneet <15...@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <ja...@potiuk.com>
---
airflow-core/docs/public-airflow-interface.rst | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/airflow-core/docs/public-airflow-interface.rst b/airflow-core/docs/public-airflow-interface.rst
index b0b7cfe5af2..0f685c16cce 100644
--- a/airflow-core/docs/public-airflow-interface.rst
+++ b/airflow-core/docs/public-airflow-interface.rst
@@ -47,6 +47,21 @@ For comprehensive Task SDK documentation, see the `Task SDK Reference <https://a
Using Airflow Public Interfaces
===============================
+.. note::
+
+ As of **Airflow 3.0**, users should use the ``airflow.sdk`` namespace as the official **Public Interface**, as defined in `AIP-72 <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-72+Task+Execution+Interface+aka+Task+SDK>`_.
+
+ Direct interaction with internal modules or the metadata database is not possible.
+ For stable, production-safe integration, it is recommended to use:
+
+ - The official **REST API**
+ - The **Python Client SDK** (`airflow-client-python`)
+ - The new **Task SDK** (``airflow.sdk``)
+
+ Related docs:
+ - `Release Notes 3.0 <https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html>`_
+ - `Task SDK Overview <https://airflow.apache.org/docs/apache-airflow/stable/concepts/taskflow.html>`_
+
The following are some examples of the public interface of Airflow:
* When you are writing your own operators or hooks. This is commonly done when no hook or operator exists for your use case, or when perhaps when one exists but you need to customize the behavior.
(airflow) 03/12: [Backport v3-0-test] Add back dag parsing pre-import optimization (#50371) (#52698)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 125e2e5f1dcc8312f7a541c2816b2d948522597d
Author: Kevin Liu <58...@users.noreply.github.com>
AuthorDate: Fri Jul 4 16:25:27 2025 +0800
[Backport v3-0-test] Add back dag parsing pre-import optimization (#50371) (#52698)
* Add back dag parsing pre-import optimization (#50371)
Co-authored-by: Tzu-ping Chung <ur...@gmail.com>
* Apply suggestions from code review
---------
Co-authored-by: Tzu-ping Chung <ur...@gmail.com>
Co-authored-by: Kaxil Naik <ka...@gmail.com>
---
.../src/airflow/config_templates/config.yml | 20 ++---
airflow-core/src/airflow/configuration.py | 1 +
.../src/airflow/dag_processing/processor.py | 28 +++++++
.../tests/unit/dag_processing/test_processor.py | 85 +++++++++++++++++++++-
4 files changed, 122 insertions(+), 12 deletions(-)
diff --git a/airflow-core/src/airflow/config_templates/config.yml b/airflow-core/src/airflow/config_templates/config.yml
index 92134fa4bf1..503084e13a9 100644
--- a/airflow-core/src/airflow/config_templates/config.yml
+++ b/airflow-core/src/airflow/config_templates/config.yml
@@ -2199,16 +2199,6 @@ scheduler:
type: integer
default: "20"
see_also: ":ref:`scheduler:ha:tunables`"
- parsing_pre_import_modules:
- description: |
- The scheduler reads dag files to extract the airflow modules that are going to be used,
- and imports them ahead of time to avoid having to re-do it for each parsing process.
- This flag can be set to ``False`` to disable this behavior in case an airflow module needs
- to be freshly imported each time (at the cost of increased DAG parsing time).
- version_added: 2.6.0
- type: boolean
- example: ~
- default: "True"
dag_stale_not_seen_duration:
description: |
Time in seconds after which dags, which were not updated by Dag Processor are deactivated.
@@ -2563,3 +2553,13 @@ dag_processor:
type: integer
example: ~
default: "10"
+ parsing_pre_import_modules:
+ description: |
+ The dag_processor reads dag files to extract the airflow modules that are going to be used,
+ and imports them ahead of time to avoid having to re-do it for each parsing process.
+ This flag can be set to ``False`` to disable this behavior in case an airflow module needs
+ to be freshly imported each time (at the cost of increased DAG parsing time).
+ version_added: 2.6.0
+ type: boolean
+ example: ~
+ default: "True"
diff --git a/airflow-core/src/airflow/configuration.py b/airflow-core/src/airflow/configuration.py
index 2bf3df98163..5fd67f53343 100644
--- a/airflow-core/src/airflow/configuration.py
+++ b/airflow-core/src/airflow/configuration.py
@@ -364,6 +364,7 @@ class AirflowConfigParser(ConfigParser):
("fab", "navbar_text_hover_color"): ("webserver", "navbar_text_hover_color", "3.0.2"),
("api", "secret_key"): ("webserver", "secret_key", "3.0.2"),
("api", "enable_swagger_ui"): ("webserver", "enable_swagger_ui", "3.0.2"),
+ ("dag_processor", "parsing_pre_import_modules"): ("scheduler", "parsing_pre_import_modules", "3.0.3"),
}
# A mapping of new section -> (old section, since_version).
diff --git a/airflow-core/src/airflow/dag_processing/processor.py b/airflow-core/src/airflow/dag_processing/processor.py
index 011393f22c8..a54c56ddbe2 100644
--- a/airflow-core/src/airflow/dag_processing/processor.py
+++ b/airflow-core/src/airflow/dag_processing/processor.py
@@ -16,6 +16,7 @@
# under the License.
from __future__ import annotations
+import importlib
import os
import sys
import traceback
@@ -45,6 +46,7 @@ from airflow.sdk.execution_time.comms import (
from airflow.sdk.execution_time.supervisor import WatchedSubprocess
from airflow.serialization.serialized_objects import LazyDeserializedDAG, SerializedDAG
from airflow.stats import Stats
+from airflow.utils.file import iter_airflow_imports
if TYPE_CHECKING:
from structlog.typing import FilteringBoundLogger
@@ -98,6 +100,27 @@ ToDagProcessor = Annotated[
]
+def _pre_import_airflow_modules(file_path: str, log: FilteringBoundLogger) -> None:
+ """
+ Pre-import Airflow modules found in the given file.
+
+ This prevents modules from being re-imported in each processing process,
+ saving CPU time and memory.
+ (The default value of "parsing_pre_import_modules" is set to True)
+
+ :param file_path: Path to the file to scan for imports
+ :param log: Logger instance to use for warnings
+ """
+ if not conf.getboolean("dag_processor", "parsing_pre_import_modules", fallback=True):
+ return
+
+ for module in iter_airflow_imports(file_path):
+ try:
+ importlib.import_module(module)
+ except ModuleNotFoundError as e:
+ log.warning("Error when trying to pre-import module '%s' found in %s: %s", module, file_path, e)
+
+
def _parse_file_entrypoint():
import structlog
@@ -127,6 +150,7 @@ def _parse_file_entrypoint():
def _parse_file(msg: DagFileParseRequest, log: FilteringBoundLogger) -> DagFileParsingResult | None:
# TODO: Set known_pool names on DagBag!
+
bag = DagBag(
dag_folder=msg.file,
bundle_path=msg.bundle_path,
@@ -250,6 +274,10 @@ class DagFileProcessorProcess(WatchedSubprocess):
client: Client,
**kwargs,
) -> Self:
+ logger = kwargs["logger"]
+
+ _pre_import_airflow_modules(os.fspath(path), logger)
+
proc: Self = super().start(target=target, client=client, **kwargs)
proc._on_child_started(callbacks, path, bundle_path)
return proc
diff --git a/airflow-core/tests/unit/dag_processing/test_processor.py b/airflow-core/tests/unit/dag_processing/test_processor.py
index 8d77da61caf..e8ab15a0222 100644
--- a/airflow-core/tests/unit/dag_processing/test_processor.py
+++ b/airflow-core/tests/unit/dag_processing/test_processor.py
@@ -37,6 +37,7 @@ from airflow.dag_processing.processor import (
DagFileParsingResult,
DagFileProcessorProcess,
_parse_file,
+ _pre_import_airflow_modules,
)
from airflow.models import DagBag, TaskInstance
from airflow.models.baseoperator import BaseOperator
@@ -140,8 +141,13 @@ class TestDagFileProcessor:
assert "a.py" in resp.import_errors
def test_top_level_variable_access(
- self, spy_agency: SpyAgency, tmp_path: pathlib.Path, monkeypatch: pytest.MonkeyPatch, inprocess_client
+ self,
+ spy_agency: SpyAgency,
+ tmp_path: pathlib.Path,
+ monkeypatch: pytest.MonkeyPatch,
+ inprocess_client,
):
+ logger = MagicMock()
logger_filehandle = MagicMock()
def dag_in_a_fn():
@@ -158,6 +164,7 @@ class TestDagFileProcessor:
path=path,
bundle_path=tmp_path,
callbacks=[],
+ logger=logger,
logger_filehandle=logger_filehandle,
client=inprocess_client,
)
@@ -171,8 +178,13 @@ class TestDagFileProcessor:
assert result.serialized_dags[0].dag_id == "test_abc"
def test_top_level_variable_access_not_found(
- self, spy_agency: SpyAgency, tmp_path: pathlib.Path, monkeypatch: pytest.MonkeyPatch, inprocess_client
+ self,
+ spy_agency: SpyAgency,
+ tmp_path: pathlib.Path,
+ monkeypatch: pytest.MonkeyPatch,
+ inprocess_client,
):
+ logger = MagicMock()
logger_filehandle = MagicMock()
def dag_in_a_fn():
@@ -187,6 +199,7 @@ class TestDagFileProcessor:
path=path,
bundle_path=tmp_path,
callbacks=[],
+ logger=logger,
logger_filehandle=logger_filehandle,
client=inprocess_client,
)
@@ -203,6 +216,7 @@ class TestDagFileProcessor:
def test_top_level_variable_set(self, tmp_path: pathlib.Path, inprocess_client):
from airflow.models.variable import Variable as VariableORM
+ logger = MagicMock()
logger_filehandle = MagicMock()
def dag_in_a_fn():
@@ -218,6 +232,7 @@ class TestDagFileProcessor:
path=path,
bundle_path=tmp_path,
callbacks=[],
+ logger=logger,
logger_filehandle=logger_filehandle,
client=inprocess_client,
)
@@ -238,6 +253,7 @@ class TestDagFileProcessor:
def test_top_level_variable_delete(self, tmp_path: pathlib.Path, inprocess_client):
from airflow.models.variable import Variable as VariableORM
+ logger = MagicMock()
logger_filehandle = MagicMock()
def dag_in_a_fn():
@@ -259,6 +275,7 @@ class TestDagFileProcessor:
path=path,
bundle_path=tmp_path,
callbacks=[],
+ logger=logger,
logger_filehandle=logger_filehandle,
client=inprocess_client,
)
@@ -278,6 +295,7 @@ class TestDagFileProcessor:
def test_top_level_connection_access(
self, tmp_path: pathlib.Path, monkeypatch: pytest.MonkeyPatch, inprocess_client
):
+ logger = MagicMock()
logger_filehandle = MagicMock()
def dag_in_a_fn():
@@ -295,6 +313,7 @@ class TestDagFileProcessor:
path=path,
bundle_path=tmp_path,
callbacks=[],
+ logger=logger,
logger_filehandle=logger_filehandle,
client=inprocess_client,
)
@@ -308,6 +327,7 @@ class TestDagFileProcessor:
assert result.serialized_dags[0].dag_id == "test_my_conn"
def test_top_level_connection_access_not_found(self, tmp_path: pathlib.Path, inprocess_client):
+ logger = MagicMock()
logger_filehandle = MagicMock()
def dag_in_a_fn():
@@ -323,6 +343,7 @@ class TestDagFileProcessor:
path=path,
bundle_path=tmp_path,
callbacks=[],
+ logger=logger,
logger_filehandle=logger_filehandle,
client=inprocess_client,
)
@@ -355,6 +376,7 @@ class TestDagFileProcessor:
path=dag1_path,
bundle_path=tmp_path,
callbacks=[],
+ logger=MagicMock(),
logger_filehandle=MagicMock(),
client=inprocess_client,
)
@@ -366,6 +388,65 @@ class TestDagFileProcessor:
assert result.import_errors == {}
assert result.serialized_dags[0].dag_id == "dag_name"
+ def test__pre_import_airflow_modules_when_disabled(self):
+ logger = MagicMock()
+ with (
+ env_vars({"AIRFLOW__DAG_PROCESSOR__PARSING_PRE_IMPORT_MODULES": "false"}),
+ patch("airflow.dag_processing.processor.iter_airflow_imports") as mock_iter,
+ ):
+ _pre_import_airflow_modules("test.py", logger)
+
+ mock_iter.assert_not_called()
+ logger.warning.assert_not_called()
+
+ def test__pre_import_airflow_modules_when_enabled(self):
+ logger = MagicMock()
+ with (
+ env_vars({"AIRFLOW__DAG_PROCESSOR__PARSING_PRE_IMPORT_MODULES": "true"}),
+ patch("airflow.dag_processing.processor.iter_airflow_imports", return_value=["airflow.models"]),
+ patch("airflow.dag_processing.processor.importlib.import_module") as mock_import,
+ ):
+ _pre_import_airflow_modules("test.py", logger)
+
+ mock_import.assert_called_once_with("airflow.models")
+ logger.warning.assert_not_called()
+
+ def test__pre_import_airflow_modules_warns_on_missing_module(self):
+ logger = MagicMock()
+ with (
+ env_vars({"AIRFLOW__DAG_PROCESSOR__PARSING_PRE_IMPORT_MODULES": "true"}),
+ patch(
+ "airflow.dag_processing.processor.iter_airflow_imports", return_value=["non_existent_module"]
+ ),
+ patch(
+ "airflow.dag_processing.processor.importlib.import_module", side_effect=ModuleNotFoundError()
+ ),
+ ):
+ _pre_import_airflow_modules("test.py", logger)
+
+ logger.warning.assert_called_once()
+ warning_args = logger.warning.call_args[0]
+ assert "Error when trying to pre-import module" in warning_args[0]
+ assert "non_existent_module" in warning_args[1]
+ assert "test.py" in warning_args[2]
+
+ def test__pre_import_airflow_modules_partial_success_and_warning(self):
+ logger = MagicMock()
+ with (
+ env_vars({"AIRFLOW__DAG_PROCESSOR__PARSING_PRE_IMPORT_MODULES": "true"}),
+ patch(
+ "airflow.dag_processing.processor.iter_airflow_imports",
+ return_value=["airflow.models", "non_existent_module"],
+ ),
+ patch(
+ "airflow.dag_processing.processor.importlib.import_module",
+ side_effect=[None, ModuleNotFoundError()],
+ ),
+ ):
+ _pre_import_airflow_modules("test.py", logger)
+
+ assert logger.warning.call_count == 1
+
def write_dag_in_a_fn_to_file(fn: Callable[[], None], folder: pathlib.Path) -> pathlib.Path:
# Create the dag in a fn, and use inspect.getsource to write it to a file so that
(airflow) 11/12: [v3-0-test] Unify connection not found exceptions between AF2 and AF3 (#52968) (#53093)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 33cf4223a09b2b92f416ce7bdfe9eb0a94a88537
Author: Amogh Desai <am...@gmail.com>
AuthorDate: Fri Jul 11 11:47:07 2025 +0530
[v3-0-test] Unify connection not found exceptions between AF2 and AF3 (#52968) (#53093)
---
airflow-core/src/airflow/models/connection.py | 3 +--
airflow-core/tests/unit/dag_processing/test_processor.py | 2 +-
task-sdk/src/airflow/sdk/definitions/connection.py | 10 ++++++++--
task-sdk/src/airflow/sdk/execution_time/context.py | 4 +++-
task-sdk/tests/task_sdk/definitions/test_connections.py | 12 ++++++++++--
5 files changed, 23 insertions(+), 8 deletions(-)
diff --git a/airflow-core/src/airflow/models/connection.py b/airflow-core/src/airflow/models/connection.py
index 2727c926e0f..9205083c9d3 100644
--- a/airflow-core/src/airflow/models/connection.py
+++ b/airflow-core/src/airflow/models/connection.py
@@ -478,8 +478,7 @@ class Connection(Base, LoggingMixin):
return conn
except AirflowRuntimeError as e:
if e.error.error == ErrorType.CONNECTION_NOT_FOUND:
- log.debug("Unable to retrieve connection from MetastoreBackend using Task SDK")
- raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined")
+ raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined") from None
raise
# check cache first
diff --git a/airflow-core/tests/unit/dag_processing/test_processor.py b/airflow-core/tests/unit/dag_processing/test_processor.py
index 71b601dfdd8..1649cb63896 100644
--- a/airflow-core/tests/unit/dag_processing/test_processor.py
+++ b/airflow-core/tests/unit/dag_processing/test_processor.py
@@ -324,7 +324,7 @@ class TestDagFileProcessor:
assert result is not None
assert result.import_errors != {}
if result.import_errors:
- assert "CONNECTION_NOT_FOUND" in next(iter(result.import_errors.values()))
+ assert "The conn_id `my_conn` isn't defined" in next(iter(result.import_errors.values()))
def test_import_module_in_bundle_root(self, tmp_path: pathlib.Path, inprocess_client):
tmp_path.joinpath("util.py").write_text("NAME = 'dag_name'")
diff --git a/task-sdk/src/airflow/sdk/definitions/connection.py b/task-sdk/src/airflow/sdk/definitions/connection.py
index c66b264dce0..837a1d4e25d 100644
--- a/task-sdk/src/airflow/sdk/definitions/connection.py
+++ b/task-sdk/src/airflow/sdk/definitions/connection.py
@@ -24,7 +24,8 @@ from typing import Any
import attrs
-from airflow.exceptions import AirflowException
+from airflow.exceptions import AirflowException, AirflowNotFoundException
+from airflow.sdk.exceptions import AirflowRuntimeError, ErrorType
log = logging.getLogger(__name__)
@@ -139,7 +140,12 @@ class Connection:
def get(cls, conn_id: str) -> Any:
from airflow.sdk.execution_time.context import _get_connection
- return _get_connection(conn_id)
+ try:
+ return _get_connection(conn_id)
+ except AirflowRuntimeError as e:
+ if e.error.error == ErrorType.CONNECTION_NOT_FOUND:
+ raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined") from None
+ raise
@property
def extra_dejson(self) -> dict:
diff --git a/task-sdk/src/airflow/sdk/execution_time/context.py b/task-sdk/src/airflow/sdk/execution_time/context.py
index 744b3b08886..9ee2e03c9b8 100644
--- a/task-sdk/src/airflow/sdk/execution_time/context.py
+++ b/task-sdk/src/airflow/sdk/execution_time/context.py
@@ -272,7 +272,9 @@ class ConnectionAccessor:
"""Wrapper to access Connection entries in template."""
def __getattr__(self, conn_id: str) -> Any:
- return _get_connection(conn_id)
+ from airflow.sdk.definitions.connection import Connection
+
+ return Connection.get(conn_id)
def __repr__(self) -> str:
return "<ConnectionAccessor (dynamic access)>"
diff --git a/task-sdk/tests/task_sdk/definitions/test_connections.py b/task-sdk/tests/task_sdk/definitions/test_connections.py
index 3bbb63a7697..6e4d977c659 100644
--- a/task-sdk/tests/task_sdk/definitions/test_connections.py
+++ b/task-sdk/tests/task_sdk/definitions/test_connections.py
@@ -23,9 +23,10 @@ from urllib.parse import urlparse
import pytest
from airflow.configuration import initialize_secrets_backends
-from airflow.exceptions import AirflowException
+from airflow.exceptions import AirflowException, AirflowNotFoundException
from airflow.sdk import Connection
-from airflow.sdk.execution_time.comms import ConnectionResult
+from airflow.sdk.exceptions import ErrorType
+from airflow.sdk.execution_time.comms import ConnectionResult, ErrorResponse
from airflow.secrets import DEFAULT_SECRETS_SEARCH_PATH_WORKERS
from tests_common.test_utils.config import conf_vars
@@ -121,6 +122,13 @@ class TestConnections:
extra=None,
)
+ def test_conn_get_not_found(self, mock_supervisor_comms):
+ error_response = ErrorResponse(error=ErrorType.CONNECTION_NOT_FOUND)
+ mock_supervisor_comms.send.return_value = error_response
+
+ with pytest.raises(AirflowNotFoundException, match="The conn_id `mysql_conn` isn't defined"):
+ _ = Connection.get(conn_id="mysql_conn")
+
class TestConnectionsFromSecrets:
def test_get_connection_secrets_backend(self, mock_supervisor_comms, tmp_path):
(airflow) 02/12: [v3-0-test] Fix reading huge (XCom) resposne in TaskSDK task process (#53186) (#53194)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit d159feef80daa1c47125ff27fbc63184364e3da4
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Fri Jul 11 21:26:31 2025 +0530
[v3-0-test] Fix reading huge (XCom) resposne in TaskSDK task process (#53186) (#53194)
If you tried to send a large XCom value, it would fail in the task/child
process side with this error:
> RuntimeError: unable to read full response in child. (We read 36476, but expected 1310046)
(The exact number that was able to read dependent on any different factors,
like the OS, the current state of the socket and other things. Sometimes it
would read up to 256kb fine, othertimes only 35kb as here)
This is because the kernel level read-side socket buffer is full, so that was
as much as the Supervisor could send. The fix is to read in a loop until we
get it all.
(cherry picked from commit b9620bf4b77ec45b0e776aa56cab2b6ed7744dbe)
Co-authored-by: Ash Berlin-Taylor <as...@apache.org>
---
task-sdk/src/airflow/sdk/execution_time/comms.py | 19 +++++++-------
.../tests/task_sdk/execution_time/test_comms.py | 30 ++++++++++++++++++++++
2 files changed, 40 insertions(+), 9 deletions(-)
diff --git a/task-sdk/src/airflow/sdk/execution_time/comms.py b/task-sdk/src/airflow/sdk/execution_time/comms.py
index 97ed1761ad8..2c6dfea4e60 100644
--- a/task-sdk/src/airflow/sdk/execution_time/comms.py
+++ b/task-sdk/src/airflow/sdk/execution_time/comms.py
@@ -228,15 +228,16 @@ class CommsDecoder(Generic[ReceiveMsgType, SendMsgType]):
length = int.from_bytes(len_bytes, byteorder="big")
buffer = bytearray(length)
- nread = self.socket.recv_into(buffer)
- if nread != length:
- raise RuntimeError(
- f"unable to read full response in child. (We read {nread}, but expected {length})"
- )
- if nread == 0:
- raise EOFError(f"Request socket closed before response was complete ({self.id_counter=})")
-
- resp = self.resp_decoder.decode(buffer)
+ mv = memoryview(buffer)
+
+ pos = 0
+ while pos < length:
+ nread = self.socket.recv_into(mv[pos:])
+ if nread == 0:
+ raise EOFError(f"Request socket closed before response was complete ({self.id_counter=})")
+ pos += nread
+
+ resp = self.resp_decoder.decode(mv)
if maxfds:
return resp, fds or []
return resp
diff --git a/task-sdk/tests/task_sdk/execution_time/test_comms.py b/task-sdk/tests/task_sdk/execution_time/test_comms.py
index 5adaa2562ab..48c7ad74a15 100644
--- a/task-sdk/tests/task_sdk/execution_time/test_comms.py
+++ b/task-sdk/tests/task_sdk/execution_time/test_comms.py
@@ -17,6 +17,7 @@
from __future__ import annotations
+import threading
import uuid
from socket import socketpair
@@ -81,3 +82,32 @@ class TestCommsDecoder:
assert msg.dag_rel_path == "/dev/null"
assert msg.bundle_info == BundleInfo(name="any-name", version="any-version")
assert msg.start_date == timezone.datetime(2024, 12, 1, 1)
+
+ def test_huge_payload(self):
+ r, w = socketpair()
+
+ msg = {
+ "type": "XComResult",
+ "key": "a",
+ "value": ("a" * 10 * 1024 * 1024) + "b", # A 10mb xcom value
+ }
+
+ w.settimeout(1.0)
+ bytes = msgspec.msgpack.encode(_ResponseFrame(0, msg, None))
+
+ # Since `sendall` blocks, we need to do the send in another thread, so we can perform the read here
+ t = threading.Thread(target=w.sendall, args=(len(bytes).to_bytes(4, byteorder="big") + bytes,))
+ t.start()
+
+ decoder = CommsDecoder(socket=r, log=None)
+
+ try:
+ msg = decoder._get_response()
+ finally:
+ t.join(2)
+
+ assert msg is not None
+
+ # It actually failed to read at all for large values, but lets just make sure we get it all
+ assert len(msg.value) == 10 * 1024 * 1024 + 1
+ assert msg.value[-1] == "b"
(airflow) 05/12: [v3-0-test] Workaround the need to use latest version on Lucas pre-commit's (#52931) (#52942)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 1edbd2dd3d4cd3b6ffbe71058482f8aaba81900a
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Sun Jul 6 21:13:35 2025 +0200
[v3-0-test] Workaround the need to use latest version on Lucas pre-commit's (#52931) (#52942)
We are still waiting for release of the Lucas's pre-commit after
the https://github.com/Lucas-C/pre-commit-hooks/pull/103 has
been merged - and for now we need to use `bleeding-edge` for
the repo.
Also upgrades zizmor and solves using "env." in shell commands
that might lead to security issues.
(cherry picked from commit 535d71d91f36d3d9859d32e4695f0e81a2696a4b)
---
.github/actions/migration_tests/action.yml | 16 ++++++++--------
.github/workflows/basic-tests.yml | 2 ++
.pre-commit-config.yaml | 5 +++--
3 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/.github/actions/migration_tests/action.yml b/.github/actions/migration_tests/action.yml
index 0198cbf472b..34951bac5ff 100644
--- a/.github/actions/migration_tests/action.yml
+++ b/.github/actions/migration_tests/action.yml
@@ -24,9 +24,9 @@ runs:
- name: "Test migration file 2 to 3 migration: ${{env.BACKEND}}"
shell: bash
run: |
- breeze shell "${{ env.AIRFLOW_2_CMD }}" --use-airflow-version 2.11.0 --answer y &&
- breeze shell "export AIRFLOW__DATABASE__EXTERNAL_DB_MANAGERS=${{env.DB_MANGERS}}
- ${{ env.AIRFLOW_3_CMD }}" --no-db-cleanup
+ breeze shell "${AIRFLOW_2_CMD}" --use-airflow-version 2.11.0 --answer y &&
+ breeze shell "export AIRFLOW__DATABASE__EXTERNAL_DB_MANAGERS=${DB_MANGERS}
+ ${AIRFLOW_3_CMD}" --no-db-cleanup
env:
COMPOSE_PROJECT_NAME: "docker-compose"
DB_RESET: "false"
@@ -47,9 +47,9 @@ runs:
- name: "Test ORM migration 2 to 3: ${{env.BACKEND}}"
shell: bash
run: >
- breeze shell "${{ env.AIRFLOW_2_CMD }}" --use-airflow-version 2.11.0 --answer y &&
- breeze shell "export AIRFLOW__DATABASE__EXTERNAL_DB_MANAGERS=${{env.DB_MANGERS}}
- ${{ env.AIRFLOW_3_CMD }}" --no-db-cleanup
+ breeze shell "${AIRFLOW_2_CMD}" --use-airflow-version 2.11.0 --answer y &&
+ breeze shell "export AIRFLOW__DATABASE__EXTERNAL_DB_MANAGERS=${DB_MANGERS}
+ ${AIRFLOW_3_CMD}" --no-db-cleanup
env:
COMPOSE_PROJECT_NAME: "docker-compose"
DB_RESET: "false"
@@ -69,7 +69,7 @@ runs:
- name: "Test ORM migration ${{env.BACKEND}}"
shell: bash
run: >
- breeze shell "export AIRFLOW__DATABASE__EXTERNAL_DB_MANAGERS=${{env.DB_MANAGERS}} &&
+ breeze shell "export AIRFLOW__DATABASE__EXTERNAL_DB_MANAGERS=${DB_MANAGERS} &&
airflow db reset -y &&
airflow db migrate --to-revision heads &&
airflow db downgrade -n 2.7.0 -y &&
@@ -86,7 +86,7 @@ runs:
shell: bash
run: >
breeze shell
- "export AIRFLOW__DATABASE__EXTERNAL_DB_MANAGERS=${{env.DB_MANAGERS}} &&
+ "export AIRFLOW__DATABASE__EXTERNAL_DB_MANAGERS=${DB_MANAGERS} &&
airflow db reset -y &&
airflow db downgrade -n 2.7.0 -y &&
airflow db migrate -s"
diff --git a/.github/workflows/basic-tests.yml b/.github/workflows/basic-tests.yml
index 59a326d49b1..bc23674f442 100644
--- a/.github/workflows/basic-tests.yml
+++ b/.github/workflows/basic-tests.yml
@@ -267,6 +267,8 @@ jobs:
skip-pre-commits: ${{ inputs.skip-pre-commits }}
- name: "Autoupdate all pre-commits"
run: pre-commit autoupdate
+ - name: "Autoupdate Lucas-C/pre-commit-hooks to bleeding edge"
+ run: pre-commit autoupdate --bleeding-edge --freeze --repo https://github.com/Lucas-C/pre-commit-hooks
- name: "Run automated upgrade for black"
run: >
pre-commit run
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 3dc226a64df..3adeefd79b6 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -49,7 +49,8 @@ repos:
- "--maxlevel"
- "2"
- repo: https://github.com/Lucas-C/pre-commit-hooks
- rev: v1.5.5
+ # replace hash with version once PR #103 merged comes in a release
+ rev: abdd8b62891099da34162217ecb3872d22184a51
hooks:
- id: insert-license
name: Add license for all SQL files
@@ -356,7 +357,7 @@ repos:
- --skip=providers/.*/src/airflow/providers/*/*.rst,providers/*/docs/changelog.rst,docs/*/commits.rst,providers/*/docs/commits.rst,providers/*/*/docs/commits.rst,docs/apache-airflow/tutorial/pipeline_example.csv,*.min.js,*.lock,INTHEWILD.md,*.svg
- --exclude-file=.codespellignorelines
- repo: https://github.com/woodruffw/zizmor-pre-commit
- rev: v1.7.0
+ rev: v1.11.0
hooks:
- id: zizmor
name: Run zizmor to check for github workflow syntax errors
(airflow) 12/12: [v3-0-test] Move --no-binary flag to uv command line from pyproject.toml (#53179) (#53185)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 61f39160b62e9f450fb1019f67e61b5bacb465ef
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Fri Jul 11 16:09:24 2025 +0200
[v3-0-test] Move --no-binary flag to uv command line from pyproject.toml (#53179) (#53185)
When the flag is specified in pyproject toml, it also forces no
binary installation of xmlsec and lxml for local uv sync which might
fail if some system libraries are not installed, so it is better
to do it in the image by passing the right flags to installer
directly.
(cherry picked from commit 47bbe55355757953507b8ae64cd05f9380e16748)
Co-authored-by: Jarek Potiuk <ja...@potiuk.com>
---
Dockerfile | 21 +++++++++++----
Dockerfile.ci | 31 +++++++++++++++++-----
scripts/docker/common.sh | 9 ++++---
scripts/docker/entrypoint_ci.sh | 10 +++++--
.../docker/install_airflow_when_building_images.sh | 12 +++++++--
5 files changed, 64 insertions(+), 19 deletions(-)
diff --git a/Dockerfile b/Dockerfile
index 023fb244294..141e6e129d7 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -473,11 +473,14 @@ function common::get_packaging_tool() {
echo
export PACKAGING_TOOL="uv"
export PACKAGING_TOOL_CMD="uv pip"
- if [[ ${AIRFLOW_INSTALLATION_METHOD=} == "." && -f "./pyproject.toml" ]]; then
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
+ if [[ ${AIRFLOW_INSTALLATION_METHOD=} == "." && -f "./pyproject.toml" ]]; then
# for uv only install dev group when we install from sources
- export EXTRA_INSTALL_FLAGS="--group=dev"
+ export EXTRA_INSTALL_FLAGS="--group=dev --no-binary lxml --no-binary xmlsec"
else
- export EXTRA_INSTALL_FLAGS=""
+ export EXTRA_INSTALL_FLAGS="--no-binary lxml --no-binary xmlsec"
fi
export EXTRA_UNINSTALL_FLAGS=""
export UPGRADE_TO_HIGHEST_RESOLUTION="--upgrade --resolution highest"
@@ -875,8 +878,12 @@ function install_from_sources() {
echo
echo "${COLOR_BLUE}Attempting to upgrade all packages to highest versions.${COLOR_RESET}"
echo
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
set -x
- uv sync --all-packages --resolution highest --group dev --group docs --group docs-gen --group leveldb ${extra_sync_flags}
+ uv sync --all-packages --resolution highest --group dev --group docs --group docs-gen \
+ --group leveldb ${extra_sync_flags} --no-binary-package lxml --no-binary-package xmlsec
else
# We only use uv here but Installing using constraints is not supported with `uv sync`, so we
# do not use ``uv sync`` because we are not committing and using uv.lock yet.
@@ -933,8 +940,12 @@ function install_from_sources() {
echo
echo "${COLOR_BLUE}Falling back to no-constraints installation.${COLOR_RESET}"
echo
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
set -x
- uv sync --all-packages --group dev --group docs --group docs-gen --group leveldb ${extra_sync_flags}
+ uv sync --all-packages --group dev --group docs --group docs-gen \
+ --group leveldb ${extra_sync_flags} --no-binary-package lxml --no-binary-packag xmlsec
set +x
fi
fi
diff --git a/Dockerfile.ci b/Dockerfile.ci
index 7fc0d572bf0..8168cc5187c 100644
--- a/Dockerfile.ci
+++ b/Dockerfile.ci
@@ -412,11 +412,14 @@ function common::get_packaging_tool() {
echo
export PACKAGING_TOOL="uv"
export PACKAGING_TOOL_CMD="uv pip"
- if [[ ${AIRFLOW_INSTALLATION_METHOD=} == "." && -f "./pyproject.toml" ]]; then
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
+ if [[ ${AIRFLOW_INSTALLATION_METHOD=} == "." && -f "./pyproject.toml" ]]; then
# for uv only install dev group when we install from sources
- export EXTRA_INSTALL_FLAGS="--group=dev"
+ export EXTRA_INSTALL_FLAGS="--group=dev --no-binary lxml --no-binary xmlsec"
else
- export EXTRA_INSTALL_FLAGS=""
+ export EXTRA_INSTALL_FLAGS="--no-binary lxml --no-binary xmlsec"
fi
export EXTRA_UNINSTALL_FLAGS=""
export UPGRADE_TO_HIGHEST_RESOLUTION="--upgrade --resolution highest"
@@ -628,8 +631,12 @@ function install_from_sources() {
echo
echo "${COLOR_BLUE}Attempting to upgrade all packages to highest versions.${COLOR_RESET}"
echo
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
set -x
- uv sync --all-packages --resolution highest --group dev --group docs --group docs-gen --group leveldb ${extra_sync_flags}
+ uv sync --all-packages --resolution highest --group dev --group docs --group docs-gen \
+ --group leveldb ${extra_sync_flags} --no-binary-package lxml --no-binary-package xmlsec
else
# We only use uv here but Installing using constraints is not supported with `uv sync`, so we
# do not use ``uv sync`` because we are not committing and using uv.lock yet.
@@ -686,8 +693,12 @@ function install_from_sources() {
echo
echo "${COLOR_BLUE}Falling back to no-constraints installation.${COLOR_RESET}"
echo
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
set -x
- uv sync --all-packages --group dev --group docs --group docs-gen --group leveldb ${extra_sync_flags}
+ uv sync --all-packages --group dev --group docs --group docs-gen \
+ --group leveldb ${extra_sync_flags} --no-binary-package lxml --no-binary-packag xmlsec
set +x
fi
fi
@@ -1125,13 +1136,19 @@ function check_force_lowest_dependencies() {
exit 0
fi
cd "${AIRFLOW_SOURCES}/providers/${provider_id/.//}" || exit 1
- uv sync --resolution lowest-direct
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
+ uv sync --resolution lowest-direct --no-binary-package lxml --no-binary-package xmlsec
else
echo
echo "${COLOR_BLUE}Forcing dependencies to lowest versions for Airflow.${COLOR_RESET}"
echo
cd "${AIRFLOW_SOURCES}/airflow-core"
- uv sync --resolution lowest-direct
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
+ uv sync --resolution lowest-direct --no-binary-package lxml --no-binary-package xmlsec
fi
}
diff --git a/scripts/docker/common.sh b/scripts/docker/common.sh
index 4568aa2183b..259ac7a76ec 100644
--- a/scripts/docker/common.sh
+++ b/scripts/docker/common.sh
@@ -42,11 +42,14 @@ function common::get_packaging_tool() {
echo
export PACKAGING_TOOL="uv"
export PACKAGING_TOOL_CMD="uv pip"
- if [[ ${AIRFLOW_INSTALLATION_METHOD=} == "." && -f "./pyproject.toml" ]]; then
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
+ if [[ ${AIRFLOW_INSTALLATION_METHOD=} == "." && -f "./pyproject.toml" ]]; then
# for uv only install dev group when we install from sources
- export EXTRA_INSTALL_FLAGS="--group=dev"
+ export EXTRA_INSTALL_FLAGS="--group=dev --no-binary lxml --no-binary xmlsec"
else
- export EXTRA_INSTALL_FLAGS=""
+ export EXTRA_INSTALL_FLAGS="--no-binary lxml --no-binary xmlsec"
fi
export EXTRA_UNINSTALL_FLAGS=""
export UPGRADE_TO_HIGHEST_RESOLUTION="--upgrade --resolution highest"
diff --git a/scripts/docker/entrypoint_ci.sh b/scripts/docker/entrypoint_ci.sh
index e62e0a58c80..b8a4e994ceb 100755
--- a/scripts/docker/entrypoint_ci.sh
+++ b/scripts/docker/entrypoint_ci.sh
@@ -336,13 +336,19 @@ function check_force_lowest_dependencies() {
exit 0
fi
cd "${AIRFLOW_SOURCES}/providers/${provider_id/.//}" || exit 1
- uv sync --resolution lowest-direct
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
+ uv sync --resolution lowest-direct --no-binary-package lxml --no-binary-package xmlsec
else
echo
echo "${COLOR_BLUE}Forcing dependencies to lowest versions for Airflow.${COLOR_RESET}"
echo
cd "${AIRFLOW_SOURCES}/airflow-core"
- uv sync --resolution lowest-direct
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
+ uv sync --resolution lowest-direct --no-binary-package lxml --no-binary-package xmlsec
fi
}
diff --git a/scripts/docker/install_airflow_when_building_images.sh b/scripts/docker/install_airflow_when_building_images.sh
index 3291a6b4cc0..78ddc7eaf85 100644
--- a/scripts/docker/install_airflow_when_building_images.sh
+++ b/scripts/docker/install_airflow_when_building_images.sh
@@ -56,8 +56,12 @@ function install_from_sources() {
echo
echo "${COLOR_BLUE}Attempting to upgrade all packages to highest versions.${COLOR_RESET}"
echo
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
set -x
- uv sync --all-packages --resolution highest --group dev --group docs --group docs-gen --group leveldb ${extra_sync_flags}
+ uv sync --all-packages --resolution highest --group dev --group docs --group docs-gen \
+ --group leveldb ${extra_sync_flags} --no-binary-package lxml --no-binary-package xmlsec
else
# We only use uv here but Installing using constraints is not supported with `uv sync`, so we
# do not use ``uv sync`` because we are not committing and using uv.lock yet.
@@ -114,8 +118,12 @@ function install_from_sources() {
echo
echo "${COLOR_BLUE}Falling back to no-constraints installation.${COLOR_RESET}"
echo
+ # --no-binary is needed in order to avoid libxml and xmlsec using different version of libxml2
+ # (binary lxml embeds its own libxml2, while xmlsec uses system one).
+ # See https://bugs.launchpad.net/lxml/+bug/2110068
set -x
- uv sync --all-packages --group dev --group docs --group docs-gen --group leveldb ${extra_sync_flags}
+ uv sync --all-packages --group dev --group docs --group docs-gen \
+ --group leveldb ${extra_sync_flags} --no-binary-package lxml --no-binary-packag xmlsec
set +x
fi
fi
(airflow) 10/12: Run Task failure callbacks on DAG Processor when task is externally killed (#53058) (#53143)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 9359b88bc9c726e2f7f32de949a3a78d1feaa1b1
Author: Kaxil Naik <ka...@apache.org>
AuthorDate: Thu Jul 10 22:08:57 2025 +0530
Run Task failure callbacks on DAG Processor when task is externally killed (#53058) (#53143)
Until https://github.com/apache/airflow/issues/44354 is implemented, tasks killed externally or when supervisor process dies unexpectedly, users have no way of knowing this happened.
This has been a blocker for Airflow 3.0 adoption for some:
- https://github.com/apache/airflow/issues/44354
- https://apache-airflow.slack.com/archives/C07813CNKA8/p1751057525231389
https://github.com/apache/airflow/issues/44354 is more involved and we might not get to it for Airflow 3.1 -- so this is a good fix until then similar to how we run Dag Run callback.
(cherry-picked from a5211f2efd5ccc565cbc16baee6144dba09918bc)
---
.../execution_api/datamodels/taskinstance.py | 4 +-
.../src/airflow/callbacks/callback_requests.py | 2 +
.../src/airflow/dag_processing/processor.py | 72 ++++-
.../src/airflow/jobs/scheduler_job_runner.py | 18 +-
.../tests/unit/callbacks/test_callback_requests.py | 29 +-
.../tests/unit/dag_processing/test_processor.py | 328 ++++++++++++++++++---
airflow-core/tests/unit/jobs/test_scheduler_job.py | 141 ++++-----
.../src/airflow/sdk/api/datamodels/_generated.py | 2 +-
8 files changed, 472 insertions(+), 124 deletions(-)
diff --git a/airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py b/airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
index c43c931f3e2..2d7968bbc62 100644
--- a/airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
+++ b/airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
@@ -302,7 +302,7 @@ class TIRunContext(BaseModel):
dag_run: DagRun
"""DAG run information for the task instance."""
- task_reschedule_count: Annotated[int, Field(default=0)]
+ task_reschedule_count: int = 0
"""How many times the task has been rescheduled."""
max_tries: int
@@ -328,7 +328,7 @@ class TIRunContext(BaseModel):
xcom_keys_to_clear: Annotated[list[str], Field(default_factory=list)]
"""List of Xcom keys that need to be cleared and purged on by the worker."""
- should_retry: bool
+ should_retry: bool = False
"""If the ti encounters an error, whether it should enter retry or failed state."""
diff --git a/airflow-core/src/airflow/callbacks/callback_requests.py b/airflow-core/src/airflow/callbacks/callback_requests.py
index 8cf8c770357..3220497a209 100644
--- a/airflow-core/src/airflow/callbacks/callback_requests.py
+++ b/airflow-core/src/airflow/callbacks/callback_requests.py
@@ -61,6 +61,8 @@ class TaskCallbackRequest(BaseCallbackRequest):
"""Simplified Task Instance representation"""
task_callback_type: TaskInstanceState | None = None
"""Whether on success, on failure, on retry"""
+ context_from_server: ti_datamodel.TIRunContext | None = None
+ """Task execution context from the Server"""
type: Literal["TaskCallbackRequest"] = "TaskCallbackRequest"
@property
diff --git a/airflow-core/src/airflow/dag_processing/processor.py b/airflow-core/src/airflow/dag_processing/processor.py
index a54c56ddbe2..36022c61dfd 100644
--- a/airflow-core/src/airflow/dag_processing/processor.py
+++ b/airflow-core/src/airflow/dag_processing/processor.py
@@ -16,12 +16,14 @@
# under the License.
from __future__ import annotations
+import contextlib
import importlib
import os
import sys
import traceback
+from collections.abc import Callable, Sequence
from pathlib import Path
-from typing import TYPE_CHECKING, Annotated, BinaryIO, Callable, ClassVar, Literal, Union
+from typing import TYPE_CHECKING, Annotated, BinaryIO, ClassVar, Literal, Union
import attrs
from pydantic import BaseModel, Field, TypeAdapter
@@ -44,9 +46,11 @@ from airflow.sdk.execution_time.comms import (
VariableResult,
)
from airflow.sdk.execution_time.supervisor import WatchedSubprocess
+from airflow.sdk.execution_time.task_runner import RuntimeTaskInstance
from airflow.serialization.serialized_objects import LazyDeserializedDAG, SerializedDAG
from airflow.stats import Stats
from airflow.utils.file import iter_airflow_imports
+from airflow.utils.state import TaskInstanceState
if TYPE_CHECKING:
from structlog.typing import FilteringBoundLogger
@@ -200,10 +204,7 @@ def _execute_callbacks(
for request in callback_requests:
log.debug("Processing Callback Request", request=request.to_json())
if isinstance(request, TaskCallbackRequest):
- raise NotImplementedError(
- "Haven't coded Task callback yet - https://github.com/apache/airflow/issues/44354!"
- )
- # _execute_task_callbacks(dagbag, request)
+ _execute_task_callbacks(dagbag, request, log)
if isinstance(request, DagCallbackRequest):
_execute_dag_callbacks(dagbag, request, log)
@@ -237,6 +238,67 @@ def _execute_dag_callbacks(dagbag: DagBag, request: DagCallbackRequest, log: Fil
Stats.incr("dag.callback_exceptions", tags={"dag_id": request.dag_id})
+def _execute_task_callbacks(dagbag: DagBag, request: TaskCallbackRequest, log: FilteringBoundLogger) -> None:
+ if not request.is_failure_callback:
+ log.warning(
+ "Task callback requested but is not a failure callback",
+ dag_id=request.ti.dag_id,
+ task_id=request.ti.task_id,
+ run_id=request.ti.run_id,
+ )
+ return
+
+ dag = dagbag.dags[request.ti.dag_id]
+ task = dag.get_task(request.ti.task_id)
+
+ if request.task_callback_type is TaskInstanceState.UP_FOR_RETRY:
+ callbacks = task.on_retry_callback
+ else:
+ callbacks = task.on_failure_callback
+
+ if not callbacks:
+ log.warning(
+ "Callback requested but no callback found",
+ dag_id=request.ti.dag_id,
+ task_id=request.ti.task_id,
+ run_id=request.ti.run_id,
+ ti_id=request.ti.id,
+ )
+ return
+
+ callbacks = callbacks if isinstance(callbacks, Sequence) else [callbacks]
+ ctx_from_server = request.context_from_server
+
+ if ctx_from_server is not None:
+ runtime_ti = RuntimeTaskInstance.model_construct(
+ **request.ti.model_dump(exclude_unset=True),
+ task=task,
+ _ti_context_from_server=ctx_from_server,
+ max_tries=ctx_from_server.max_tries,
+ )
+ else:
+ runtime_ti = RuntimeTaskInstance.model_construct(
+ **request.ti.model_dump(exclude_unset=True),
+ task=task,
+ )
+ context = runtime_ti.get_template_context()
+
+ def get_callback_representation(callback):
+ with contextlib.suppress(AttributeError):
+ return callback.__name__
+ with contextlib.suppress(AttributeError):
+ return callback.__class__.__name__
+ return callback
+
+ for idx, callback in enumerate(callbacks):
+ callback_repr = get_callback_representation(callback)
+ log.info("Executing Task callback at index %d: %s", idx, callback_repr)
+ try:
+ callback(context)
+ except Exception:
+ log.exception("Error in callback at index %d: %s", idx, callback_repr)
+
+
def in_process_api_server() -> InProcessExecutionAPI:
from airflow.api_fastapi.execution_api.app import InProcessExecutionAPI
diff --git a/airflow-core/src/airflow/jobs/scheduler_job_runner.py b/airflow-core/src/airflow/jobs/scheduler_job_runner.py
index 4185cbf6d44..0fa598be412 100644
--- a/airflow-core/src/airflow/jobs/scheduler_job_runner.py
+++ b/airflow-core/src/airflow/jobs/scheduler_job_runner.py
@@ -38,6 +38,7 @@ from sqlalchemy.orm import joinedload, lazyload, load_only, make_transient, sele
from sqlalchemy.sql import expression
from airflow import settings
+from airflow.api_fastapi.execution_api.datamodels.taskinstance import TIRunContext
from airflow.callbacks.callback_requests import DagCallbackRequest, TaskCallbackRequest
from airflow.configuration import conf
from airflow.dag_processing.bundles.base import BundleUsageTrackingManager
@@ -945,10 +946,16 @@ class SchedulerJobRunner(BaseJobRunner, LoggingMixin):
bundle_version=ti.dag_version.bundle_version,
ti=ti,
msg=msg,
+ context_from_server=TIRunContext(
+ dag_run=ti.dag_run,
+ max_tries=ti.max_tries,
+ variables=[],
+ connections=[],
+ xcom_keys_to_clear=[],
+ ),
)
executor.send_callback(request)
- else:
- ti.handle_failure(error=msg, session=session)
+ ti.handle_failure(error=msg, session=session)
return len(event_buffer)
@@ -2296,6 +2303,13 @@ class SchedulerJobRunner(BaseJobRunner, LoggingMixin):
bundle_version=ti.dag_run.bundle_version,
ti=ti,
msg=str(task_instance_heartbeat_timeout_message_details),
+ context_from_server=TIRunContext(
+ dag_run=ti.dag_run,
+ max_tries=ti.max_tries,
+ variables=[],
+ connections=[],
+ xcom_keys_to_clear=[],
+ ),
)
session.add(
Log(
diff --git a/airflow-core/tests/unit/callbacks/test_callback_requests.py b/airflow-core/tests/unit/callbacks/test_callback_requests.py
index 37a7a3023d8..d27b7ee343c 100644
--- a/airflow-core/tests/unit/callbacks/test_callback_requests.py
+++ b/airflow-core/tests/unit/callbacks/test_callback_requests.py
@@ -28,7 +28,7 @@ from airflow.models.dag import DAG
from airflow.models.taskinstance import TaskInstance
from airflow.providers.standard.operators.bash import BashOperator
from airflow.utils import timezone
-from airflow.utils.state import State
+from airflow.utils.state import State, TaskInstanceState
pytestmark = pytest.mark.db_test
@@ -85,3 +85,30 @@ class TestCallbackRequest:
json_str = input.to_json()
result = TaskCallbackRequest.from_json(json_str)
assert input == result
+
+ @pytest.mark.parametrize(
+ "task_callback_type,expected_is_failure",
+ [
+ (None, True),
+ (TaskInstanceState.FAILED, True),
+ (TaskInstanceState.UP_FOR_RETRY, True),
+ (TaskInstanceState.UPSTREAM_FAILED, True),
+ (TaskInstanceState.SUCCESS, False),
+ (TaskInstanceState.RUNNING, False),
+ ],
+ )
+ def test_is_failure_callback_property(
+ self, task_callback_type, expected_is_failure, create_task_instance
+ ):
+ """Test is_failure_callback property with different task callback types"""
+ ti = create_task_instance()
+
+ request = TaskCallbackRequest(
+ filepath="filepath",
+ ti=ti,
+ bundle_name="testing",
+ bundle_version=None,
+ task_callback_type=task_callback_type,
+ )
+
+ assert request.is_failure_callback == expected_is_failure
diff --git a/airflow-core/tests/unit/dag_processing/test_processor.py b/airflow-core/tests/unit/dag_processing/test_processor.py
index e8ab15a0222..71b601dfdd8 100644
--- a/airflow-core/tests/unit/dag_processing/test_processor.py
+++ b/airflow-core/tests/unit/dag_processing/test_processor.py
@@ -21,8 +21,10 @@ import inspect
import pathlib
import sys
import textwrap
+import uuid
+from collections.abc import Callable
from socket import socketpair
-from typing import TYPE_CHECKING, Callable
+from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
import pytest
@@ -30,24 +32,27 @@ import structlog
from pydantic import TypeAdapter
from airflow.api_fastapi.execution_api.app import InProcessExecutionAPI
+from airflow.api_fastapi.execution_api.datamodels.taskinstance import (
+ TaskInstance as TIDataModel,
+ TIRunContext,
+)
from airflow.callbacks.callback_requests import CallbackRequest, DagCallbackRequest, TaskCallbackRequest
-from airflow.configuration import conf
from airflow.dag_processing.processor import (
DagFileParseRequest,
DagFileParsingResult,
DagFileProcessorProcess,
+ _execute_task_callbacks,
_parse_file,
_pre_import_airflow_modules,
)
-from airflow.models import DagBag, TaskInstance
+from airflow.models import DagBag, DagRun
from airflow.models.baseoperator import BaseOperator
-from airflow.models.serialized_dag import SerializedDagModel
+from airflow.sdk import DAG
from airflow.sdk.api.client import Client
from airflow.sdk.execution_time import comms
from airflow.utils import timezone
from airflow.utils.session import create_session
-from airflow.utils.state import DagRunState, TaskInstanceState
-from airflow.utils.types import DagRunTriggeredByType, DagRunType
+from airflow.utils.state import TaskInstanceState
from tests_common.test_utils.config import conf_vars, env_vars
@@ -93,42 +98,6 @@ class TestDagFileProcessor:
log=structlog.get_logger(),
)
- @pytest.mark.xfail(reason="TODO: AIP-72")
- @pytest.mark.parametrize(
- ["has_serialized_dag"],
- [pytest.param(True, id="dag_in_db"), pytest.param(False, id="no_dag_found")],
- )
- @patch.object(TaskInstance, "handle_failure")
- def test_execute_on_failure_callbacks_without_dag(self, mock_ti_handle_failure, has_serialized_dag):
- dagbag = DagBag(dag_folder="/dev/null", include_examples=True, read_dags_from_db=False)
- with create_session() as session:
- session.query(TaskInstance).delete()
- dag = dagbag.get_dag("example_branch_operator")
- assert dag is not None
- dag.sync_to_db()
- dagrun = dag.create_dagrun(
- state=DagRunState.RUNNING,
- logical_date=DEFAULT_DATE,
- run_type=DagRunType.SCHEDULED,
- data_interval=dag.infer_automated_data_interval(DEFAULT_DATE),
- run_after=DEFAULT_DATE,
- triggered_by=DagRunTriggeredByType.TEST,
- session=session,
- )
- task = dag.get_task(task_id="run_this_first")
- ti = TaskInstance(task, run_id=dagrun.run_id, state=TaskInstanceState.QUEUED)
- session.add(ti)
-
- if has_serialized_dag:
- assert SerializedDagModel.write_dag(dag, bundle_name="testing", session=session) is True
- session.flush()
-
- requests = [TaskCallbackRequest(full_filepath="A", ti=ti, msg="Message")]
- self._process_file(dag.fileloc, requests)
- mock_ti_handle_failure.assert_called_once_with(
- error="Message", test_mode=conf.getboolean("core", "unit_test_mode"), session=session
- )
-
def test_dagbag_import_errors_captured(self, spy_agency: SpyAgency):
@spy_agency.spy_for(DagBag.collect_dags, owner=DagBag)
def fake_collect_dags(dagbag: DagBag, *args, **kwargs):
@@ -554,10 +523,7 @@ def test_parse_file_with_dag_callbacks(spy_agency):
assert called is True
-@pytest.mark.xfail(reason="TODO: AIP-72: Task level callbacks not yet supported")
def test_parse_file_with_task_callbacks(spy_agency):
- from airflow import DAG
-
called = False
def on_failure(context):
@@ -572,15 +538,283 @@ def test_parse_file_with_task_callbacks(spy_agency):
spy_agency.spy_on(DagBag.collect_dags, call_fake=fake_collect_dags, owner=DagBag)
+ # Create a minimal TaskInstance for the request
+ ti_data = TIDataModel(
+ id=uuid.uuid4(),
+ dag_id="a",
+ task_id="b",
+ run_id="test_run",
+ map_index=-1,
+ try_number=1,
+ dag_version_id=uuid.uuid4(),
+ )
+
requests = [
TaskCallbackRequest(
filepath="A",
msg="Message",
- ti=None,
+ ti=ti_data,
bundle_name="testing",
bundle_version=None,
)
]
- _parse_file(DagFileParseRequest(file="A", callback_requests=requests), log=structlog.get_logger())
+ _parse_file(
+ DagFileParseRequest(file="A", bundle_path="test", callback_requests=requests),
+ log=structlog.get_logger(),
+ )
assert called is True
+
+
+class TestExecuteTaskCallbacks:
+ """Test the _execute_task_callbacks function"""
+
+ def test_execute_task_callbacks_failure_callback(self, spy_agency):
+ """Test _execute_task_callbacks executes failure callbacks"""
+ called = False
+ context_received = None
+
+ def on_failure(context):
+ nonlocal called, context_received
+ called = True
+ context_received = context
+
+ with DAG(dag_id="test_dag") as dag:
+ BaseOperator(task_id="test_task", on_failure_callback=on_failure)
+
+ def fake_collect_dags(self, *args, **kwargs):
+ self.dags[dag.dag_id] = dag
+
+ spy_agency.spy_on(DagBag.collect_dags, call_fake=fake_collect_dags, owner=DagBag)
+
+ dagbag = DagBag()
+ dagbag.collect_dags()
+
+ ti_data = TIDataModel(
+ id=uuid.uuid4(),
+ dag_id="test_dag",
+ task_id="test_task",
+ run_id="test_run",
+ try_number=1,
+ dag_version_id=uuid.uuid4(),
+ )
+
+ request = TaskCallbackRequest(
+ filepath="test.py",
+ msg="Task failed",
+ ti=ti_data,
+ bundle_name="testing",
+ bundle_version=None,
+ task_callback_type=TaskInstanceState.FAILED,
+ )
+
+ log = structlog.get_logger()
+ _execute_task_callbacks(dagbag, request, log)
+
+ assert called is True
+ assert context_received is not None
+ assert context_received["dag"] == dag
+ assert "ti" in context_received
+
+ def test_execute_task_callbacks_retry_callback(self, spy_agency):
+ """Test _execute_task_callbacks executes retry callbacks"""
+ called = False
+ context_received = None
+
+ def on_retry(context):
+ nonlocal called, context_received
+ called = True
+ context_received = context
+
+ with DAG(dag_id="test_dag") as dag:
+ BaseOperator(task_id="test_task", on_retry_callback=on_retry)
+
+ def fake_collect_dags(self, *args, **kwargs):
+ self.dags[dag.dag_id] = dag
+
+ spy_agency.spy_on(DagBag.collect_dags, call_fake=fake_collect_dags, owner=DagBag)
+
+ dagbag = DagBag()
+ dagbag.collect_dags()
+
+ ti_data = TIDataModel(
+ id=uuid.uuid4(),
+ dag_id="test_dag",
+ task_id="test_task",
+ run_id="test_run",
+ map_index=-1,
+ try_number=1,
+ dag_version_id=uuid.uuid4(),
+ state=TaskInstanceState.UP_FOR_RETRY,
+ )
+
+ request = TaskCallbackRequest(
+ filepath="test.py",
+ msg="Task retrying",
+ ti=ti_data,
+ bundle_name="testing",
+ bundle_version=None,
+ task_callback_type=TaskInstanceState.UP_FOR_RETRY,
+ )
+
+ log = structlog.get_logger()
+ _execute_task_callbacks(dagbag, request, log)
+
+ assert called is True
+ assert context_received is not None
+ assert context_received["dag"] == dag
+ assert "ti" in context_received
+
+ def test_execute_task_callbacks_with_context_from_server(self, spy_agency):
+ """Test _execute_task_callbacks with context_from_server creates full context"""
+ called = False
+ context_received = None
+
+ def on_failure(context):
+ nonlocal called, context_received
+ called = True
+ context_received = context
+
+ with DAG(dag_id="test_dag") as dag:
+ BaseOperator(task_id="test_task", on_failure_callback=on_failure)
+
+ def fake_collect_dags(self, *args, **kwargs):
+ self.dags[dag.dag_id] = dag
+
+ spy_agency.spy_on(DagBag.collect_dags, call_fake=fake_collect_dags, owner=DagBag)
+
+ dagbag = DagBag()
+ dagbag.collect_dags()
+
+ dag_run = DagRun(
+ dag_id="test_dag",
+ run_id="test_run",
+ logical_date=timezone.utcnow(),
+ start_date=timezone.utcnow(),
+ run_type="manual",
+ )
+ dag_run.run_after = timezone.utcnow()
+
+ ti_data = TIDataModel(
+ id=uuid.uuid4(),
+ dag_id="test_dag",
+ task_id="test_task",
+ run_id="test_run",
+ try_number=1,
+ dag_version_id=uuid.uuid4(),
+ )
+
+ context_from_server = TIRunContext(
+ dag_run=dag_run,
+ max_tries=3,
+ )
+
+ request = TaskCallbackRequest(
+ filepath="test.py",
+ msg="Task failed",
+ ti=ti_data,
+ bundle_name="testing",
+ bundle_version=None,
+ task_callback_type=TaskInstanceState.FAILED,
+ context_from_server=context_from_server,
+ )
+
+ log = structlog.get_logger()
+ _execute_task_callbacks(dagbag, request, log)
+
+ assert called is True
+ assert context_received is not None
+ # When context_from_server is provided, we get a full RuntimeTaskInstance context
+ assert "dag_run" in context_received
+ assert "logical_date" in context_received
+
+ def test_execute_task_callbacks_not_failure_callback(self, spy_agency):
+ """Test _execute_task_callbacks when request is not a failure callback"""
+ called = False
+
+ def on_failure(context):
+ nonlocal called
+ called = True
+
+ with DAG(dag_id="test_dag") as dag:
+ BaseOperator(task_id="test_task", on_failure_callback=on_failure)
+
+ def fake_collect_dags(self, *args, **kwargs):
+ self.dags[dag.dag_id] = dag
+
+ spy_agency.spy_on(DagBag.collect_dags, call_fake=fake_collect_dags, owner=DagBag)
+
+ dagbag = DagBag()
+ dagbag.collect_dags()
+
+ ti_data = TIDataModel(
+ id=uuid.uuid4(),
+ dag_id="test_dag",
+ task_id="test_task",
+ run_id="test_run",
+ try_number=1,
+ dag_version_id=uuid.uuid4(),
+ state=TaskInstanceState.SUCCESS,
+ )
+
+ request = TaskCallbackRequest(
+ filepath="test.py",
+ msg="Task succeeded",
+ ti=ti_data,
+ bundle_name="testing",
+ bundle_version=None,
+ task_callback_type=TaskInstanceState.SUCCESS,
+ )
+
+ log = structlog.get_logger()
+ _execute_task_callbacks(dagbag, request, log)
+
+ # Should not call the callback since it's not a failure callback
+ assert called is False
+
+ def test_execute_task_callbacks_multiple_callbacks(self, spy_agency):
+ """Test _execute_task_callbacks with multiple callbacks"""
+ call_count = 0
+
+ def on_failure_1(context):
+ nonlocal call_count
+ call_count += 1
+
+ def on_failure_2(context):
+ nonlocal call_count
+ call_count += 1
+
+ with DAG(dag_id="test_dag") as dag:
+ BaseOperator(task_id="test_task", on_failure_callback=[on_failure_1, on_failure_2])
+
+ def fake_collect_dags(self, *args, **kwargs):
+ self.dags[dag.dag_id] = dag
+
+ spy_agency.spy_on(DagBag.collect_dags, call_fake=fake_collect_dags, owner=DagBag)
+
+ dagbag = DagBag()
+ dagbag.collect_dags()
+
+ ti_data = TIDataModel(
+ id=uuid.uuid4(),
+ dag_id="test_dag",
+ task_id="test_task",
+ run_id="test_run",
+ try_number=1,
+ dag_version_id=uuid.uuid4(),
+ state=TaskInstanceState.FAILED,
+ )
+
+ request = TaskCallbackRequest(
+ filepath="test.py",
+ msg="Task failed",
+ ti=ti_data,
+ bundle_name="testing",
+ bundle_version=None,
+ task_callback_type=TaskInstanceState.FAILED,
+ )
+
+ log = structlog.get_logger()
+ _execute_task_callbacks(dagbag, request, log)
+
+ assert call_count == 2
diff --git a/airflow-core/tests/unit/jobs/test_scheduler_job.py b/airflow-core/tests/unit/jobs/test_scheduler_job.py
index bced9e15be4..30a3ba98007 100644
--- a/airflow-core/tests/unit/jobs/test_scheduler_job.py
+++ b/airflow-core/tests/unit/jobs/test_scheduler_job.py
@@ -408,9 +408,7 @@ class TestSchedulerJob:
self.job_runner._process_executor_events(executor=executor, session=session)
ti1.refresh_from_db()
- # The state will remain in queued here and
- # will be set to failed in dag parsing process
- assert ti1.state == State.QUEUED
+ assert ti1.state == State.FAILED
mock_task_callback.assert_called_once_with(
filepath=dag.relative_fileloc,
ti=mock.ANY,
@@ -420,10 +418,11 @@ class TestSchedulerJob:
"<TaskInstance: test_process_executor_events_with_callback.dummy_task test [queued]> "
"finished with state failed, but the task instance's state attribute is queued. "
"Learn more: https://airflow.apache.org/docs/apache-airflow/stable/troubleshooting.html#task-state-changed-externally",
+ context_from_server=mock.ANY,
)
scheduler_job.executor.callback_sink.send.assert_called_once_with(task_callback)
scheduler_job.executor.callback_sink.reset_mock()
- mock_stats_incr.assert_called_once_with(
+ mock_stats_incr.assert_any_call(
"scheduler.tasks.killed_externally",
tags={
"dag_id": "test_process_executor_events_with_callback",
@@ -5880,6 +5879,11 @@ class TestSchedulerJob:
assert callback_request.ti.run_id == ti.run_id
assert callback_request.ti.map_index == ti.map_index
+ # Verify context_from_server is passed
+ assert callback_request.context_from_server is not None
+ assert callback_request.context_from_server.dag_run.logical_date == ti.dag_run.logical_date
+ assert callback_request.context_from_server.max_tries == ti.max_tries
+
@pytest.mark.usefixtures("testing_dag_bundle")
def test_task_instance_heartbeat_timeout_message(self, session, create_dagrun):
"""
@@ -5947,68 +5951,6 @@ class TestSchedulerJob:
"External Executor Id": "abcdefg",
}
- @pytest.mark.usefixtures("testing_dag_bundle")
- def test_find_task_instances_without_heartbeats_handle_failure_callbacks_are_correctly_passed_to_dag_processor(
- self, create_dagrun, session
- ):
- """
- Check that the same set of failure callbacks for task instances without heartbeats are passed to the dag
- file processors until the next task instance heartbeat timeout detection logic is invoked.
- """
- with conf_vars({("core", "load_examples"): "False"}):
- dagbag = DagBag(
- dag_folder=os.path.join(settings.DAGS_FOLDER, "test_example_bash_operator.py"),
- read_dags_from_db=False,
- )
- session.query(Job).delete()
- dag = dagbag.get_dag("test_example_bash_operator")
- DAG.bulk_write_to_db("testing", None, [dag])
- SerializedDagModel.write_dag(dag=dag, bundle_name="testing")
- data_interval = dag.infer_automated_data_interval(DEFAULT_LOGICAL_DATE)
- dag_run = create_dagrun(
- dag,
- state=DagRunState.RUNNING,
- logical_date=DEFAULT_DATE,
- run_type=DagRunType.SCHEDULED,
- data_interval=data_interval,
- )
- task = dag.get_task(task_id="run_this_last")
- dag_version_id = DagVersion.get_latest_version(dag.dag_id).id
- ti = TaskInstance(task, run_id=dag_run.run_id, state=State.RUNNING, dag_version_id=dag_version_id)
- ti.last_heartbeat_at = timezone.utcnow() - timedelta(minutes=6)
- ti.start_date = timezone.utcnow() - timedelta(minutes=10)
-
- # TODO: If there was an actual Relationship between TI and Job
- # we wouldn't need this extra commit
- session.add(ti)
- session.flush()
-
- scheduler_job = Job(executor=self.null_exec)
- self.job_runner = SchedulerJobRunner(job=scheduler_job)
-
- self.job_runner._find_and_purge_task_instances_without_heartbeats()
-
- scheduler_job.executor.callback_sink.send.assert_called_once()
-
- expected_failure_callback_requests = [
- TaskCallbackRequest(
- filepath=dag.relative_fileloc,
- ti=ti,
- msg=str(self.job_runner._generate_task_instance_heartbeat_timeout_message_details(ti)),
- bundle_name="testing",
- bundle_version=dag_run.bundle_version,
- )
- ]
- callback_requests = scheduler_job.executor.callback_sink.send.call_args.args
- assert len(callback_requests) == 1
- assert {
- task_instances_without_heartbeats.ti.id
- for task_instances_without_heartbeats in expected_failure_callback_requests
- } == {result.ti.id for result in callback_requests}
- expected_failure_callback_requests[0].ti = None
- callback_requests[0].ti = None
- assert expected_failure_callback_requests[0] == callback_requests[0]
-
@mock.patch.object(settings, "USE_JOB_SCHEDULE", False)
def run_scheduler_until_dagrun_terminal(self):
"""
@@ -6519,6 +6461,73 @@ class TestSchedulerJob:
for i in range(100):
assert f"it's duplicate {i}" in dag_warning.message
+ def test_scheduler_passes_context_from_server_on_heartbeat_timeout(self, dag_maker, session):
+ """Test that scheduler passes context_from_server when handling heartbeat timeouts."""
+ with dag_maker(dag_id="test_dag", session=session):
+ EmptyOperator(task_id="test_task")
+
+ dag_run = dag_maker.create_dagrun(run_id="test_run", state=DagRunState.RUNNING)
+
+ mock_executor = MagicMock()
+ scheduler_job = Job(executor=mock_executor)
+ self.job_runner = SchedulerJobRunner(scheduler_job)
+
+ # Create a task instance that appears to be running but hasn't heartbeat
+ ti = dag_run.get_task_instance(task_id="test_task")
+ ti.state = TaskInstanceState.RUNNING
+ ti.queued_by_job_id = scheduler_job.id
+ # Set last_heartbeat_at to a time that would trigger timeout
+ ti.last_heartbeat_at = timezone.utcnow() - timedelta(seconds=600) # 10 minutes ago
+ session.merge(ti)
+ session.commit()
+
+ # Run the heartbeat timeout check
+ self.job_runner._find_and_purge_task_instances_without_heartbeats()
+
+ # Verify TaskCallbackRequest was created with context_from_server
+ mock_executor.send_callback.assert_called_once()
+ callback_request = mock_executor.send_callback.call_args[0][0]
+
+ assert isinstance(callback_request, TaskCallbackRequest)
+ assert callback_request.context_from_server is not None
+ assert callback_request.context_from_server.dag_run.logical_date == dag_run.logical_date
+ assert callback_request.context_from_server.max_tries == ti.max_tries
+
+ def test_scheduler_passes_context_from_server_on_task_failure(self, dag_maker, session):
+ """Test that scheduler passes context_from_server when handling task failures."""
+ with dag_maker(dag_id="test_dag", session=session):
+ EmptyOperator(task_id="test_task", on_failure_callback=lambda: print("failure"))
+
+ dag_run = dag_maker.create_dagrun(run_id="test_run", state=DagRunState.RUNNING)
+
+ # Create a task instance that's running
+ ti = dag_run.get_task_instance(task_id="test_task")
+ ti.state = TaskInstanceState.RUNNING
+ session.merge(ti)
+ session.commit()
+
+ # Mock the executor to simulate a task failure
+ mock_executor = MagicMock(spec=BaseExecutor)
+ mock_executor.has_task = mock.MagicMock(return_value=False)
+ scheduler_job = Job(executor=mock_executor)
+ self.job_runner = SchedulerJobRunner(scheduler_job)
+
+ # Simulate executor reporting task as failed
+ executor_event = {ti.key: (TaskInstanceState.FAILED, None)}
+ mock_executor.get_event_buffer.return_value = executor_event
+
+ # Process the executor events
+ self.job_runner._process_executor_events(mock_executor, session)
+
+ # Verify TaskCallbackRequest was created with context_from_server
+ mock_executor.send_callback.assert_called_once()
+ callback_request = mock_executor.send_callback.call_args[0][0]
+
+ assert isinstance(callback_request, TaskCallbackRequest)
+ assert callback_request.context_from_server is not None
+ assert callback_request.context_from_server.dag_run.logical_date == dag_run.logical_date
+ assert callback_request.context_from_server.max_tries == ti.max_tries
+
@pytest.mark.need_serialized_dag
def test_schedule_dag_run_with_upstream_skip(dag_maker, session):
diff --git a/task-sdk/src/airflow/sdk/api/datamodels/_generated.py b/task-sdk/src/airflow/sdk/api/datamodels/_generated.py
index ac1e51d5e55..1dabd8c9022 100644
--- a/task-sdk/src/airflow/sdk/api/datamodels/_generated.py
+++ b/task-sdk/src/airflow/sdk/api/datamodels/_generated.py
@@ -518,7 +518,7 @@ class TIRunContext(BaseModel):
next_method: Annotated[str | None, Field(title="Next Method")] = None
next_kwargs: Annotated[dict[str, Any] | str | None, Field(title="Next Kwargs")] = None
xcom_keys_to_clear: Annotated[list[str] | None, Field(title="Xcom Keys To Clear")] = None
- should_retry: Annotated[bool, Field(title="Should Retry")]
+ should_retry: Annotated[bool | None, Field(title="Should Retry")] = False
class TITerminalStatePayload(BaseModel):
(airflow) 06/12: [v3-0-test] Make the update_installer_and_pre_commit script more resilient to re matches (#52960) (#52961)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit b494abdc875f677f207306dee68d0e59da6c2a8f
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Mon Jul 7 08:33:09 2025 +0200
[v3-0-test] Make the update_installer_and_pre_commit script more resilient to re matches (#52960) (#52961)
(cherry picked from commit 8e5c284913f550b5b583e06f400f4e761c87fbd5)
Co-authored-by: Amogh Desai <am...@gmail.com>
---
scripts/ci/pre_commit/update_installers_and_pre_commit.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/ci/pre_commit/update_installers_and_pre_commit.py b/scripts/ci/pre_commit/update_installers_and_pre_commit.py
index 34b6a0b85ff..612acdaaa2f 100755
--- a/scripts/ci/pre_commit/update_installers_and_pre_commit.py
+++ b/scripts/ci/pre_commit/update_installers_and_pre_commit.py
@@ -96,8 +96,8 @@ UV_PATTERNS: list[tuple[re.Pattern, Quoting]] = [
(re.compile(r"(AIRFLOW_UV_VERSION=)([0-9.]+)"), Quoting.UNQUOTED),
(re.compile(r"(uv>=)([0-9.]+)"), Quoting.UNQUOTED),
(re.compile(r"(AIRFLOW_UV_VERSION = )(\"[0-9.]+\")"), Quoting.DOUBLE_QUOTED),
- (re.compile(r"(UV_VERSION = )(\"[0-9.]+\")"), Quoting.DOUBLE_QUOTED),
- (re.compile(r"(UV_VERSION=)(\"[0-9.]+\")"), Quoting.DOUBLE_QUOTED),
+ (re.compile(r"^(\s*UV_VERSION = )(\"[0-9.]+\")", re.MULTILINE), Quoting.DOUBLE_QUOTED),
+ (re.compile(r"^(\s*UV_VERSION=)(\"[0-9.]+\")", re.MULTILINE), Quoting.DOUBLE_QUOTED),
(re.compile(r"(\| *`AIRFLOW_UV_VERSION` *\| *)(`[0-9.]+`)( *\|)"), Quoting.REVERSE_SINGLE_QUOTED),
(
re.compile(
(airflow) 07/12: Remove upper-binding for "python-requires" (#52980) (#52984)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 1b87286f24a086d96f68d9336d8f6c5d461e17ea
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Jul 7 18:52:10 2025 +0200
Remove upper-binding for "python-requires" (#52980) (#52984)
Follow up after #52967 -> from later discussions it turned out that
it's not really the ~= that is wrong and ambiguous, but that just
upper-binding of Python version is generally considered as a bad
idea - and it's not Astral's view but it's general consensus that
upper-binding of "python-requires" is bad. Since ~= implies
upper-binding, simply replacing it with >= is likely the best option
we can choose.
(cherry picked from commit e9eb4813e45f9e10125ce0e5befe110512b5c5f7)
---
airflow-core/pyproject.toml | 2 +-
airflow-ctl/pyproject.toml | 2 +-
chart/pyproject.toml | 2 +-
clients/python/pyproject.toml | 2 +-
dev/breeze/README.md | 2 +-
dev/breeze/pyproject.toml | 2 +-
dev/breeze/src/airflow_breeze/utils/packages.py | 4 +-
dev/breeze/uv.lock | 2 +-
dev/pyproject.toml | 2 +-
docker-stack-docs/pyproject.toml | 2 +-
docker-tests/pyproject.toml | 2 +-
helm-tests/pyproject.toml | 2 +-
kubernetes-tests/pyproject.toml | 2 +-
providers-summary-docs/pyproject.toml | 2 +-
providers-summary-docs/uv.lock | 2 +-
providers/airbyte/pyproject.toml | 2 +-
.../src/airflow/providers/airbyte/__init__.py | 4 +-
providers/alibaba/pyproject.toml | 2 +-
.../src/airflow/providers/alibaba/__init__.py | 4 +-
providers/amazon/README.rst | 46 ++++---
providers/amazon/pyproject.toml | 2 +-
.../src/airflow/providers/amazon/__init__.py | 4 +-
providers/apache/beam/README.rst | 2 +-
providers/apache/beam/pyproject.toml | 2 +-
providers/apache/cassandra/pyproject.toml | 2 +-
.../airflow/providers/apache/cassandra/__init__.py | 4 +-
providers/apache/drill/pyproject.toml | 2 +-
.../src/airflow/providers/apache/drill/__init__.py | 4 +-
providers/apache/druid/README.rst | 2 +-
providers/apache/druid/pyproject.toml | 2 +-
.../src/airflow/providers/apache/druid/__init__.py | 4 +-
providers/apache/flink/pyproject.toml | 2 +-
.../src/airflow/providers/apache/flink/__init__.py | 4 +-
providers/apache/hdfs/pyproject.toml | 2 +-
.../src/airflow/providers/apache/hdfs/__init__.py | 4 +-
providers/apache/hive/pyproject.toml | 2 +-
.../src/airflow/providers/apache/hive/__init__.py | 4 +-
providers/apache/iceberg/pyproject.toml | 2 +-
.../airflow/providers/apache/iceberg/__init__.py | 4 +-
providers/apache/impala/pyproject.toml | 2 +-
.../airflow/providers/apache/impala/__init__.py | 4 +-
providers/apache/kafka/pyproject.toml | 2 +-
.../src/airflow/providers/apache/kafka/__init__.py | 4 +-
providers/apache/kylin/pyproject.toml | 2 +-
.../src/airflow/providers/apache/kylin/__init__.py | 4 +-
providers/apache/livy/pyproject.toml | 2 +-
.../src/airflow/providers/apache/livy/__init__.py | 4 +-
providers/apache/pig/pyproject.toml | 2 +-
.../src/airflow/providers/apache/pig/__init__.py | 4 +-
providers/apache/pinot/pyproject.toml | 2 +-
.../src/airflow/providers/apache/pinot/__init__.py | 4 +-
providers/apache/spark/README.rst | 2 +-
providers/apache/spark/pyproject.toml | 2 +-
.../src/airflow/providers/apache/spark/__init__.py | 4 +-
providers/apprise/README.rst | 2 +-
providers/apprise/pyproject.toml | 2 +-
.../src/airflow/providers/apprise/__init__.py | 4 +-
providers/arangodb/pyproject.toml | 2 +-
.../src/airflow/providers/arangodb/__init__.py | 4 +-
providers/asana/pyproject.toml | 2 +-
.../asana/src/airflow/providers/asana/__init__.py | 4 +-
providers/atlassian/jira/README.rst | 2 +-
providers/atlassian/jira/pyproject.toml | 2 +-
.../airflow/providers/atlassian/jira/__init__.py | 4 +-
providers/celery/pyproject.toml | 2 +-
.../src/airflow/providers/celery/__init__.py | 4 +-
providers/cloudant/pyproject.toml | 2 +-
.../src/airflow/providers/cloudant/__init__.py | 4 +-
providers/cncf/kubernetes/pyproject.toml | 2 +-
.../airflow/providers/cncf/kubernetes/__init__.py | 4 +-
providers/cohere/pyproject.toml | 2 +-
.../src/airflow/providers/cohere/__init__.py | 4 +-
providers/common/compat/pyproject.toml | 2 +-
.../airflow/providers/common/compat/__init__.py | 4 +-
providers/common/io/pyproject.toml | 2 +-
.../io/src/airflow/providers/common/io/__init__.py | 4 +-
providers/common/messaging/README.rst | 27 +---
providers/common/messaging/pyproject.toml | 2 +-
providers/common/sql/pyproject.toml | 2 +-
.../src/airflow/providers/common/sql/__init__.py | 4 +-
providers/databricks/README.rst | 2 +-
providers/databricks/pyproject.toml | 2 +-
providers/datadog/pyproject.toml | 2 +-
.../src/airflow/providers/datadog/__init__.py | 4 +-
providers/dbt/cloud/pyproject.toml | 2 +-
.../src/airflow/providers/dbt/cloud/__init__.py | 4 +-
providers/dingding/pyproject.toml | 2 +-
.../src/airflow/providers/dingding/__init__.py | 4 +-
providers/discord/README.rst | 2 +-
providers/discord/pyproject.toml | 2 +-
.../src/airflow/providers/discord/__init__.py | 4 +-
providers/docker/pyproject.toml | 2 +-
.../src/airflow/providers/docker/__init__.py | 4 +-
providers/edge3/pyproject.toml | 2 +-
providers/elasticsearch/pyproject.toml | 2 +-
.../airflow/providers/elasticsearch/__init__.py | 4 +-
providers/exasol/pyproject.toml | 2 +-
.../src/airflow/providers/exasol/__init__.py | 4 +-
providers/fab/README.rst | 10 +-
providers/fab/pyproject.toml | 2 +-
providers/facebook/pyproject.toml | 2 +-
.../src/airflow/providers/facebook/__init__.py | 4 +-
providers/ftp/pyproject.toml | 2 +-
.../ftp/src/airflow/providers/ftp/__init__.py | 4 +-
providers/git/pyproject.toml | 2 +-
providers/github/pyproject.toml | 2 +-
.../src/airflow/providers/github/__init__.py | 4 +-
providers/google/README.rst | 147 +++++++++++----------
providers/google/pyproject.toml | 2 +-
providers/grpc/pyproject.toml | 2 +-
.../grpc/src/airflow/providers/grpc/__init__.py | 4 +-
providers/hashicorp/pyproject.toml | 2 +-
.../src/airflow/providers/hashicorp/__init__.py | 4 +-
providers/http/pyproject.toml | 2 +-
.../http/src/airflow/providers/http/__init__.py | 4 +-
providers/imap/pyproject.toml | 2 +-
.../imap/src/airflow/providers/imap/__init__.py | 4 +-
providers/influxdb/pyproject.toml | 2 +-
.../src/airflow/providers/influxdb/__init__.py | 4 +-
providers/jdbc/pyproject.toml | 2 +-
.../jdbc/src/airflow/providers/jdbc/__init__.py | 4 +-
providers/jenkins/pyproject.toml | 2 +-
.../src/airflow/providers/jenkins/__init__.py | 4 +-
providers/microsoft/azure/README.rst | 67 +++++-----
providers/microsoft/azure/pyproject.toml | 2 +-
.../airflow/providers/microsoft/azure/__init__.py | 4 +-
providers/microsoft/mssql/pyproject.toml | 2 +-
.../airflow/providers/microsoft/mssql/__init__.py | 4 +-
providers/microsoft/psrp/pyproject.toml | 2 +-
.../airflow/providers/microsoft/psrp/__init__.py | 4 +-
providers/microsoft/winrm/pyproject.toml | 2 +-
.../airflow/providers/microsoft/winrm/__init__.py | 4 +-
providers/mongo/pyproject.toml | 2 +-
.../mongo/src/airflow/providers/mongo/__init__.py | 4 +-
providers/mysql/README.rst | 12 +-
providers/mysql/pyproject.toml | 2 +-
.../mysql/src/airflow/providers/mysql/__init__.py | 4 +-
providers/neo4j/pyproject.toml | 2 +-
.../neo4j/src/airflow/providers/neo4j/__init__.py | 4 +-
providers/odbc/pyproject.toml | 2 +-
.../odbc/src/airflow/providers/odbc/__init__.py | 4 +-
providers/openai/pyproject.toml | 2 +-
.../src/airflow/providers/openai/__init__.py | 4 +-
providers/openfaas/pyproject.toml | 2 +-
.../src/airflow/providers/openfaas/__init__.py | 4 +-
providers/openlineage/pyproject.toml | 2 +-
.../src/airflow/providers/openlineage/__init__.py | 4 +-
providers/opensearch/pyproject.toml | 2 +-
.../src/airflow/providers/opensearch/__init__.py | 4 +-
providers/opsgenie/README.rst | 2 +-
providers/opsgenie/pyproject.toml | 2 +-
.../src/airflow/providers/opsgenie/__init__.py | 4 +-
providers/oracle/pyproject.toml | 2 +-
.../src/airflow/providers/oracle/__init__.py | 4 +-
providers/pagerduty/README.rst | 2 +-
providers/pagerduty/pyproject.toml | 2 +-
.../src/airflow/providers/pagerduty/__init__.py | 4 +-
providers/papermill/pyproject.toml | 2 +-
.../src/airflow/providers/papermill/__init__.py | 4 +-
providers/pgvector/pyproject.toml | 2 +-
.../src/airflow/providers/pgvector/__init__.py | 4 +-
providers/pinecone/pyproject.toml | 2 +-
.../src/airflow/providers/pinecone/__init__.py | 4 +-
providers/postgres/pyproject.toml | 2 +-
.../src/airflow/providers/postgres/__init__.py | 4 +-
providers/presto/pyproject.toml | 2 +-
.../src/airflow/providers/presto/__init__.py | 4 +-
providers/qdrant/pyproject.toml | 2 +-
.../src/airflow/providers/qdrant/__init__.py | 4 +-
providers/redis/pyproject.toml | 2 +-
.../redis/src/airflow/providers/redis/__init__.py | 4 +-
providers/salesforce/pyproject.toml | 2 +-
.../src/airflow/providers/salesforce/__init__.py | 4 +-
providers/samba/pyproject.toml | 2 +-
.../samba/src/airflow/providers/samba/__init__.py | 4 +-
providers/segment/pyproject.toml | 2 +-
.../src/airflow/providers/segment/__init__.py | 4 +-
providers/sendgrid/pyproject.toml | 2 +-
providers/sftp/pyproject.toml | 2 +-
.../sftp/src/airflow/providers/sftp/__init__.py | 4 +-
providers/singularity/pyproject.toml | 2 +-
.../src/airflow/providers/singularity/__init__.py | 4 +-
providers/slack/README.rst | 2 +-
providers/slack/pyproject.toml | 2 +-
.../slack/src/airflow/providers/slack/__init__.py | 4 +-
providers/smtp/README.rst | 2 +-
providers/smtp/pyproject.toml | 2 +-
.../smtp/src/airflow/providers/smtp/__init__.py | 4 +-
providers/snowflake/README.rst | 2 +-
providers/snowflake/pyproject.toml | 2 +-
providers/sqlite/pyproject.toml | 2 +-
.../src/airflow/providers/sqlite/__init__.py | 4 +-
providers/ssh/pyproject.toml | 2 +-
.../ssh/src/airflow/providers/ssh/__init__.py | 4 +-
providers/standard/pyproject.toml | 2 +-
.../src/airflow/providers/standard/__init__.py | 4 +-
providers/tableau/pyproject.toml | 2 +-
.../src/airflow/providers/tableau/__init__.py | 4 +-
providers/telegram/pyproject.toml | 2 +-
.../src/airflow/providers/telegram/__init__.py | 4 +-
providers/teradata/pyproject.toml | 2 +-
.../src/airflow/providers/teradata/__init__.py | 4 +-
providers/trino/pyproject.toml | 2 +-
.../trino/src/airflow/providers/trino/__init__.py | 4 +-
providers/vertica/pyproject.toml | 2 +-
.../src/airflow/providers/vertica/__init__.py | 4 +-
providers/weaviate/pyproject.toml | 2 +-
.../src/airflow/providers/weaviate/__init__.py | 4 +-
providers/yandex/pyproject.toml | 2 +-
.../src/airflow/providers/yandex/__init__.py | 4 +-
providers/ydb/pyproject.toml | 2 +-
.../ydb/src/airflow/providers/ydb/__init__.py | 4 +-
providers/zendesk/pyproject.toml | 2 +-
.../src/airflow/providers/zendesk/__init__.py | 4 +-
pyproject.toml | 2 +-
task-sdk/pyproject.toml | 2 +-
216 files changed, 446 insertions(+), 457 deletions(-)
diff --git a/airflow-core/pyproject.toml b/airflow-core/pyproject.toml
index afa52e1d53b..8af8cd3b6fd 100644
--- a/airflow-core/pyproject.toml
+++ b/airflow-core/pyproject.toml
@@ -35,7 +35,7 @@ name = "apache-airflow-core"
description = "Core packages for Apache Airflow, schedule and API server"
readme = { file = "README.md", content-type = "text/markdown" }
license-files.globs = ["LICENSE", "3rd-party-licenses/*.txt", "NOTICE"]
-requires-python = "~=3.9,<3.13"
+requires-python = ">=3.9,!=3.13"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/airflow-ctl/pyproject.toml b/airflow-ctl/pyproject.toml
index 02f6919aee2..e0cf57a3cca 100644
--- a/airflow-ctl/pyproject.toml
+++ b/airflow-ctl/pyproject.toml
@@ -20,7 +20,7 @@ name = "apache-airflow-ctl"
dynamic = ["version"]
description = "Apache Airflow command line tool for communicating with an Apache Airflow, using the API."
readme = { file = "README.md", content-type = "text/markdown" }
-requires-python = ">=3.9, <3.13"
+requires-python = ">=3.9, !=3.13"
dependencies = [
# TODO there could be still missing deps such as airflow-core
"argcomplete>=1.10",
diff --git a/chart/pyproject.toml b/chart/pyproject.toml
index 30813c38db2..e6839cf0078 100644
--- a/chart/pyproject.toml
+++ b/chart/pyproject.toml
@@ -24,7 +24,7 @@ build-backend = "hatchling.build"
[project]
name = "apache-airflow-helm-chart"
description = "Programmatically author, schedule and monitor data pipelines"
-requires-python = "~=3.9,<3.13"
+requires-python = ">=3.9,!=3.13"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/clients/python/pyproject.toml b/clients/python/pyproject.toml
index ec4baeb1f55..896cc8a5f5e 100644
--- a/clients/python/pyproject.toml
+++ b/clients/python/pyproject.toml
@@ -25,7 +25,7 @@ dynamic = ["version"]
description = "Apache Airflow API (Stable)"
readme = "README.md"
license-files.globs = ["LICENSE", "NOTICE"]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/dev/breeze/README.md b/dev/breeze/README.md
index a754f22f20d..95c6ed78be7 100644
--- a/dev/breeze/README.md
+++ b/dev/breeze/README.md
@@ -135,6 +135,6 @@ PLEASE DO NOT MODIFY THE HASH BELOW! IT IS AUTOMATICALLY UPDATED BY PRE-COMMIT.
---------------------------------------------------------------------------------------------------------
-Package config hash: 63f150c89cf47e8a20a044eb3b80cbd900900c11b2c40c2d0c3e70c92b7ca6c003cbde7974901b677410844b209b3eb1c09008b76ea6b8359f606c5a64164d1f
+Package config hash: 45d0d735430fd1c151071f8ce20ac90b65d5bdc4317678ce1a03004a9f0853745a7b8cbf70520fc55afac15f742e961eeb7f5ffbf948f1b230f5cba788fceb65
---------------------------------------------------------------------------------------------------------
diff --git a/dev/breeze/pyproject.toml b/dev/breeze/pyproject.toml
index 4e88e19315f..ada1c0b0b21 100644
--- a/dev/breeze/pyproject.toml
+++ b/dev/breeze/pyproject.toml
@@ -41,7 +41,7 @@ classifiers = [
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
dependencies = [
"black>=23.11.0",
diff --git a/dev/breeze/src/airflow_breeze/utils/packages.py b/dev/breeze/src/airflow_breeze/utils/packages.py
index 08b63372f0d..6e6206695c6 100644
--- a/dev/breeze/src/airflow_breeze/utils/packages.py
+++ b/dev/breeze/src/airflow_breeze/utils/packages.py
@@ -592,7 +592,7 @@ def get_min_airflow_version(provider_id: str) -> str:
def get_python_requires(provider_id: str) -> str:
- python_requires = "~=3.9"
+ python_requires = ">=3.9"
provider_details = get_provider_details(provider_id=provider_id)
for p in provider_details.excluded_python_versions:
python_requires += f", !={p}"
@@ -646,7 +646,7 @@ def get_provider_jinja_context(
cross_providers_dependencies = get_cross_provider_dependent_packages(provider_id=provider_id)
# Most providers require the same python versions, but some may have exclusions
- requires_python_version: str = f"~={DEFAULT_PYTHON_MAJOR_MINOR_VERSION}"
+ requires_python_version: str = f">={DEFAULT_PYTHON_MAJOR_MINOR_VERSION}"
for excluded_python_version in provider_details.excluded_python_versions:
requires_python_version += f",!={excluded_python_version}"
diff --git a/dev/breeze/uv.lock b/dev/breeze/uv.lock
index 6ba16eb6dbc..bff3056de93 100644
--- a/dev/breeze/uv.lock
+++ b/dev/breeze/uv.lock
@@ -1,6 +1,6 @@
version = 1
revision = 1
-requires-python = ">=3.9, <4"
+requires-python = ">=3.9"
resolution-markers = [
"python_full_version == '3.12.*'",
"python_full_version == '3.11.*'",
diff --git a/dev/pyproject.toml b/dev/pyproject.toml
index 48e83b18894..a2cde925922 100644
--- a/dev/pyproject.toml
+++ b/dev/pyproject.toml
@@ -26,7 +26,7 @@ description = "Development tools for Apache Airflow"
classifiers = [
"Private :: Do Not Upload",
]
-requires-python = "~=3.9,<3.13"
+requires-python = ">=3.9,!=3.13"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/docker-stack-docs/pyproject.toml b/docker-stack-docs/pyproject.toml
index 2258fa88754..b74dd62242d 100644
--- a/docker-stack-docs/pyproject.toml
+++ b/docker-stack-docs/pyproject.toml
@@ -24,7 +24,7 @@ build-backend = "hatchling.build"
[project]
name = "apache-airflow-docker-stack"
description = "Programmatically author, schedule and monitor data pipelines"
-requires-python = "~=3.9,<3.13"
+requires-python = ">=3.9,!=3.13"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/docker-tests/pyproject.toml b/docker-tests/pyproject.toml
index c242dc53e27..2ab09b61cdd 100644
--- a/docker-tests/pyproject.toml
+++ b/docker-tests/pyproject.toml
@@ -26,7 +26,7 @@ description = "Docker tests for Apache Airflow"
classifiers = [
"Private :: Do Not Upload",
]
-requires-python = "~=3.9,<3.13"
+requires-python = ">=3.9,!=3.13"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/helm-tests/pyproject.toml b/helm-tests/pyproject.toml
index b370c400b61..f430ac70174 100644
--- a/helm-tests/pyproject.toml
+++ b/helm-tests/pyproject.toml
@@ -26,7 +26,7 @@ description = "Helm tests for Apache Airflow"
classifiers = [
"Private :: Do Not Upload",
]
-requires-python = "~=3.9,<3.13"
+requires-python = ">=3.9,!=3.13"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/kubernetes-tests/pyproject.toml b/kubernetes-tests/pyproject.toml
index 0c201884c81..aa4f5b54100 100644
--- a/kubernetes-tests/pyproject.toml
+++ b/kubernetes-tests/pyproject.toml
@@ -24,7 +24,7 @@ description = "Kubernetes tests for Apache Airflow"
classifiers = [
"Private :: Do Not Upload",
]
-requires-python = "~=3.9,<3.13"
+requires-python = ">=3.9,!=3.13"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/providers-summary-docs/pyproject.toml b/providers-summary-docs/pyproject.toml
index 4ec9fa33e71..5f9b76c73a3 100644
--- a/providers-summary-docs/pyproject.toml
+++ b/providers-summary-docs/pyproject.toml
@@ -24,7 +24,7 @@ build-backend = "hatchling.build"
[project]
name = "apache-airflow-providers"
description = "Programmatically author, schedule and monitor data pipelines"
-requires-python = "~=3.9,<3.13"
+requires-python = ">=3.9,!=3.13"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/providers-summary-docs/uv.lock b/providers-summary-docs/uv.lock
index 1ae3bdba1f6..d0b245715f0 100644
--- a/providers-summary-docs/uv.lock
+++ b/providers-summary-docs/uv.lock
@@ -1,6 +1,6 @@
version = 1
revision = 1
-requires-python = ">=3.9, <3.13"
+requires-python = ">=3.10, !=3.13"
[[package]]
name = "apache-airflow-providers"
diff --git a/providers/airbyte/pyproject.toml b/providers/airbyte/pyproject.toml
index f573261181d..662c2a5db28 100644
--- a/providers/airbyte/pyproject.toml
+++ b/providers/airbyte/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/airbyte/src/airflow/providers/airbyte/__init__.py b/providers/airbyte/src/airflow/providers/airbyte/__init__.py
index 7d95a82c2a2..d8cf64567ba 100644
--- a/providers/airbyte/src/airflow/providers/airbyte/__init__.py
+++ b/providers/airbyte/src/airflow/providers/airbyte/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.0.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-airbyte:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-airbyte:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/alibaba/pyproject.toml b/providers/alibaba/pyproject.toml
index 3dd479355ed..5f7dab4df28 100644
--- a/providers/alibaba/pyproject.toml
+++ b/providers/alibaba/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/alibaba/src/airflow/providers/alibaba/__init__.py b/providers/alibaba/src/airflow/providers/alibaba/__init__.py
index 0e31e0223a3..1ddae2f1027 100644
--- a/providers/alibaba/src/airflow/providers/alibaba/__init__.py
+++ b/providers/alibaba/src/airflow/providers/alibaba/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.0.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-alibaba:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-alibaba:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/amazon/README.rst b/providers/amazon/README.rst
index 2129a2dae12..4f599c1fd26 100644
--- a/providers/amazon/README.rst
+++ b/providers/amazon/README.rst
@@ -23,7 +23,7 @@
Package ``apache-airflow-providers-amazon``
-Release: ``9.6.0``
+Release: ``9.7.0``
Amazon integration (including `Amazon Web Services (AWS) <https://aws.amazon.com/>`__).
@@ -36,7 +36,7 @@ This is a provider package for ``amazon`` provider. All classes for this provide
are in ``airflow.providers.amazon`` python package.
You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-amazon/9.6.0/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-amazon/9.7.0/>`_.
Installation
------------
@@ -54,7 +54,7 @@ Requirements
PIP package Version required
========================================== =====================
``apache-airflow`` ``>=2.9.0``
-``apache-airflow-providers-common-compat`` ``>=1.6.0``
+``apache-airflow-providers-common-compat`` ``>=1.6.1``
``apache-airflow-providers-common-sql`` ``>=1.20.0``
``apache-airflow-providers-http``
``boto3`` ``>=1.37.0``
@@ -68,6 +68,7 @@ PIP package Version required
``jmespath`` ``>=0.7.0``
``python3-saml`` ``>=1.16.0``
``xmlsec`` ``>=1.3.14,!=1.3.15``
+``lxml`` ``>=5.3.2,<5.4.0``
``sagemaker-studio`` ``>=1.0.9``
========================================== =====================
@@ -84,24 +85,25 @@ You can install such cross-provider dependencies when installing from PyPI. For
pip install apache-airflow-providers-amazon[apache.hive]
-====================================================================================================================== ===================
-Dependent package Extra
-====================================================================================================================== ===================
-`apache-airflow-providers-apache-hive <https://airflow.apache.org/docs/apache-airflow-providers-apache-hive>`_ ``apache.hive``
-`apache-airflow-providers-cncf-kubernetes <https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes>`_ ``cncf.kubernetes``
-`apache-airflow-providers-common-compat <https://airflow.apache.org/docs/apache-airflow-providers-common-compat>`_ ``common.compat``
-`apache-airflow-providers-common-sql <https://airflow.apache.org/docs/apache-airflow-providers-common-sql>`_ ``common.sql``
-`apache-airflow-providers-exasol <https://airflow.apache.org/docs/apache-airflow-providers-exasol>`_ ``exasol``
-`apache-airflow-providers-ftp <https://airflow.apache.org/docs/apache-airflow-providers-ftp>`_ ``ftp``
-`apache-airflow-providers-google <https://airflow.apache.org/docs/apache-airflow-providers-google>`_ ``google``
-`apache-airflow-providers-http <https://airflow.apache.org/docs/apache-airflow-providers-http>`_ ``http``
-`apache-airflow-providers-imap <https://airflow.apache.org/docs/apache-airflow-providers-imap>`_ ``imap``
-`apache-airflow-providers-microsoft-azure <https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure>`_ ``microsoft.azure``
-`apache-airflow-providers-mongo <https://airflow.apache.org/docs/apache-airflow-providers-mongo>`_ ``mongo``
-`apache-airflow-providers-openlineage <https://airflow.apache.org/docs/apache-airflow-providers-openlineage>`_ ``openlineage``
-`apache-airflow-providers-salesforce <https://airflow.apache.org/docs/apache-airflow-providers-salesforce>`_ ``salesforce``
-`apache-airflow-providers-ssh <https://airflow.apache.org/docs/apache-airflow-providers-ssh>`_ ``ssh``
-====================================================================================================================== ===================
+======================================================================================================================== ====================
+Dependent package Extra
+======================================================================================================================== ====================
+`apache-airflow-providers-apache-hive <https://airflow.apache.org/docs/apache-airflow-providers-apache-hive>`_ ``apache.hive``
+`apache-airflow-providers-cncf-kubernetes <https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes>`_ ``cncf.kubernetes``
+`apache-airflow-providers-common-compat <https://airflow.apache.org/docs/apache-airflow-providers-common-compat>`_ ``common.compat``
+`apache-airflow-providers-common-messaging <https://airflow.apache.org/docs/apache-airflow-providers-common-messaging>`_ ``common.messaging``
+`apache-airflow-providers-common-sql <https://airflow.apache.org/docs/apache-airflow-providers-common-sql>`_ ``common.sql``
+`apache-airflow-providers-exasol <https://airflow.apache.org/docs/apache-airflow-providers-exasol>`_ ``exasol``
+`apache-airflow-providers-ftp <https://airflow.apache.org/docs/apache-airflow-providers-ftp>`_ ``ftp``
+`apache-airflow-providers-google <https://airflow.apache.org/docs/apache-airflow-providers-google>`_ ``google``
+`apache-airflow-providers-http <https://airflow.apache.org/docs/apache-airflow-providers-http>`_ ``http``
+`apache-airflow-providers-imap <https://airflow.apache.org/docs/apache-airflow-providers-imap>`_ ``imap``
+`apache-airflow-providers-microsoft-azure <https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure>`_ ``microsoft.azure``
+`apache-airflow-providers-mongo <https://airflow.apache.org/docs/apache-airflow-providers-mongo>`_ ``mongo``
+`apache-airflow-providers-openlineage <https://airflow.apache.org/docs/apache-airflow-providers-openlineage>`_ ``openlineage``
+`apache-airflow-providers-salesforce <https://airflow.apache.org/docs/apache-airflow-providers-salesforce>`_ ``salesforce``
+`apache-airflow-providers-ssh <https://airflow.apache.org/docs/apache-airflow-providers-ssh>`_ ``ssh``
+======================================================================================================================== ====================
The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-amazon/9.6.0/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-amazon/9.7.0/changelog.html>`_.
diff --git a/providers/amazon/pyproject.toml b/providers/amazon/pyproject.toml
index efe94b5b6f6..08d2201fa42 100644
--- a/providers/amazon/pyproject.toml
+++ b/providers/amazon/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/amazon/src/airflow/providers/amazon/__init__.py b/providers/amazon/src/airflow/providers/amazon/__init__.py
index 44e1ea686f8..705ef1e9f9c 100644
--- a/providers/amazon/src/airflow/providers/amazon/__init__.py
+++ b/providers/amazon/src/airflow/providers/amazon/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "9.7.0"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-amazon:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-amazon:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/beam/README.rst b/providers/apache/beam/README.rst
index 60096440d59..3b4acc22b2d 100644
--- a/providers/apache/beam/README.rst
+++ b/providers/apache/beam/README.rst
@@ -56,7 +56,7 @@ PIP package Version required
``apache-airflow`` ``>=2.9.0``
``apache-beam`` ``>=2.53.0; python_version < "3.12"``
``apache-beam`` ``>=2.57.0; python_version >= "3.12"``
-``pyarrow`` ``>=14.0.1``
+``pyarrow`` ``>=16.1.0``
``numpy`` ``>=1.26.0``
================== ======================================
diff --git a/providers/apache/beam/pyproject.toml b/providers/apache/beam/pyproject.toml
index aefab025b24..86344bbd2d4 100644
--- a/providers/apache/beam/pyproject.toml
+++ b/providers/apache/beam/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/cassandra/pyproject.toml b/providers/apache/cassandra/pyproject.toml
index a91b25cc007..8a7133873ce 100644
--- a/providers/apache/cassandra/pyproject.toml
+++ b/providers/apache/cassandra/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/cassandra/src/airflow/providers/apache/cassandra/__init__.py b/providers/apache/cassandra/src/airflow/providers/apache/cassandra/__init__.py
index febe3bab559..ea1e77adde7 100644
--- a/providers/apache/cassandra/src/airflow/providers/apache/cassandra/__init__.py
+++ b/providers/apache/cassandra/src/airflow/providers/apache/cassandra/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.7.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-cassandra:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-cassandra:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/drill/pyproject.toml b/providers/apache/drill/pyproject.toml
index 126c479b88b..eef4f2d6b2b 100644
--- a/providers/apache/drill/pyproject.toml
+++ b/providers/apache/drill/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/drill/src/airflow/providers/apache/drill/__init__.py b/providers/apache/drill/src/airflow/providers/apache/drill/__init__.py
index e7897d6c7cd..5c26053416d 100644
--- a/providers/apache/drill/src/airflow/providers/apache/drill/__init__.py
+++ b/providers/apache/drill/src/airflow/providers/apache/drill/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.0.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-drill:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-drill:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/druid/README.rst b/providers/apache/druid/README.rst
index 951afb8990e..64cf664deb9 100644
--- a/providers/apache/druid/README.rst
+++ b/providers/apache/druid/README.rst
@@ -55,7 +55,7 @@ PIP package Version required
======================================= ==================
``apache-airflow`` ``>=2.9.0``
``apache-airflow-providers-common-sql`` ``>=1.26.0``
-``pydruid`` ``>=0.4.1``
+``pydruid`` ``>=0.6.6``
======================================= ==================
Cross provider package dependencies
diff --git a/providers/apache/druid/pyproject.toml b/providers/apache/druid/pyproject.toml
index 2e3c911e979..bb3b1382ec0 100644
--- a/providers/apache/druid/pyproject.toml
+++ b/providers/apache/druid/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/druid/src/airflow/providers/apache/druid/__init__.py b/providers/apache/druid/src/airflow/providers/apache/druid/__init__.py
index beb73c11549..4fec06672f3 100644
--- a/providers/apache/druid/src/airflow/providers/apache/druid/__init__.py
+++ b/providers/apache/druid/src/airflow/providers/apache/druid/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.1.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-druid:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-druid:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/flink/pyproject.toml b/providers/apache/flink/pyproject.toml
index c7d958e5c52..d9d17f5522e 100644
--- a/providers/apache/flink/pyproject.toml
+++ b/providers/apache/flink/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/flink/src/airflow/providers/apache/flink/__init__.py b/providers/apache/flink/src/airflow/providers/apache/flink/__init__.py
index 148ca5b33d7..b392bace8ed 100644
--- a/providers/apache/flink/src/airflow/providers/apache/flink/__init__.py
+++ b/providers/apache/flink/src/airflow/providers/apache/flink/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.6.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-flink:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-flink:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/hdfs/pyproject.toml b/providers/apache/hdfs/pyproject.toml
index 950833f074f..fdffd665fea 100644
--- a/providers/apache/hdfs/pyproject.toml
+++ b/providers/apache/hdfs/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/hdfs/src/airflow/providers/apache/hdfs/__init__.py b/providers/apache/hdfs/src/airflow/providers/apache/hdfs/__init__.py
index c9293a9ae7e..3092a59952b 100644
--- a/providers/apache/hdfs/src/airflow/providers/apache/hdfs/__init__.py
+++ b/providers/apache/hdfs/src/airflow/providers/apache/hdfs/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.8.0"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-hdfs:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-hdfs:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/hive/pyproject.toml b/providers/apache/hive/pyproject.toml
index 18f7c81c58a..4546e8c4d8b 100644
--- a/providers/apache/hive/pyproject.toml
+++ b/providers/apache/hive/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/hive/src/airflow/providers/apache/hive/__init__.py b/providers/apache/hive/src/airflow/providers/apache/hive/__init__.py
index 0f545351461..011948df92f 100644
--- a/providers/apache/hive/src/airflow/providers/apache/hive/__init__.py
+++ b/providers/apache/hive/src/airflow/providers/apache/hive/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "9.0.6"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-hive:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-hive:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/iceberg/pyproject.toml b/providers/apache/iceberg/pyproject.toml
index 8c0d2f5a186..a7c426370ed 100644
--- a/providers/apache/iceberg/pyproject.toml
+++ b/providers/apache/iceberg/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/iceberg/src/airflow/providers/apache/iceberg/__init__.py b/providers/apache/iceberg/src/airflow/providers/apache/iceberg/__init__.py
index 17c5e7ce4de..22e408a837b 100644
--- a/providers/apache/iceberg/src/airflow/providers/apache/iceberg/__init__.py
+++ b/providers/apache/iceberg/src/airflow/providers/apache/iceberg/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.2.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-iceberg:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-iceberg:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/impala/pyproject.toml b/providers/apache/impala/pyproject.toml
index 7e06348922e..07108f051be 100644
--- a/providers/apache/impala/pyproject.toml
+++ b/providers/apache/impala/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/impala/src/airflow/providers/apache/impala/__init__.py b/providers/apache/impala/src/airflow/providers/apache/impala/__init__.py
index 6e6793d1cea..a2d6ab4ee88 100644
--- a/providers/apache/impala/src/airflow/providers/apache/impala/__init__.py
+++ b/providers/apache/impala/src/airflow/providers/apache/impala/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.6.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-impala:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-impala:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/kafka/pyproject.toml b/providers/apache/kafka/pyproject.toml
index 11d61285198..ea9ed3ed7ee 100644
--- a/providers/apache/kafka/pyproject.toml
+++ b/providers/apache/kafka/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/kafka/src/airflow/providers/apache/kafka/__init__.py b/providers/apache/kafka/src/airflow/providers/apache/kafka/__init__.py
index 30c68eaac67..c2ea7906c79 100644
--- a/providers/apache/kafka/src/airflow/providers/apache/kafka/__init__.py
+++ b/providers/apache/kafka/src/airflow/providers/apache/kafka/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.8.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-kafka:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-kafka:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/kylin/pyproject.toml b/providers/apache/kylin/pyproject.toml
index bca5f89a167..74114e4f93d 100644
--- a/providers/apache/kylin/pyproject.toml
+++ b/providers/apache/kylin/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/kylin/src/airflow/providers/apache/kylin/__init__.py b/providers/apache/kylin/src/airflow/providers/apache/kylin/__init__.py
index 858e9206ea1..1d882f755d9 100644
--- a/providers/apache/kylin/src/airflow/providers/apache/kylin/__init__.py
+++ b/providers/apache/kylin/src/airflow/providers/apache/kylin/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.8.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-kylin:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-kylin:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/livy/pyproject.toml b/providers/apache/livy/pyproject.toml
index fc2a709ea13..00f615d7690 100644
--- a/providers/apache/livy/pyproject.toml
+++ b/providers/apache/livy/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/livy/src/airflow/providers/apache/livy/__init__.py b/providers/apache/livy/src/airflow/providers/apache/livy/__init__.py
index d02cce4f4da..6831da90e4a 100644
--- a/providers/apache/livy/src/airflow/providers/apache/livy/__init__.py
+++ b/providers/apache/livy/src/airflow/providers/apache/livy/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.3.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-livy:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-livy:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/pig/pyproject.toml b/providers/apache/pig/pyproject.toml
index 5f18b3f43d2..f393d756fd2 100644
--- a/providers/apache/pig/pyproject.toml
+++ b/providers/apache/pig/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/pig/src/airflow/providers/apache/pig/__init__.py b/providers/apache/pig/src/airflow/providers/apache/pig/__init__.py
index c77e5ed8ef9..f60f2ca9715 100644
--- a/providers/apache/pig/src/airflow/providers/apache/pig/__init__.py
+++ b/providers/apache/pig/src/airflow/providers/apache/pig/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.6.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-pig:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-pig:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/pinot/pyproject.toml b/providers/apache/pinot/pyproject.toml
index dc8a69a9531..5cd4b644dfb 100644
--- a/providers/apache/pinot/pyproject.toml
+++ b/providers/apache/pinot/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/pinot/src/airflow/providers/apache/pinot/__init__.py b/providers/apache/pinot/src/airflow/providers/apache/pinot/__init__.py
index 3d7e50123b9..4ef978db09d 100644
--- a/providers/apache/pinot/src/airflow/providers/apache/pinot/__init__.py
+++ b/providers/apache/pinot/src/airflow/providers/apache/pinot/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.7.0"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-pinot:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-pinot:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apache/spark/README.rst b/providers/apache/spark/README.rst
index 17a103bec4f..f9548b677b8 100644
--- a/providers/apache/spark/README.rst
+++ b/providers/apache/spark/README.rst
@@ -55,7 +55,7 @@ PIP package Version required
========================================== ==================
``apache-airflow`` ``>=2.9.0``
``apache-airflow-providers-common-compat`` ``>=1.5.0``
-``pyspark`` ``>=3.1.3``
+``pyspark`` ``>=3.5.2``
``grpcio-status`` ``>=1.59.0``
========================================== ==================
diff --git a/providers/apache/spark/pyproject.toml b/providers/apache/spark/pyproject.toml
index e74f03d5409..66e2e7b7970 100644
--- a/providers/apache/spark/pyproject.toml
+++ b/providers/apache/spark/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apache/spark/src/airflow/providers/apache/spark/__init__.py b/providers/apache/spark/src/airflow/providers/apache/spark/__init__.py
index 42ce84227da..3592643370b 100644
--- a/providers/apache/spark/src/airflow/providers/apache/spark/__init__.py
+++ b/providers/apache/spark/src/airflow/providers/apache/spark/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.2.0"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apache-spark:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apache-spark:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/apprise/README.rst b/providers/apprise/README.rst
index 85fb044a935..0e2930a8f51 100644
--- a/providers/apprise/README.rst
+++ b/providers/apprise/README.rst
@@ -54,7 +54,7 @@ Requirements
PIP package Version required
========================================== ==================
``apache-airflow`` ``>=2.9.0``
-``apache-airflow-providers-common-compat`` ``>=1.6.0``
+``apache-airflow-providers-common-compat`` ``>=1.6.1``
``apprise`` ``>=1.8.0``
========================================== ==================
diff --git a/providers/apprise/pyproject.toml b/providers/apprise/pyproject.toml
index 761dc77b049..7a9e0cba30a 100644
--- a/providers/apprise/pyproject.toml
+++ b/providers/apprise/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/apprise/src/airflow/providers/apprise/__init__.py b/providers/apprise/src/airflow/providers/apprise/__init__.py
index 145ba610da6..0f2305877e8 100644
--- a/providers/apprise/src/airflow/providers/apprise/__init__.py
+++ b/providers/apprise/src/airflow/providers/apprise/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "2.0.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-apprise:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-apprise:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/arangodb/pyproject.toml b/providers/arangodb/pyproject.toml
index 4ceee3fe01a..0ea420b5a2f 100644
--- a/providers/arangodb/pyproject.toml
+++ b/providers/arangodb/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/arangodb/src/airflow/providers/arangodb/__init__.py b/providers/arangodb/src/airflow/providers/arangodb/__init__.py
index 09fa111ca8b..a2f7129478e 100644
--- a/providers/arangodb/src/airflow/providers/arangodb/__init__.py
+++ b/providers/arangodb/src/airflow/providers/arangodb/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "2.7.4"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-arangodb:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-arangodb:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/asana/pyproject.toml b/providers/asana/pyproject.toml
index 5c33af8da7b..3db7604e968 100644
--- a/providers/asana/pyproject.toml
+++ b/providers/asana/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/asana/src/airflow/providers/asana/__init__.py b/providers/asana/src/airflow/providers/asana/__init__.py
index c9ba2427f26..694840f1353 100644
--- a/providers/asana/src/airflow/providers/asana/__init__.py
+++ b/providers/asana/src/airflow/providers/asana/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "2.9.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-asana:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-asana:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/atlassian/jira/README.rst b/providers/atlassian/jira/README.rst
index 79bd93f082f..fbdbcebbdf9 100644
--- a/providers/atlassian/jira/README.rst
+++ b/providers/atlassian/jira/README.rst
@@ -54,7 +54,7 @@ Requirements
PIP package Version required
========================================== ==================
``apache-airflow`` ``>=2.9.0``
-``apache-airflow-providers-common-compat`` ``>=1.6.0``
+``apache-airflow-providers-common-compat`` ``>=1.6.1``
``atlassian-python-api`` ``>3.41.10``
========================================== ==================
diff --git a/providers/atlassian/jira/pyproject.toml b/providers/atlassian/jira/pyproject.toml
index 5ec24372c19..8f6e4bcb11b 100644
--- a/providers/atlassian/jira/pyproject.toml
+++ b/providers/atlassian/jira/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/atlassian/jira/src/airflow/providers/atlassian/jira/__init__.py b/providers/atlassian/jira/src/airflow/providers/atlassian/jira/__init__.py
index fc1e72a4435..19823e652de 100644
--- a/providers/atlassian/jira/src/airflow/providers/atlassian/jira/__init__.py
+++ b/providers/atlassian/jira/src/airflow/providers/atlassian/jira/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.0.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-atlassian-jira:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-atlassian-jira:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/celery/pyproject.toml b/providers/celery/pyproject.toml
index 8ab5c13abc0..f2400fd5b0e 100644
--- a/providers/celery/pyproject.toml
+++ b/providers/celery/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/celery/src/airflow/providers/celery/__init__.py b/providers/celery/src/airflow/providers/celery/__init__.py
index 83f6aa6e2d8..2b76829b9f4 100644
--- a/providers/celery/src/airflow/providers/celery/__init__.py
+++ b/providers/celery/src/airflow/providers/celery/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.10.6"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-celery:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-celery:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/cloudant/pyproject.toml b/providers/cloudant/pyproject.toml
index 97b6acf7336..4a603ad2e8b 100644
--- a/providers/cloudant/pyproject.toml
+++ b/providers/cloudant/pyproject.toml
@@ -49,7 +49,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9,!=3.9"
+requires-python = ">=3.9,!=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/cloudant/src/airflow/providers/cloudant/__init__.py b/providers/cloudant/src/airflow/providers/cloudant/__init__.py
index c05fc90bad5..e8ae12aeeb9 100644
--- a/providers/cloudant/src/airflow/providers/cloudant/__init__.py
+++ b/providers/cloudant/src/airflow/providers/cloudant/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.1.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-cloudant:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-cloudant:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/cncf/kubernetes/pyproject.toml b/providers/cncf/kubernetes/pyproject.toml
index 2ab2e8878ce..889c1e12880 100644
--- a/providers/cncf/kubernetes/pyproject.toml
+++ b/providers/cncf/kubernetes/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/__init__.py b/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/__init__.py
index 13f8738e40f..bd5e0923bb1 100644
--- a/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/__init__.py
+++ b/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "10.4.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-cncf-kubernetes:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-cncf-kubernetes:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/cohere/pyproject.toml b/providers/cohere/pyproject.toml
index 74806dba1ab..ce0769fbbab 100644
--- a/providers/cohere/pyproject.toml
+++ b/providers/cohere/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/cohere/src/airflow/providers/cohere/__init__.py b/providers/cohere/src/airflow/providers/cohere/__init__.py
index d991aeb48d8..7d6d53bb69c 100644
--- a/providers/cohere/src/airflow/providers/cohere/__init__.py
+++ b/providers/cohere/src/airflow/providers/cohere/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.4.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-cohere:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-cohere:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/common/compat/pyproject.toml b/providers/common/compat/pyproject.toml
index c21782e077e..77a21f6f157 100644
--- a/providers/common/compat/pyproject.toml
+++ b/providers/common/compat/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/common/compat/src/airflow/providers/common/compat/__init__.py b/providers/common/compat/src/airflow/providers/common/compat/__init__.py
index 1c777b668e1..c3a154282fe 100644
--- a/providers/common/compat/src/airflow/providers/common/compat/__init__.py
+++ b/providers/common/compat/src/airflow/providers/common/compat/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.6.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-common-compat:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-common-compat:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/common/io/pyproject.toml b/providers/common/io/pyproject.toml
index b13f18d345d..1ab9882fffb 100644
--- a/providers/common/io/pyproject.toml
+++ b/providers/common/io/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/common/io/src/airflow/providers/common/io/__init__.py b/providers/common/io/src/airflow/providers/common/io/__init__.py
index c1b9c8c755a..2bb3c398ff4 100644
--- a/providers/common/io/src/airflow/providers/common/io/__init__.py
+++ b/providers/common/io/src/airflow/providers/common/io/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.5.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-common-io:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-common-io:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/common/messaging/README.rst b/providers/common/messaging/README.rst
index ebdc3d6f487..7ec4a238cd2 100644
--- a/providers/common/messaging/README.rst
+++ b/providers/common/messaging/README.rst
@@ -23,7 +23,7 @@
Package ``apache-airflow-providers-common-messaging``
-Release: ``1.0.0``
+Release: ``1.0.1``
``Common Messaging Provider``
@@ -36,7 +36,7 @@ This is a provider package for ``common.messaging`` provider. All classes for th
are in ``airflow.providers.common.messaging`` python package.
You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-messaging/1.0.0/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-messaging/1.0.1/>`_.
Installation
------------
@@ -53,27 +53,8 @@ Requirements
================== ==================
PIP package Version required
================== ==================
-``apache-airflow`` ``>=3.0.0``
+``apache-airflow`` ``>=3.0.1``
================== ==================
-Cross provider package dependencies
------------------------------------
-
-Those are dependencies that might be needed in order to use all the features of the package.
-You need to install the specified providers in order to use them.
-
-You can install such cross-provider dependencies when installing from PyPI. For example:
-
-.. code-block:: bash
-
- pip install apache-airflow-providers-common-messaging[amazon]
-
-
-==================================================================================================== ==========
-Dependent package Extra
-==================================================================================================== ==========
-`apache-airflow-providers-amazon <https://airflow.apache.org/docs/apache-airflow-providers-amazon>`_ ``amazon``
-==================================================================================================== ==========
-
The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-messaging/1.0.0/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-messaging/1.0.1/changelog.html>`_.
diff --git a/providers/common/messaging/pyproject.toml b/providers/common/messaging/pyproject.toml
index cac7e940db5..109921842e2 100644
--- a/providers/common/messaging/pyproject.toml
+++ b/providers/common/messaging/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/common/sql/pyproject.toml b/providers/common/sql/pyproject.toml
index 3399ee0fa35..3c8ad4bc727 100644
--- a/providers/common/sql/pyproject.toml
+++ b/providers/common/sql/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/common/sql/src/airflow/providers/common/sql/__init__.py b/providers/common/sql/src/airflow/providers/common/sql/__init__.py
index a6c4eaca403..4ebaf00cca1 100644
--- a/providers/common/sql/src/airflow/providers/common/sql/__init__.py
+++ b/providers/common/sql/src/airflow/providers/common/sql/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.26.0"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-common-sql:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-common-sql:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/databricks/README.rst b/providers/databricks/README.rst
index f4bc9568d5b..8492896f33b 100644
--- a/providers/databricks/README.rst
+++ b/providers/databricks/README.rst
@@ -60,7 +60,7 @@ PIP package Version required
``aiohttp`` ``>=3.9.2,<4``
``mergedeep`` ``>=1.3.4``
``pandas`` ``>=2.1.2,<2.2``
-``pyarrow`` ``>=14.0.1``
+``pyarrow`` ``>=16.1.0``
======================================= ==================
Cross provider package dependencies
diff --git a/providers/databricks/pyproject.toml b/providers/databricks/pyproject.toml
index b39d9a13d4d..7fc30dc8c3a 100644
--- a/providers/databricks/pyproject.toml
+++ b/providers/databricks/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/datadog/pyproject.toml b/providers/datadog/pyproject.toml
index 554ab857150..f31466c781e 100644
--- a/providers/datadog/pyproject.toml
+++ b/providers/datadog/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/datadog/src/airflow/providers/datadog/__init__.py b/providers/datadog/src/airflow/providers/datadog/__init__.py
index 87dcb12eda2..9c39175b4d5 100644
--- a/providers/datadog/src/airflow/providers/datadog/__init__.py
+++ b/providers/datadog/src/airflow/providers/datadog/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.8.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-datadog:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-datadog:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/dbt/cloud/pyproject.toml b/providers/dbt/cloud/pyproject.toml
index aed93cc864e..2c086544513 100644
--- a/providers/dbt/cloud/pyproject.toml
+++ b/providers/dbt/cloud/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/dbt/cloud/src/airflow/providers/dbt/cloud/__init__.py b/providers/dbt/cloud/src/airflow/providers/dbt/cloud/__init__.py
index c0d869198af..dfe5dc2d7b1 100644
--- a/providers/dbt/cloud/src/airflow/providers/dbt/cloud/__init__.py
+++ b/providers/dbt/cloud/src/airflow/providers/dbt/cloud/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.3.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-dbt-cloud:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-dbt-cloud:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/dingding/pyproject.toml b/providers/dingding/pyproject.toml
index 7536b5a3d34..853db81f5f4 100644
--- a/providers/dingding/pyproject.toml
+++ b/providers/dingding/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/dingding/src/airflow/providers/dingding/__init__.py b/providers/dingding/src/airflow/providers/dingding/__init__.py
index 5e36a9da9f0..f09200ba734 100644
--- a/providers/dingding/src/airflow/providers/dingding/__init__.py
+++ b/providers/dingding/src/airflow/providers/dingding/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.7.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-dingding:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-dingding:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/discord/README.rst b/providers/discord/README.rst
index 9a21efa79fa..ed6bc5d24d8 100644
--- a/providers/discord/README.rst
+++ b/providers/discord/README.rst
@@ -54,7 +54,7 @@ Requirements
PIP package Version required
========================================== ==================
``apache-airflow`` ``>=2.9.0``
-``apache-airflow-providers-common-compat`` ``>=1.6.0``
+``apache-airflow-providers-common-compat`` ``>=1.6.1``
``apache-airflow-providers-http``
========================================== ==================
diff --git a/providers/discord/pyproject.toml b/providers/discord/pyproject.toml
index 0655faccbb2..21640848650 100644
--- a/providers/discord/pyproject.toml
+++ b/providers/discord/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/discord/src/airflow/providers/discord/__init__.py b/providers/discord/src/airflow/providers/discord/__init__.py
index 31643aec039..6fcaea26854 100644
--- a/providers/discord/src/airflow/providers/discord/__init__.py
+++ b/providers/discord/src/airflow/providers/discord/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.9.5"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-discord:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-discord:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/docker/pyproject.toml b/providers/docker/pyproject.toml
index 2c9cf5b9376..0a68efcb039 100644
--- a/providers/docker/pyproject.toml
+++ b/providers/docker/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/docker/src/airflow/providers/docker/__init__.py b/providers/docker/src/airflow/providers/docker/__init__.py
index ac9ec76f708..43954cf34b8 100644
--- a/providers/docker/src/airflow/providers/docker/__init__.py
+++ b/providers/docker/src/airflow/providers/docker/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.3.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-docker:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-docker:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/edge3/pyproject.toml b/providers/edge3/pyproject.toml
index d2ee5f6a9c9..df2772420e1 100644
--- a/providers/edge3/pyproject.toml
+++ b/providers/edge3/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/elasticsearch/pyproject.toml b/providers/elasticsearch/pyproject.toml
index 92f7ee7c773..50bc5a60b14 100644
--- a/providers/elasticsearch/pyproject.toml
+++ b/providers/elasticsearch/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/elasticsearch/src/airflow/providers/elasticsearch/__init__.py b/providers/elasticsearch/src/airflow/providers/elasticsearch/__init__.py
index 0cdcfb5090f..a8a8c1053c6 100644
--- a/providers/elasticsearch/src/airflow/providers/elasticsearch/__init__.py
+++ b/providers/elasticsearch/src/airflow/providers/elasticsearch/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "6.2.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-elasticsearch:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-elasticsearch:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/exasol/pyproject.toml b/providers/exasol/pyproject.toml
index f828fb639b5..5a1839cb4dd 100644
--- a/providers/exasol/pyproject.toml
+++ b/providers/exasol/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/exasol/src/airflow/providers/exasol/__init__.py b/providers/exasol/src/airflow/providers/exasol/__init__.py
index 85cce78cd81..6ea58feb4b2 100644
--- a/providers/exasol/src/airflow/providers/exasol/__init__.py
+++ b/providers/exasol/src/airflow/providers/exasol/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.7.5"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-exasol:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-exasol:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/fab/README.rst b/providers/fab/README.rst
index 3f5ded2e195..f008798a6c2 100644
--- a/providers/fab/README.rst
+++ b/providers/fab/README.rst
@@ -23,7 +23,7 @@
Package ``apache-airflow-providers-fab``
-Release: ``2.0.1``
+Release: ``2.2.0``
`Flask App Builder <https://flask-appbuilder.readthedocs.io/>`__
@@ -36,7 +36,7 @@ This is a provider package for ``fab`` provider. All classes for this provider p
are in ``airflow.providers.fab`` python package.
You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-fab/2.0.1/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-fab/2.2.0/>`_.
Installation
------------
@@ -53,11 +53,11 @@ Requirements
========================================== ==================
PIP package Version required
========================================== ==================
-``apache-airflow`` ``>=3.0.0``
+``apache-airflow`` ``>=3.0.2``
``apache-airflow-providers-common-compat`` ``>=1.2.1``
``blinker`` ``>=1.6.2``
``flask`` ``>=2.2.1,<2.3``
-``flask-appbuilder`` ``==4.5.3``
+``flask-appbuilder`` ``==4.6.3``
``flask-login`` ``>=0.6.2``
``flask-session`` ``>=0.4.0,<0.6``
``flask-wtf`` ``>=1.1.0``
@@ -87,4 +87,4 @@ Dependent package
================================================================================================================== =================
The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-fab/2.0.1/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-fab/2.2.0/changelog.html>`_.
diff --git a/providers/fab/pyproject.toml b/providers/fab/pyproject.toml
index 080aea46f17..a7167928eaa 100644
--- a/providers/fab/pyproject.toml
+++ b/providers/fab/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
license-files = ["NOTICE", "*/LICENSE*"]
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/facebook/pyproject.toml b/providers/facebook/pyproject.toml
index 7b7ca0caf5f..2367015a650 100644
--- a/providers/facebook/pyproject.toml
+++ b/providers/facebook/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/facebook/src/airflow/providers/facebook/__init__.py b/providers/facebook/src/airflow/providers/facebook/__init__.py
index 9d2c47bf0d1..6697a75e85b 100644
--- a/providers/facebook/src/airflow/providers/facebook/__init__.py
+++ b/providers/facebook/src/airflow/providers/facebook/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.7.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-facebook:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-facebook:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/ftp/pyproject.toml b/providers/ftp/pyproject.toml
index ba0a5e86613..9706da02cda 100644
--- a/providers/ftp/pyproject.toml
+++ b/providers/ftp/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/ftp/src/airflow/providers/ftp/__init__.py b/providers/ftp/src/airflow/providers/ftp/__init__.py
index 7c36e04d7f3..5679aced98e 100644
--- a/providers/ftp/src/airflow/providers/ftp/__init__.py
+++ b/providers/ftp/src/airflow/providers/ftp/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.12.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-ftp:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-ftp:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/git/pyproject.toml b/providers/git/pyproject.toml
index 047a4917f8c..b585f10fadf 100644
--- a/providers/git/pyproject.toml
+++ b/providers/git/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/github/pyproject.toml b/providers/github/pyproject.toml
index 026c74d63db..bff55f284bf 100644
--- a/providers/github/pyproject.toml
+++ b/providers/github/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/github/src/airflow/providers/github/__init__.py b/providers/github/src/airflow/providers/github/__init__.py
index 5d3024dc4f1..080f905e099 100644
--- a/providers/github/src/airflow/providers/github/__init__.py
+++ b/providers/github/src/airflow/providers/github/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "2.8.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-github:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-github:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/google/README.rst b/providers/google/README.rst
index b147db439dc..537c384e551 100644
--- a/providers/google/README.rst
+++ b/providers/google/README.rst
@@ -57,78 +57,81 @@ The package supports the following python versions: 3.9,3.10,3.11,3.12
Requirements
------------
-========================================== ======================================
-PIP package Version required
-========================================== ======================================
-``apache-airflow`` ``>=2.9.0``
-``apache-airflow-providers-common-compat`` ``>=1.4.0``
-``apache-airflow-providers-common-sql`` ``>=1.20.0``
-``asgiref`` ``>=3.5.2``
-``dill`` ``>=0.2.3``
-``gcloud-aio-auth`` ``>=5.2.0``
-``gcloud-aio-bigquery`` ``>=6.1.2``
-``gcloud-aio-storage`` ``>=9.0.0``
-``gcsfs`` ``>=2023.10.0``
-``google-ads`` ``>=26.0.0``
-``google-analytics-admin`` ``>=0.9.0``
-``google-api-core`` ``>=2.11.0,!=2.16.0,!=2.18.0``
-``google-api-python-client`` ``>=2.0.2``
-``google-auth`` ``>=2.29.0``
-``google-auth-httplib2`` ``>=0.0.1``
-``google-cloud-aiplatform[evaluation]`` ``>=1.73.0``
-``google-cloud-alloydb`` ``>=0.4.0``
-``google-cloud-automl`` ``>=2.12.0``
-``google-cloud-bigquery`` ``!=3.21.*,!=3.22.0,!=3.23.*,>=3.4.0``
-``google-cloud-bigquery-datatransfer`` ``>=3.13.0``
-``google-cloud-bigtable`` ``>=2.17.0``
-``google-cloud-build`` ``>=3.31.0``
-``google-cloud-compute`` ``>=1.10.0``
-``google-cloud-container`` ``>=2.52.0``
-``google-cloud-datacatalog`` ``>=3.23.0``
-``google-cloud-dataflow-client`` ``>=0.8.6``
-``google-cloud-dataform`` ``>=0.5.0``
-``google-cloud-dataplex`` ``>=2.6.0``
-``google-cloud-dataproc`` ``>=5.12.0``
-``google-cloud-dataproc-metastore`` ``>=1.12.0``
-``google-cloud-dlp`` ``>=3.12.0``
-``google-cloud-kms`` ``>=2.15.0``
-``google-cloud-language`` ``>=2.9.0``
-``google-cloud-logging`` ``>=3.5.0``
-``google-cloud-managedkafka`` ``>=0.1.6``
-``google-cloud-memcache`` ``>=1.7.0``
-``google-cloud-monitoring`` ``>=2.18.0``
-``google-cloud-orchestration-airflow`` ``>=1.10.0``
-``google-cloud-os-login`` ``>=2.9.1``
-``google-cloud-pubsub`` ``>=2.21.3``
-``google-cloud-redis`` ``>=2.12.0``
-``google-cloud-secret-manager`` ``>=2.16.0``
-``google-cloud-spanner`` ``>=3.11.1,!=3.49.0``
-``google-cloud-speech`` ``>=2.18.0``
-``google-cloud-storage`` ``>=2.7.0``
-``google-cloud-storage-transfer`` ``>=1.4.1``
-``google-cloud-tasks`` ``>=2.13.0``
-``google-cloud-texttospeech`` ``>=2.14.1``
-``google-cloud-translate`` ``>=3.16.0``
-``google-cloud-videointelligence`` ``>=2.11.0``
-``google-cloud-vision`` ``>=3.4.0``
-``google-cloud-workflows`` ``>=1.10.0``
-``google-cloud-run`` ``>=0.10.0``
-``google-cloud-batch`` ``>=0.13.0``
-``grpcio-gcp`` ``>=0.2.2``
-``httpx`` ``>=0.25.0``
-``looker-sdk`` ``>=22.4.0,!=24.18.0``
-``pandas-gbq`` ``>=0.7.0``
-``pandas`` ``>=2.1.2,<2.2``
-``proto-plus`` ``>=1.19.6``
-``pyarrow`` ``>=14.0.1``
-``python-slugify`` ``>=7.0.0``
-``PyOpenSSL`` ``>=23.0.0``
-``sqlalchemy-bigquery`` ``>=1.2.1``
-``sqlalchemy-spanner`` ``>=1.6.2,!=1.12.0``
-``tenacity`` ``>=8.1.0``
-``immutabledict`` ``>=4.2.0``
-``types-protobuf`` ``!=5.29.1.20250402``
-========================================== ======================================
+=========================================== ======================================
+PIP package Version required
+=========================================== ======================================
+``apache-airflow`` ``>=2.10.0``
+``apache-airflow-providers-common-compat`` ``>=1.4.0``
+``apache-airflow-providers-common-sql`` ``>=1.20.0``
+``asgiref`` ``>=3.5.2``
+``dill`` ``>=0.2.3``
+``gcloud-aio-auth`` ``>=5.2.0``
+``gcloud-aio-bigquery`` ``>=6.1.2``
+``gcloud-aio-storage`` ``>=9.0.0``
+``gcsfs`` ``>=2023.10.0``
+``google-ads`` ``>=26.0.0``
+``google-analytics-admin`` ``>=0.9.0``
+``google-api-core`` ``>=2.11.0,!=2.16.0,!=2.18.0``
+``google-api-python-client`` ``>=2.0.2``
+``google-auth`` ``>=2.29.0``
+``google-auth-httplib2`` ``>=0.0.1``
+``google-cloud-aiplatform[evaluation,ray]`` ``>=1.73.0; python_version < "3.12"``
+``google-cloud-aiplatform[evaluation]`` ``>=1.73.0; python_version >= "3.12"``
+``ray[default]`` ``>=2.42.0; python_version >= "3.12"``
+``google-cloud-bigquery-storage`` ``>=2.31.0; python_version >= "3.12"``
+``google-cloud-alloydb`` ``>=0.4.0``
+``google-cloud-automl`` ``>=2.12.0``
+``google-cloud-bigquery`` ``!=3.21.*,!=3.22.0,!=3.23.*,>=3.4.0``
+``google-cloud-bigquery-datatransfer`` ``>=3.13.0``
+``google-cloud-bigtable`` ``>=2.17.0``
+``google-cloud-build`` ``>=3.31.0``
+``google-cloud-compute`` ``>=1.10.0``
+``google-cloud-container`` ``>=2.52.0``
+``google-cloud-datacatalog`` ``>=3.23.0``
+``google-cloud-dataflow-client`` ``>=0.8.6``
+``google-cloud-dataform`` ``>=0.5.0``
+``google-cloud-dataplex`` ``>=2.6.0``
+``google-cloud-dataproc`` ``>=5.12.0``
+``google-cloud-dataproc-metastore`` ``>=1.12.0``
+``google-cloud-dlp`` ``>=3.12.0``
+``google-cloud-kms`` ``>=2.15.0``
+``google-cloud-language`` ``>=2.9.0``
+``google-cloud-logging`` ``>=3.5.0``
+``google-cloud-managedkafka`` ``>=0.1.6``
+``google-cloud-memcache`` ``>=1.7.0``
+``google-cloud-monitoring`` ``>=2.18.0``
+``google-cloud-orchestration-airflow`` ``>=1.10.0``
+``google-cloud-os-login`` ``>=2.9.1``
+``google-cloud-pubsub`` ``>=2.21.3``
+``google-cloud-redis`` ``>=2.12.0``
+``google-cloud-secret-manager`` ``>=2.16.0``
+``google-cloud-spanner`` ``>=3.11.1,!=3.49.0``
+``google-cloud-speech`` ``>=2.18.0``
+``google-cloud-storage`` ``>=2.7.0``
+``google-cloud-storage-transfer`` ``>=1.4.1``
+``google-cloud-tasks`` ``>=2.13.0``
+``google-cloud-texttospeech`` ``>=2.14.1``
+``google-cloud-translate`` ``>=3.16.0``
+``google-cloud-videointelligence`` ``>=2.11.0``
+``google-cloud-vision`` ``>=3.4.0``
+``google-cloud-workflows`` ``>=1.10.0``
+``google-cloud-run`` ``>=0.10.0``
+``google-cloud-batch`` ``>=0.13.0``
+``grpcio-gcp`` ``>=0.2.2``
+``httpx`` ``>=0.25.0``
+``looker-sdk`` ``>=22.4.0,!=24.18.0``
+``pandas-gbq`` ``>=0.7.0``
+``pandas`` ``>=2.1.2,<2.2``
+``proto-plus`` ``>=1.19.6``
+``pyarrow`` ``>=16.1.0``
+``python-slugify`` ``>=7.0.0``
+``PyOpenSSL`` ``>=23.0.0``
+``sqlalchemy-bigquery`` ``>=1.2.1``
+``sqlalchemy-spanner`` ``!=1.12.0,>=1.6.2``
+``tenacity`` ``>=8.1.0``
+``immutabledict`` ``>=4.2.0``
+``types-protobuf`` ``>=5.27.0,!=5.29.1.20250402``
+=========================================== ======================================
Cross provider package dependencies
-----------------------------------
diff --git a/providers/google/pyproject.toml b/providers/google/pyproject.toml
index 4599b875316..c9039e893a4 100644
--- a/providers/google/pyproject.toml
+++ b/providers/google/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/grpc/pyproject.toml b/providers/grpc/pyproject.toml
index c941b46e4cf..87e275fe3da 100644
--- a/providers/grpc/pyproject.toml
+++ b/providers/grpc/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/grpc/src/airflow/providers/grpc/__init__.py b/providers/grpc/src/airflow/providers/grpc/__init__.py
index 1d6d4595554..c4e95bf1915 100644
--- a/providers/grpc/src/airflow/providers/grpc/__init__.py
+++ b/providers/grpc/src/airflow/providers/grpc/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.7.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-grpc:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-grpc:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/hashicorp/pyproject.toml b/providers/hashicorp/pyproject.toml
index 596bfecf963..559ad1afbe7 100644
--- a/providers/hashicorp/pyproject.toml
+++ b/providers/hashicorp/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/hashicorp/src/airflow/providers/hashicorp/__init__.py b/providers/hashicorp/src/airflow/providers/hashicorp/__init__.py
index 5a30a0d89b9..4c0f219dbe3 100644
--- a/providers/hashicorp/src/airflow/providers/hashicorp/__init__.py
+++ b/providers/hashicorp/src/airflow/providers/hashicorp/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.1.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-hashicorp:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-hashicorp:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/http/pyproject.toml b/providers/http/pyproject.toml
index f35023f8122..79c1a053dc3 100644
--- a/providers/http/pyproject.toml
+++ b/providers/http/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/http/src/airflow/providers/http/__init__.py b/providers/http/src/airflow/providers/http/__init__.py
index 461c489d6df..17d8d32c02c 100644
--- a/providers/http/src/airflow/providers/http/__init__.py
+++ b/providers/http/src/airflow/providers/http/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.2.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-http:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-http:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/imap/pyproject.toml b/providers/imap/pyproject.toml
index 144fcc3d506..318b7c8d655 100644
--- a/providers/imap/pyproject.toml
+++ b/providers/imap/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/imap/src/airflow/providers/imap/__init__.py b/providers/imap/src/airflow/providers/imap/__init__.py
index e7efed09018..7c7b511bba5 100644
--- a/providers/imap/src/airflow/providers/imap/__init__.py
+++ b/providers/imap/src/airflow/providers/imap/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.8.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-imap:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-imap:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/influxdb/pyproject.toml b/providers/influxdb/pyproject.toml
index d1c555d27ac..1a12de54971 100644
--- a/providers/influxdb/pyproject.toml
+++ b/providers/influxdb/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/influxdb/src/airflow/providers/influxdb/__init__.py b/providers/influxdb/src/airflow/providers/influxdb/__init__.py
index 5774e51abd2..b9d18e334c7 100644
--- a/providers/influxdb/src/airflow/providers/influxdb/__init__.py
+++ b/providers/influxdb/src/airflow/providers/influxdb/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "2.8.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-influxdb:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-influxdb:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/jdbc/pyproject.toml b/providers/jdbc/pyproject.toml
index 0aeedb26e61..9ce02bda7bd 100644
--- a/providers/jdbc/pyproject.toml
+++ b/providers/jdbc/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/jdbc/src/airflow/providers/jdbc/__init__.py b/providers/jdbc/src/airflow/providers/jdbc/__init__.py
index 954078274a5..11952c2b0c6 100644
--- a/providers/jdbc/src/airflow/providers/jdbc/__init__.py
+++ b/providers/jdbc/src/airflow/providers/jdbc/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.1.0"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-jdbc:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-jdbc:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/jenkins/pyproject.toml b/providers/jenkins/pyproject.toml
index d75993f8fa5..1d7ea3ca281 100644
--- a/providers/jenkins/pyproject.toml
+++ b/providers/jenkins/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/jenkins/src/airflow/providers/jenkins/__init__.py b/providers/jenkins/src/airflow/providers/jenkins/__init__.py
index 1f502543d04..1c3271b0b33 100644
--- a/providers/jenkins/src/airflow/providers/jenkins/__init__.py
+++ b/providers/jenkins/src/airflow/providers/jenkins/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.0.4"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-jenkins:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-jenkins:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/microsoft/azure/README.rst b/providers/microsoft/azure/README.rst
index 57715cf5280..c83f0826106 100644
--- a/providers/microsoft/azure/README.rst
+++ b/providers/microsoft/azure/README.rst
@@ -50,39 +50,40 @@ The package supports the following python versions: 3.9,3.10,3.11,3.12
Requirements
------------
-====================================== ===================
-PIP package Version required
-====================================== ===================
-``apache-airflow`` ``>=2.9.0``
-``adlfs`` ``>=2023.10.0``
-``azure-batch`` ``>=8.0.0``
-``azure-cosmos`` ``>=4.6.0``
-``azure-mgmt-cosmosdb`` ``>=3.0.0``
-``azure-datalake-store`` ``>=0.0.45``
-``azure-identity`` ``>=1.3.1``
-``azure-keyvault-secrets`` ``>=4.1.0``
-``azure-mgmt-datalake-store`` ``>=0.5.0``
-``azure-mgmt-resource`` ``>=2.2.0``
-``azure-storage-blob`` ``>=12.14.0``
-``azure-mgmt-storage`` ``>=16.0.0``
-``azure-storage-file-share`` ``>=12.7.0``
-``azure-servicebus`` ``>=7.12.1``
-``azure-synapse-spark`` ``>=0.2.0``
-``azure-synapse-artifacts`` ``>=0.17.0``
-``adal`` ``>=1.2.7``
-``azure-storage-file-datalake`` ``>=12.9.1``
-``azure-kusto-data`` ``>=4.1.0,!=4.6.0``
-``azure-mgmt-datafactory`` ``>=2.0.0``
-``azure-mgmt-containerregistry`` ``>=8.0.0``
-``azure-mgmt-containerinstance`` ``>=10.1.0``
-``flask-appbuilder`` ``>=4.0.0``
-``msgraph-core`` ``>=1.3.3``
-``microsoft-kiota-http`` ``>=1.8.0,<2.0.0``
-``microsoft-kiota-serialization-json`` ``>=1.8.0``
-``microsoft-kiota-serialization-text`` ``>=1.8.0``
-``microsoft-kiota-abstractions`` ``>=1.8.0,<2.0.0``
-``msal-extensions`` ``>=1.1.0``
-====================================== ===================
+======================================== ===================
+PIP package Version required
+======================================== ===================
+``apache-airflow`` ``>=2.9.0``
+``adlfs`` ``>=2023.10.0``
+``azure-batch`` ``>=8.0.0``
+``azure-cosmos`` ``>=4.6.0``
+``azure-mgmt-cosmosdb`` ``>=3.0.0``
+``azure-datalake-store`` ``>=0.0.45``
+``azure-identity`` ``>=1.3.1``
+``azure-keyvault-secrets`` ``>=4.1.0``
+``azure-mgmt-datalake-store`` ``>=0.5.0``
+``azure-mgmt-resource`` ``>=2.2.0``
+``azure-storage-blob`` ``>=12.14.0``
+``azure-mgmt-storage`` ``>=16.0.0``
+``azure-storage-file-share`` ``>=12.7.0``
+``azure-servicebus`` ``>=7.12.1``
+``azure-synapse-spark`` ``>=0.2.0``
+``azure-synapse-artifacts`` ``>=0.17.0``
+``adal`` ``>=1.2.7``
+``azure-storage-file-datalake`` ``>=12.9.1``
+``azure-kusto-data`` ``>=4.1.0,!=4.6.0``
+``azure-mgmt-datafactory`` ``>=2.0.0``
+``azure-mgmt-containerregistry`` ``>=8.0.0``
+``azure-mgmt-containerinstance`` ``>=10.1.0``
+``flask-appbuilder`` ``>=4.0.0``
+``msgraph-core`` ``>=1.3.3``
+``microsoft-kiota-http`` ``>=1.9.0,<2.0.0``
+``microsoft-kiota-serialization-json`` ``>=1.9.0``
+``microsoft-kiota-serialization-text`` ``>=1.9.0``
+``microsoft-kiota-abstractions`` ``>=1.9.0,<2.0.0``
+``microsoft-kiota-authentication-azure`` ``>=1.9.0,<2.0.0``
+``msal-extensions`` ``>=1.1.0``
+======================================== ===================
Cross provider package dependencies
-----------------------------------
diff --git a/providers/microsoft/azure/pyproject.toml b/providers/microsoft/azure/pyproject.toml
index 4e492a172ae..1c7dbb26945 100644
--- a/providers/microsoft/azure/pyproject.toml
+++ b/providers/microsoft/azure/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/microsoft/azure/src/airflow/providers/microsoft/azure/__init__.py b/providers/microsoft/azure/src/airflow/providers/microsoft/azure/__init__.py
index 544964e6c32..4bf1850f73e 100644
--- a/providers/microsoft/azure/src/airflow/providers/microsoft/azure/__init__.py
+++ b/providers/microsoft/azure/src/airflow/providers/microsoft/azure/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "12.3.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-microsoft-azure:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-microsoft-azure:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/microsoft/mssql/pyproject.toml b/providers/microsoft/mssql/pyproject.toml
index 4c87f27650f..08c0ad43117 100644
--- a/providers/microsoft/mssql/pyproject.toml
+++ b/providers/microsoft/mssql/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/microsoft/mssql/src/airflow/providers/microsoft/mssql/__init__.py b/providers/microsoft/mssql/src/airflow/providers/microsoft/mssql/__init__.py
index 09e406a73db..c80b084d713 100644
--- a/providers/microsoft/mssql/src/airflow/providers/microsoft/mssql/__init__.py
+++ b/providers/microsoft/mssql/src/airflow/providers/microsoft/mssql/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.2.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-microsoft-mssql:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-microsoft-mssql:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/microsoft/psrp/pyproject.toml b/providers/microsoft/psrp/pyproject.toml
index b3d20ba7aca..adfea050098 100644
--- a/providers/microsoft/psrp/pyproject.toml
+++ b/providers/microsoft/psrp/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/microsoft/psrp/src/airflow/providers/microsoft/psrp/__init__.py b/providers/microsoft/psrp/src/airflow/providers/microsoft/psrp/__init__.py
index b5453850fce..dcef28fbc11 100644
--- a/providers/microsoft/psrp/src/airflow/providers/microsoft/psrp/__init__.py
+++ b/providers/microsoft/psrp/src/airflow/providers/microsoft/psrp/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.0.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-microsoft-psrp:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-microsoft-psrp:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/microsoft/winrm/pyproject.toml b/providers/microsoft/winrm/pyproject.toml
index 87b1afde68b..77517861a39 100644
--- a/providers/microsoft/winrm/pyproject.toml
+++ b/providers/microsoft/winrm/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/microsoft/winrm/src/airflow/providers/microsoft/winrm/__init__.py b/providers/microsoft/winrm/src/airflow/providers/microsoft/winrm/__init__.py
index 736d1e35998..0d44282cf24 100644
--- a/providers/microsoft/winrm/src/airflow/providers/microsoft/winrm/__init__.py
+++ b/providers/microsoft/winrm/src/airflow/providers/microsoft/winrm/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.9.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-microsoft-winrm:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-microsoft-winrm:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/mongo/pyproject.toml b/providers/mongo/pyproject.toml
index e57106c016c..deeb6ccae29 100644
--- a/providers/mongo/pyproject.toml
+++ b/providers/mongo/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/mongo/src/airflow/providers/mongo/__init__.py b/providers/mongo/src/airflow/providers/mongo/__init__.py
index 3e86c1ef64d..cde39ca32ee 100644
--- a/providers/mongo/src/airflow/providers/mongo/__init__.py
+++ b/providers/mongo/src/airflow/providers/mongo/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.0.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-mongo:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-mongo:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/mysql/README.rst b/providers/mysql/README.rst
index aef72c846d4..b7219a4b780 100644
--- a/providers/mysql/README.rst
+++ b/providers/mysql/README.rst
@@ -50,15 +50,17 @@ The package supports the following python versions: 3.9,3.10,3.11,3.12
Requirements
------------
-======================================= =====================================
+======================================= ==================================================================
PIP package Version required
-======================================= =====================================
+======================================= ==================================================================
``apache-airflow`` ``>=2.9.0``
``apache-airflow-providers-common-sql`` ``>=1.20.0``
-``mysqlclient`` ``>=1.4.0; sys_platform != "darwin"``
-``mysql-connector-python`` ``>=8.0.29``
+``mysqlclient`` ``>=2.1.0; sys_platform != "darwin" and python_version < "3.12"``
+``mysqlclient`` ``>=2.2.1; sys_platform != "darwin" and python_version >= "3.12"``
+``mysql-connector-python`` ``>=8.0.29; python_version < "3.12"``
+``mysql-connector-python`` ``>=8.2.0; python_version >= "3.12"``
``aiomysql`` ``>=0.2.0``
-======================================= =====================================
+======================================= ==================================================================
Cross provider package dependencies
-----------------------------------
diff --git a/providers/mysql/pyproject.toml b/providers/mysql/pyproject.toml
index 2a49fdd42c1..b48e6a268fe 100644
--- a/providers/mysql/pyproject.toml
+++ b/providers/mysql/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/mysql/src/airflow/providers/mysql/__init__.py b/providers/mysql/src/airflow/providers/mysql/__init__.py
index bff07c5c9a4..0c7f4511816 100644
--- a/providers/mysql/src/airflow/providers/mysql/__init__.py
+++ b/providers/mysql/src/airflow/providers/mysql/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "6.2.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-mysql:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-mysql:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/neo4j/pyproject.toml b/providers/neo4j/pyproject.toml
index a0b7199bcee..a0444f4bc24 100644
--- a/providers/neo4j/pyproject.toml
+++ b/providers/neo4j/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/neo4j/src/airflow/providers/neo4j/__init__.py b/providers/neo4j/src/airflow/providers/neo4j/__init__.py
index a872e681dc1..cc64c6c9071 100644
--- a/providers/neo4j/src/airflow/providers/neo4j/__init__.py
+++ b/providers/neo4j/src/airflow/providers/neo4j/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.8.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-neo4j:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-neo4j:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/odbc/pyproject.toml b/providers/odbc/pyproject.toml
index 9a1b447a38a..32aac5e4499 100644
--- a/providers/odbc/pyproject.toml
+++ b/providers/odbc/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/odbc/src/airflow/providers/odbc/__init__.py b/providers/odbc/src/airflow/providers/odbc/__init__.py
index b1fd4aa617a..9e6941f03fd 100644
--- a/providers/odbc/src/airflow/providers/odbc/__init__.py
+++ b/providers/odbc/src/airflow/providers/odbc/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.9.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-odbc:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-odbc:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/openai/pyproject.toml b/providers/openai/pyproject.toml
index 674e77e3a06..1925b6d8259 100644
--- a/providers/openai/pyproject.toml
+++ b/providers/openai/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/openai/src/airflow/providers/openai/__init__.py b/providers/openai/src/airflow/providers/openai/__init__.py
index 08270040ce5..e525a647496 100644
--- a/providers/openai/src/airflow/providers/openai/__init__.py
+++ b/providers/openai/src/airflow/providers/openai/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.5.4"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-openai:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-openai:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/openfaas/pyproject.toml b/providers/openfaas/pyproject.toml
index fde81418442..87e1b68b3cf 100644
--- a/providers/openfaas/pyproject.toml
+++ b/providers/openfaas/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/openfaas/src/airflow/providers/openfaas/__init__.py b/providers/openfaas/src/airflow/providers/openfaas/__init__.py
index eb07096060a..f18f3bd0b61 100644
--- a/providers/openfaas/src/airflow/providers/openfaas/__init__.py
+++ b/providers/openfaas/src/airflow/providers/openfaas/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.7.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-openfaas:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-openfaas:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/openlineage/pyproject.toml b/providers/openlineage/pyproject.toml
index b1f1f456b9f..6bd4dc8ceda 100644
--- a/providers/openlineage/pyproject.toml
+++ b/providers/openlineage/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/openlineage/src/airflow/providers/openlineage/__init__.py b/providers/openlineage/src/airflow/providers/openlineage/__init__.py
index 71bb3022d47..b53ea545bb0 100644
--- a/providers/openlineage/src/airflow/providers/openlineage/__init__.py
+++ b/providers/openlineage/src/airflow/providers/openlineage/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "2.1.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-openlineage:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-openlineage:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/opensearch/pyproject.toml b/providers/opensearch/pyproject.toml
index 0654e302478..703bdd60958 100644
--- a/providers/opensearch/pyproject.toml
+++ b/providers/opensearch/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/opensearch/src/airflow/providers/opensearch/__init__.py b/providers/opensearch/src/airflow/providers/opensearch/__init__.py
index 743dd733738..59e1ecd64b5 100644
--- a/providers/opensearch/src/airflow/providers/opensearch/__init__.py
+++ b/providers/opensearch/src/airflow/providers/opensearch/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.6.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-opensearch:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-opensearch:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/opsgenie/README.rst b/providers/opsgenie/README.rst
index 691210cd5ae..c0f552d2f1b 100644
--- a/providers/opsgenie/README.rst
+++ b/providers/opsgenie/README.rst
@@ -54,7 +54,7 @@ Requirements
PIP package Version required
========================================== ==================
``apache-airflow`` ``>=2.9.0``
-``apache-airflow-providers-common-compat`` ``>=1.6.0``
+``apache-airflow-providers-common-compat`` ``>=1.6.1``
``opsgenie-sdk`` ``>=2.1.5``
========================================== ==================
diff --git a/providers/opsgenie/pyproject.toml b/providers/opsgenie/pyproject.toml
index 3793a273257..5dfcbcec605 100644
--- a/providers/opsgenie/pyproject.toml
+++ b/providers/opsgenie/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/opsgenie/src/airflow/providers/opsgenie/__init__.py b/providers/opsgenie/src/airflow/providers/opsgenie/__init__.py
index 67996f06911..cb1702a7775 100644
--- a/providers/opsgenie/src/airflow/providers/opsgenie/__init__.py
+++ b/providers/opsgenie/src/airflow/providers/opsgenie/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.8.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-opsgenie:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-opsgenie:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/oracle/pyproject.toml b/providers/oracle/pyproject.toml
index 69f39f574f4..ceaded6d74b 100644
--- a/providers/oracle/pyproject.toml
+++ b/providers/oracle/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/oracle/src/airflow/providers/oracle/__init__.py b/providers/oracle/src/airflow/providers/oracle/__init__.py
index cb0f26c6603..c74583dcb96 100644
--- a/providers/oracle/src/airflow/providers/oracle/__init__.py
+++ b/providers/oracle/src/airflow/providers/oracle/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.0.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-oracle:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-oracle:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/pagerduty/README.rst b/providers/pagerduty/README.rst
index 1b4f8128af5..d1696e1c36c 100644
--- a/providers/pagerduty/README.rst
+++ b/providers/pagerduty/README.rst
@@ -54,7 +54,7 @@ Requirements
PIP package Version required
========================================== ==================
``apache-airflow`` ``>=2.9.0``
-``apache-airflow-providers-common-compat`` ``>=1.6.0``
+``apache-airflow-providers-common-compat`` ``>=1.6.1``
``pdpyras`` ``>=4.2.0``
========================================== ==================
diff --git a/providers/pagerduty/pyproject.toml b/providers/pagerduty/pyproject.toml
index 3549b7c4cbd..d39466da545 100644
--- a/providers/pagerduty/pyproject.toml
+++ b/providers/pagerduty/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/pagerduty/src/airflow/providers/pagerduty/__init__.py b/providers/pagerduty/src/airflow/providers/pagerduty/__init__.py
index d22bf7ea0e9..4156ab3df9d 100644
--- a/providers/pagerduty/src/airflow/providers/pagerduty/__init__.py
+++ b/providers/pagerduty/src/airflow/providers/pagerduty/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.0.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-pagerduty:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-pagerduty:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/papermill/pyproject.toml b/providers/papermill/pyproject.toml
index 4951fde6b2e..c2d175c67cc 100644
--- a/providers/papermill/pyproject.toml
+++ b/providers/papermill/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/papermill/src/airflow/providers/papermill/__init__.py b/providers/papermill/src/airflow/providers/papermill/__init__.py
index bd2a840468d..3033d33126f 100644
--- a/providers/papermill/src/airflow/providers/papermill/__init__.py
+++ b/providers/papermill/src/airflow/providers/papermill/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.10.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-papermill:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-papermill:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/pgvector/pyproject.toml b/providers/pgvector/pyproject.toml
index f4cbd6c978c..349e735a568 100644
--- a/providers/pgvector/pyproject.toml
+++ b/providers/pgvector/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/pgvector/src/airflow/providers/pgvector/__init__.py b/providers/pgvector/src/airflow/providers/pgvector/__init__.py
index ca25404f116..a74b89cc713 100644
--- a/providers/pgvector/src/airflow/providers/pgvector/__init__.py
+++ b/providers/pgvector/src/airflow/providers/pgvector/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.4.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-pgvector:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-pgvector:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/pinecone/pyproject.toml b/providers/pinecone/pyproject.toml
index 39326b68405..7096e965c3b 100644
--- a/providers/pinecone/pyproject.toml
+++ b/providers/pinecone/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/pinecone/src/airflow/providers/pinecone/__init__.py b/providers/pinecone/src/airflow/providers/pinecone/__init__.py
index 98e276242ca..d6cd6944133 100644
--- a/providers/pinecone/src/airflow/providers/pinecone/__init__.py
+++ b/providers/pinecone/src/airflow/providers/pinecone/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "2.2.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-pinecone:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-pinecone:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/postgres/pyproject.toml b/providers/postgres/pyproject.toml
index cc9620f563b..42c5ec1b3c7 100644
--- a/providers/postgres/pyproject.toml
+++ b/providers/postgres/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/postgres/src/airflow/providers/postgres/__init__.py b/providers/postgres/src/airflow/providers/postgres/__init__.py
index 5b97ca0299b..16f22afd9ec 100644
--- a/providers/postgres/src/airflow/providers/postgres/__init__.py
+++ b/providers/postgres/src/airflow/providers/postgres/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "6.1.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-postgres:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-postgres:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/presto/pyproject.toml b/providers/presto/pyproject.toml
index 69999b48afd..1325df2cb83 100644
--- a/providers/presto/pyproject.toml
+++ b/providers/presto/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/presto/src/airflow/providers/presto/__init__.py b/providers/presto/src/airflow/providers/presto/__init__.py
index 4ae6ff330a4..198c93935e3 100644
--- a/providers/presto/src/airflow/providers/presto/__init__.py
+++ b/providers/presto/src/airflow/providers/presto/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.8.4"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-presto:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-presto:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/qdrant/pyproject.toml b/providers/qdrant/pyproject.toml
index 7494d42ee7b..9756fbdc20c 100644
--- a/providers/qdrant/pyproject.toml
+++ b/providers/qdrant/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/qdrant/src/airflow/providers/qdrant/__init__.py b/providers/qdrant/src/airflow/providers/qdrant/__init__.py
index be4d019b382..7f760d710c1 100644
--- a/providers/qdrant/src/airflow/providers/qdrant/__init__.py
+++ b/providers/qdrant/src/airflow/providers/qdrant/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "1.3.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-qdrant:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-qdrant:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/redis/pyproject.toml b/providers/redis/pyproject.toml
index 88940af8ff0..336a299449a 100644
--- a/providers/redis/pyproject.toml
+++ b/providers/redis/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/redis/src/airflow/providers/redis/__init__.py b/providers/redis/src/airflow/providers/redis/__init__.py
index a04f212fb42..a0b7729bcf1 100644
--- a/providers/redis/src/airflow/providers/redis/__init__.py
+++ b/providers/redis/src/airflow/providers/redis/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.0.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-redis:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-redis:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/salesforce/pyproject.toml b/providers/salesforce/pyproject.toml
index bc716276586..0d998dd0a42 100644
--- a/providers/salesforce/pyproject.toml
+++ b/providers/salesforce/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/salesforce/src/airflow/providers/salesforce/__init__.py b/providers/salesforce/src/airflow/providers/salesforce/__init__.py
index ddd54a881c7..8cea4d2b67d 100644
--- a/providers/salesforce/src/airflow/providers/salesforce/__init__.py
+++ b/providers/salesforce/src/airflow/providers/salesforce/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.10.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-salesforce:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-salesforce:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/samba/pyproject.toml b/providers/samba/pyproject.toml
index 9932203c7c2..17f92962706 100644
--- a/providers/samba/pyproject.toml
+++ b/providers/samba/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/samba/src/airflow/providers/samba/__init__.py b/providers/samba/src/airflow/providers/samba/__init__.py
index 74fd4a0be2a..27dec16b767 100644
--- a/providers/samba/src/airflow/providers/samba/__init__.py
+++ b/providers/samba/src/airflow/providers/samba/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.9.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-samba:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-samba:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/segment/pyproject.toml b/providers/segment/pyproject.toml
index 34ef591c4e0..7fcde976dfd 100644
--- a/providers/segment/pyproject.toml
+++ b/providers/segment/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/segment/src/airflow/providers/segment/__init__.py b/providers/segment/src/airflow/providers/segment/__init__.py
index f35ed9c07f9..caa62100eac 100644
--- a/providers/segment/src/airflow/providers/segment/__init__.py
+++ b/providers/segment/src/airflow/providers/segment/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.7.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-segment:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-segment:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/sendgrid/pyproject.toml b/providers/sendgrid/pyproject.toml
index b36848fcf50..830b8ed3d3b 100644
--- a/providers/sendgrid/pyproject.toml
+++ b/providers/sendgrid/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/sftp/pyproject.toml b/providers/sftp/pyproject.toml
index 3b2267a0ed7..ea2559c6819 100644
--- a/providers/sftp/pyproject.toml
+++ b/providers/sftp/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/sftp/src/airflow/providers/sftp/__init__.py b/providers/sftp/src/airflow/providers/sftp/__init__.py
index 39040c38df9..bdc77dd2dae 100644
--- a/providers/sftp/src/airflow/providers/sftp/__init__.py
+++ b/providers/sftp/src/airflow/providers/sftp/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.2.0"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-sftp:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-sftp:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/singularity/pyproject.toml b/providers/singularity/pyproject.toml
index 1daa9a212c5..90778a750c4 100644
--- a/providers/singularity/pyproject.toml
+++ b/providers/singularity/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/singularity/src/airflow/providers/singularity/__init__.py b/providers/singularity/src/airflow/providers/singularity/__init__.py
index 69ac24d064b..ac1923505cd 100644
--- a/providers/singularity/src/airflow/providers/singularity/__init__.py
+++ b/providers/singularity/src/airflow/providers/singularity/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.7.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-singularity:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-singularity:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/slack/README.rst b/providers/slack/README.rst
index 2f4f1b5ef4c..6bda03211b9 100644
--- a/providers/slack/README.rst
+++ b/providers/slack/README.rst
@@ -57,7 +57,7 @@ Requirements
PIP package Version required
========================================== ==================
``apache-airflow`` ``>=2.9.0``
-``apache-airflow-providers-common-compat`` ``>=1.6.0``
+``apache-airflow-providers-common-compat`` ``>=1.6.1``
``apache-airflow-providers-common-sql`` ``>=1.20.0``
``slack_sdk`` ``>=3.19.0``
========================================== ==================
diff --git a/providers/slack/pyproject.toml b/providers/slack/pyproject.toml
index df5b38423f4..2e8680788b4 100644
--- a/providers/slack/pyproject.toml
+++ b/providers/slack/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/slack/src/airflow/providers/slack/__init__.py b/providers/slack/src/airflow/providers/slack/__init__.py
index 3f1bf327318..7815d90ca13 100644
--- a/providers/slack/src/airflow/providers/slack/__init__.py
+++ b/providers/slack/src/airflow/providers/slack/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "9.0.4"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-slack:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-slack:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/smtp/README.rst b/providers/smtp/README.rst
index 4e737aec327..22306cb1e0c 100644
--- a/providers/smtp/README.rst
+++ b/providers/smtp/README.rst
@@ -54,7 +54,7 @@ Requirements
PIP package Version required
========================================== ==================
``apache-airflow`` ``>=2.9.0``
-``apache-airflow-providers-common-compat`` ``>=1.6.0``
+``apache-airflow-providers-common-compat`` ``>=1.6.1``
========================================== ==================
Cross provider package dependencies
diff --git a/providers/smtp/pyproject.toml b/providers/smtp/pyproject.toml
index 947d4bf9e0a..3916e3025bb 100644
--- a/providers/smtp/pyproject.toml
+++ b/providers/smtp/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/smtp/src/airflow/providers/smtp/__init__.py b/providers/smtp/src/airflow/providers/smtp/__init__.py
index 5e1a5b58c43..cb9ba6d1f54 100644
--- a/providers/smtp/src/airflow/providers/smtp/__init__.py
+++ b/providers/smtp/src/airflow/providers/smtp/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "2.0.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-smtp:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-smtp:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/snowflake/README.rst b/providers/snowflake/README.rst
index 7e11644570d..6541aa95b8c 100644
--- a/providers/snowflake/README.rst
+++ b/providers/snowflake/README.rst
@@ -57,7 +57,7 @@ PIP package Version required
``apache-airflow-providers-common-compat`` ``>=1.6.0``
``apache-airflow-providers-common-sql`` ``>=1.20.0``
``pandas`` ``>=2.1.2,<2.2``
-``pyarrow`` ``>=14.0.1``
+``pyarrow`` ``>=16.1.0``
``snowflake-connector-python`` ``>=3.7.1``
``snowflake-sqlalchemy`` ``>=1.4.0``
``snowflake-snowpark-python`` ``>=1.17.0; python_version < "3.12"``
diff --git a/providers/snowflake/pyproject.toml b/providers/snowflake/pyproject.toml
index 51adcb46d2d..15a8f8a34ac 100644
--- a/providers/snowflake/pyproject.toml
+++ b/providers/snowflake/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/sqlite/pyproject.toml b/providers/sqlite/pyproject.toml
index 35b3856ca4e..66ca45067e4 100644
--- a/providers/sqlite/pyproject.toml
+++ b/providers/sqlite/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/sqlite/src/airflow/providers/sqlite/__init__.py b/providers/sqlite/src/airflow/providers/sqlite/__init__.py
index 1d170278e15..4abc7ed5d53 100644
--- a/providers/sqlite/src/airflow/providers/sqlite/__init__.py
+++ b/providers/sqlite/src/airflow/providers/sqlite/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.0.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-sqlite:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-sqlite:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/ssh/pyproject.toml b/providers/ssh/pyproject.toml
index ee063f1d48c..e162709ab9a 100644
--- a/providers/ssh/pyproject.toml
+++ b/providers/ssh/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/ssh/src/airflow/providers/ssh/__init__.py b/providers/ssh/src/airflow/providers/ssh/__init__.py
index a9076e62537..28611dfdfd7 100644
--- a/providers/ssh/src/airflow/providers/ssh/__init__.py
+++ b/providers/ssh/src/airflow/providers/ssh/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.0.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-ssh:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-ssh:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/standard/pyproject.toml b/providers/standard/pyproject.toml
index 68e3623d74d..24051f39c11 100644
--- a/providers/standard/pyproject.toml
+++ b/providers/standard/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/standard/src/airflow/providers/standard/__init__.py b/providers/standard/src/airflow/providers/standard/__init__.py
index c717ad03e9c..e8c8403ccdf 100644
--- a/providers/standard/src/airflow/providers/standard/__init__.py
+++ b/providers/standard/src/airflow/providers/standard/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "0.4.0"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-standard:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-standard:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/tableau/pyproject.toml b/providers/tableau/pyproject.toml
index 4dde8553c59..c2ce8c9ceb1 100644
--- a/providers/tableau/pyproject.toml
+++ b/providers/tableau/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/tableau/src/airflow/providers/tableau/__init__.py b/providers/tableau/src/airflow/providers/tableau/__init__.py
index df7780db7ed..56cde220aac 100644
--- a/providers/tableau/src/airflow/providers/tableau/__init__.py
+++ b/providers/tableau/src/airflow/providers/tableau/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "5.0.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-tableau:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-tableau:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/telegram/pyproject.toml b/providers/telegram/pyproject.toml
index fc159bebdd4..4bf981c6e86 100644
--- a/providers/telegram/pyproject.toml
+++ b/providers/telegram/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/telegram/src/airflow/providers/telegram/__init__.py b/providers/telegram/src/airflow/providers/telegram/__init__.py
index 0be239e7a1d..ce6a0cace03 100644
--- a/providers/telegram/src/airflow/providers/telegram/__init__.py
+++ b/providers/telegram/src/airflow/providers/telegram/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.7.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-telegram:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-telegram:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/teradata/pyproject.toml b/providers/teradata/pyproject.toml
index 5806c30df4b..3925ab0d49c 100644
--- a/providers/teradata/pyproject.toml
+++ b/providers/teradata/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/teradata/src/airflow/providers/teradata/__init__.py b/providers/teradata/src/airflow/providers/teradata/__init__.py
index 0ebaa7c3ae6..693218c6598 100644
--- a/providers/teradata/src/airflow/providers/teradata/__init__.py
+++ b/providers/teradata/src/airflow/providers/teradata/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.0.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-teradata:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-teradata:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/trino/pyproject.toml b/providers/trino/pyproject.toml
index 8d763b54966..8a07cdcabd5 100644
--- a/providers/trino/pyproject.toml
+++ b/providers/trino/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/trino/src/airflow/providers/trino/__init__.py b/providers/trino/src/airflow/providers/trino/__init__.py
index 04e5d3d25ba..c730b3139f6 100644
--- a/providers/trino/src/airflow/providers/trino/__init__.py
+++ b/providers/trino/src/airflow/providers/trino/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "6.1.2"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-trino:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-trino:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/vertica/pyproject.toml b/providers/vertica/pyproject.toml
index ab4b1eb9075..03aa8da5aa5 100644
--- a/providers/vertica/pyproject.toml
+++ b/providers/vertica/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/vertica/src/airflow/providers/vertica/__init__.py b/providers/vertica/src/airflow/providers/vertica/__init__.py
index bd2f6eee67f..7ad8282a80a 100644
--- a/providers/vertica/src/airflow/providers/vertica/__init__.py
+++ b/providers/vertica/src/airflow/providers/vertica/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.0.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-vertica:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-vertica:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/weaviate/pyproject.toml b/providers/weaviate/pyproject.toml
index 77244e5ba48..3724e4348a0 100644
--- a/providers/weaviate/pyproject.toml
+++ b/providers/weaviate/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/weaviate/src/airflow/providers/weaviate/__init__.py b/providers/weaviate/src/airflow/providers/weaviate/__init__.py
index 7b74ecd7989..b4e443e1560 100644
--- a/providers/weaviate/src/airflow/providers/weaviate/__init__.py
+++ b/providers/weaviate/src/airflow/providers/weaviate/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "3.0.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-weaviate:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-weaviate:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/yandex/pyproject.toml b/providers/yandex/pyproject.toml
index 47d52ae9c53..0f0df6b094d 100644
--- a/providers/yandex/pyproject.toml
+++ b/providers/yandex/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/yandex/src/airflow/providers/yandex/__init__.py b/providers/yandex/src/airflow/providers/yandex/__init__.py
index b2be1df7c2e..8f072363d39 100644
--- a/providers/yandex/src/airflow/providers/yandex/__init__.py
+++ b/providers/yandex/src/airflow/providers/yandex/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.0.3"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-yandex:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-yandex:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/ydb/pyproject.toml b/providers/ydb/pyproject.toml
index 86b6c26ed9e..edbbc2db84b 100644
--- a/providers/ydb/pyproject.toml
+++ b/providers/ydb/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/ydb/src/airflow/providers/ydb/__init__.py b/providers/ydb/src/airflow/providers/ydb/__init__.py
index eb665fa03b7..1ce5982fb7a 100644
--- a/providers/ydb/src/airflow/providers/ydb/__init__.py
+++ b/providers/ydb/src/airflow/providers/ydb/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "2.1.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-ydb:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-ydb:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/providers/zendesk/pyproject.toml b/providers/zendesk/pyproject.toml
index a8b04d8e0c2..38ddd3059f1 100644
--- a/providers/zendesk/pyproject.toml
+++ b/providers/zendesk/pyproject.toml
@@ -50,7 +50,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Topic :: System :: Monitoring",
]
-requires-python = "~=3.9"
+requires-python = ">=3.9"
# The dependencies should be modified in place in the generated file.
# Any change in the dependencies is preserved when the file is regenerated
diff --git a/providers/zendesk/src/airflow/providers/zendesk/__init__.py b/providers/zendesk/src/airflow/providers/zendesk/__init__.py
index a854631a835..4f8e90effc8 100644
--- a/providers/zendesk/src/airflow/providers/zendesk/__init__.py
+++ b/providers/zendesk/src/airflow/providers/zendesk/__init__.py
@@ -32,8 +32,8 @@ __all__ = ["__version__"]
__version__ = "4.9.1"
if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
- "2.9.0"
+ "2.10.0"
):
raise RuntimeError(
- f"The package `apache-airflow-providers-zendesk:{__version__}` needs Apache Airflow 2.9.0+"
+ f"The package `apache-airflow-providers-zendesk:{__version__}` needs Apache Airflow 2.10.0+"
)
diff --git a/pyproject.toml b/pyproject.toml
index 16fb8709196..46ec16781c3 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -34,7 +34,7 @@ name = "apache-airflow"
description = "Programmatically author, schedule and monitor data pipelines"
readme = { file = "generated/PYPI_README.md", content-type = "text/markdown" }
license-files.globs = ["LICENSE"]
-requires-python = "~=3.9,<3.13"
+requires-python = ">=3.9,!=3.13"
authors = [
{ name = "Apache Software Foundation", email = "dev@airflow.apache.org" },
]
diff --git a/task-sdk/pyproject.toml b/task-sdk/pyproject.toml
index 2f7ecedd56b..11b3859fda0 100644
--- a/task-sdk/pyproject.toml
+++ b/task-sdk/pyproject.toml
@@ -21,7 +21,7 @@ dynamic = ["version"]
description = "Python Task SDK for Apache Airflow DAG Authors"
readme = { file = "README.md", content-type = "text/markdown" }
license-files.globs = ["LICENSE"]
-requires-python = ">=3.9, <3.13"
+requires-python = ">=3.9, !=3.13"
authors = [
{name="Apache Software Foundation", email="dev@airflow.apache.org"},
(airflow) 04/12: [v3-0-test] Advanced auto classification for provider documentation (#52902) (#52925)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit aea7296ae2fd42593ff9736d39ba3833f9e5fe22
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Sun Jul 6 01:05:07 2025 +0200
[v3-0-test] Advanced auto classification for provider documentation (#52902) (#52925)
(cherry picked from commit 19961713421885c8306d3a51a4c3eaf5f27f6cb7)
Co-authored-by: Amogh Desai <am...@gmail.com>
---
.../prepare_providers/provider_documentation.py | 11 +++---
dev/breeze/tests/test_provider_documentation.py | 39 ++++++++++++++++++----
2 files changed, 39 insertions(+), 11 deletions(-)
diff --git a/dev/breeze/src/airflow_breeze/prepare_providers/provider_documentation.py b/dev/breeze/src/airflow_breeze/prepare_providers/provider_documentation.py
index c330047b7cd..b5f6a339cbd 100644
--- a/dev/breeze/src/airflow_breeze/prepare_providers/provider_documentation.py
+++ b/dev/breeze/src/airflow_breeze/prepare_providers/provider_documentation.py
@@ -195,7 +195,10 @@ TYPE_OF_CHANGE_DESCRIPTION = {
}
-def classification_result(changed_files):
+def classification_result(provider_id, changed_files):
+ provider_path = f"providers/{provider_id}/"
+ changed_files = list(filter(lambda f: f.startswith(provider_path), changed_files))
+
if not changed_files:
return "other"
@@ -214,7 +217,7 @@ def classification_result(changed_files):
return "other"
-def classify_provider_pr_files(commit_hash: str) -> str:
+def classify_provider_pr_files(provider_id: str, commit_hash: str) -> str:
"""
Classify a provider commit based on changed files.
@@ -235,7 +238,7 @@ def classify_provider_pr_files(commit_hash: str) -> str:
# safe to return other here
return "other"
- return classification_result(changed_files)
+ return classification_result(provider_id, changed_files)
def _get_git_log_command(
@@ -827,7 +830,7 @@ def update_release_notes(
)
change = list_of_list_of_changes[0][table_iter]
- classification = classify_provider_pr_files(change.full_hash)
+ classification = classify_provider_pr_files(provider_id, change.full_hash)
if classification == "documentation":
get_console().print(
f"[green]Automatically classifying change as DOCUMENTATION since it contains only doc changes:[/]\n"
diff --git a/dev/breeze/tests/test_provider_documentation.py b/dev/breeze/tests/test_provider_documentation.py
index b55088987c6..6ff67e267d3 100644
--- a/dev/breeze/tests/test_provider_documentation.py
+++ b/dev/breeze/tests/test_provider_documentation.py
@@ -395,16 +395,20 @@ def test_get_most_impactful_change(changes, expected):
@pytest.mark.parametrize(
- "changed_files, expected",
+ "provider_id, changed_files, expected",
[
- pytest.param(["providers/slack/docs/slack.rst"], "documentation", id="only_docs"),
- pytest.param(["providers/slack/tests/test_slack.py"], "test_or_example_only", id="only_tests"),
+ pytest.param("slack", ["providers/slack/docs/slack.rst"], "documentation", id="only_docs"),
pytest.param(
+ "slack", ["providers/slack/tests/test_slack.py"], "test_or_example_only", id="only_tests"
+ ),
+ pytest.param(
+ "slack",
["providers/slack/src/airflow/providers/slack/example_dags/example_notify.py"],
"test_or_example_only",
id="only_example_dags",
),
pytest.param(
+ "slack",
[
"providers/slack/tests/test_slack.py",
"providers/slack/src/airflow/providers/slack/example_dags/example_notify.py",
@@ -413,6 +417,7 @@ def test_get_most_impactful_change(changes, expected):
id="tests_and_example_dags",
),
pytest.param(
+ "slack",
[
"providers/slack/tests/test_slack.py",
"providers/slack/docs/slack.rst",
@@ -421,6 +426,7 @@ def test_get_most_impactful_change(changes, expected):
id="docs_and_tests",
),
pytest.param(
+ "slack",
[
"providers/slack/src/airflow/providers/slack/hooks/slack.py",
"providers/slack/docs/slack.rst",
@@ -429,6 +435,7 @@ def test_get_most_impactful_change(changes, expected):
id="docs_and_real_code",
),
pytest.param(
+ "slack",
[
"providers/slack/src/airflow/providers/slack/hooks/slack.py",
"providers/slack/tests/test_slack.py",
@@ -436,10 +443,28 @@ def test_get_most_impactful_change(changes, expected):
"other",
id="real_code_and_tests",
),
- pytest.param(["airflow/utils/db.py"], "other", id="non_provider_file"),
- pytest.param([], "other", id="empty_commit"),
+ pytest.param(
+ "google",
+ [
+ "providers/google/tests/some_test.py",
+ "providers/amazon/tests/test_something.py",
+ ],
+ "test_or_example_only",
+ id="tests_in_multiple_providers",
+ ),
+ pytest.param(
+ "amazon",
+ [
+ "providers/google/tests/some_test.py",
+ "providers/amazon/tests/test_something.py",
+ ],
+ "test_or_example_only",
+ id="tests_in_multiple_providers",
+ ),
+ pytest.param("slack", ["airflow/utils/db.py"], "other", id="non_provider_file"),
+ pytest.param("slack", [], "other", id="empty_commit"),
],
)
-def test_classify_provider_pr_files_logic(changed_files, expected):
- result = classification_result(changed_files)
+def test_classify_provider_pr_files_logic(provider_id, changed_files, expected):
+ result = classification_result(provider_id, changed_files)
assert result == expected
(airflow) 01/12: [v3-0-test] Dont use builtin len as variable (#51992) (#53193)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 5b81b39b12a1e06d2840828e1ca0d337f8f8c0ba
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Fri Jul 11 16:12:13 2025 +0100
[v3-0-test] Dont use builtin len as variable (#51992) (#53193)
(cherry picked from commit 6eca233f)
Co-authored-by: GPK <go...@gmail.com>
---
airflow-core/src/airflow/jobs/triggerer_job_runner.py | 8 ++++----
task-sdk/src/airflow/sdk/execution_time/comms.py | 12 ++++++------
2 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/airflow-core/src/airflow/jobs/triggerer_job_runner.py b/airflow-core/src/airflow/jobs/triggerer_job_runner.py
index 512f8a4f0e6..95f8d58bf0d 100644
--- a/airflow-core/src/airflow/jobs/triggerer_job_runner.py
+++ b/airflow-core/src/airflow/jobs/triggerer_job_runner.py
@@ -714,11 +714,11 @@ class TriggerCommsDecoder(CommsDecoder[ToTriggerRunner, ToTriggerSupervisor]):
async def _aread_frame(self):
len_bytes = await self._async_reader.readexactly(4)
- len = int.from_bytes(len_bytes, byteorder="big")
- if len >= 2**32:
- raise OverflowError(f"Refusing to receive messages larger than 4GiB {len=}")
+ length = int.from_bytes(len_bytes, byteorder="big")
+ if length >= 2**32:
+ raise OverflowError(f"Refusing to receive messages larger than 4GiB {length=}")
- buffer = await self._async_reader.readexactly(len)
+ buffer = await self._async_reader.readexactly(length)
return self.resp_decoder.decode(buffer)
async def _aget_response(self, expect_id: int) -> ToTriggerRunner | None:
diff --git a/task-sdk/src/airflow/sdk/execution_time/comms.py b/task-sdk/src/airflow/sdk/execution_time/comms.py
index 0589f12f866..97ed1761ad8 100644
--- a/task-sdk/src/airflow/sdk/execution_time/comms.py
+++ b/task-sdk/src/airflow/sdk/execution_time/comms.py
@@ -183,9 +183,9 @@ class CommsDecoder(Generic[ReceiveMsgType, SendMsgType]):
def send(self, msg: SendMsgType) -> ReceiveMsgType | None:
"""Send a request to the parent and block until the response is received."""
frame = _RequestFrame(id=next(self.id_counter), body=msg.model_dump())
- bytes = frame.as_bytes()
+ frame_bytes = frame.as_bytes()
- self.socket.sendall(bytes)
+ self.socket.sendall(frame_bytes)
if isinstance(msg, ResendLoggingFD):
if recv_fds is None:
return None
@@ -225,13 +225,13 @@ class CommsDecoder(Generic[ReceiveMsgType, SendMsgType]):
if len_bytes == b"":
raise EOFError("Request socket closed before length")
- len = int.from_bytes(len_bytes, byteorder="big")
+ length = int.from_bytes(len_bytes, byteorder="big")
- buffer = bytearray(len)
+ buffer = bytearray(length)
nread = self.socket.recv_into(buffer)
- if nread != len:
+ if nread != length:
raise RuntimeError(
- f"unable to read full response in child. (We read {nread}, but expected {len})"
+ f"unable to read full response in child. (We read {nread}, but expected {length})"
)
if nread == 0:
raise EOFError(f"Request socket closed before response was complete ({self.id_counter=})")
(airflow) 08/12: [v3-0-test] Remove remnants of ~= used in requires-python. (#52985) (#52987)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 6e394358cd9bf4a6d6f37da49a3b024dca1a9d54
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Jul 7 19:39:09 2025 +0200
[v3-0-test] Remove remnants of ~= used in requires-python. (#52985) (#52987)
Follow up after #52980 - there are still few more places where
the ~= was used in requires-python.
(cherry picked from commit 3f6f1db510ca9f6641c0c1bca396a20c3f930d00)
---
airflow-core/pyproject.toml | 2 +-
scripts/ci/airflow_version_check.py | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/airflow-core/pyproject.toml b/airflow-core/pyproject.toml
index 8af8cd3b6fd..8e00ed531a1 100644
--- a/airflow-core/pyproject.toml
+++ b/airflow-core/pyproject.toml
@@ -25,7 +25,7 @@ requires = [
"pluggy==1.5.0",
"smmap==5.0.2",
"tomli==2.2.1; python_version < '3.11'",
- "trove-classifiers==2025.4.11.15",
+ "trove-classifiers==2025.5.9.12",
]
build-backend = "hatchling.build"
diff --git a/scripts/ci/airflow_version_check.py b/scripts/ci/airflow_version_check.py
index 45354d29884..cdeb807b7e6 100755
--- a/scripts/ci/airflow_version_check.py
+++ b/scripts/ci/airflow_version_check.py
@@ -63,9 +63,9 @@ def check_airflow_version(airflow_version: Version) -> tuple[str, bool]:
sys.exit(1)
if airflow_version == latest_version:
latest = True
- # find requires-python = "~=VERSION" in pyproject.toml file of airflow
+ # find requires-python = ">=VERSION" in pyproject.toml file of airflow
pyproject_toml_conntent = (Path(__file__).parents[2] / "pyproject.toml").read_text()
- matched_version = re.search('requires-python = "~=([0-9]+.[0-9]+)', pyproject_toml_conntent)
+ matched_version = re.search('requires-python = ">=([0-9]+.[0-9]+)', pyproject_toml_conntent)
if matched_version:
min_version = matched_version.group(1)
else: