You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2023/06/09 18:02:13 UTC
[airflow] 02/02: Remove yandex from 2.6 branch
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch v2-6-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit c3d61c7bb5e608bf13378be81a410487635d1526
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Fri Jun 9 19:54:37 2023 +0200
Remove yandex from 2.6 branch
---
.../providers/yandex/.latest-doc-only-change.txt | 1 -
airflow/providers/yandex/CHANGELOG.rst | 200 --------
airflow/providers/yandex/__init__.py | 16 -
airflow/providers/yandex/hooks/__init__.py | 16 -
airflow/providers/yandex/hooks/yandex.py | 176 -------
.../providers/yandex/hooks/yandexcloud_dataproc.py | 35 --
airflow/providers/yandex/operators/__init__.py | 16 -
.../yandex/operators/yandexcloud_dataproc.py | 525 ---------------------
airflow/providers/yandex/provider.yaml | 70 ---
generated/provider_dependencies.json | 8 -
images/breeze/output-commands-hash.txt | 10 +-
images/breeze/output_build-docs.svg | 4 +-
...management_generate-issue-content-providers.svg | 2 +-
...e-management_prepare-provider-documentation.svg | 2 +-
...elease-management_prepare-provider-packages.svg | 2 +-
15 files changed, 10 insertions(+), 1073 deletions(-)
diff --git a/airflow/providers/yandex/.latest-doc-only-change.txt b/airflow/providers/yandex/.latest-doc-only-change.txt
deleted file mode 100644
index ff7136e07d..0000000000
--- a/airflow/providers/yandex/.latest-doc-only-change.txt
+++ /dev/null
@@ -1 +0,0 @@
-06acf40a4337759797f666d5bb27a5a393b74fed
diff --git a/airflow/providers/yandex/CHANGELOG.rst b/airflow/providers/yandex/CHANGELOG.rst
deleted file mode 100644
index 76ee80f39e..0000000000
--- a/airflow/providers/yandex/CHANGELOG.rst
+++ /dev/null
@@ -1,200 +0,0 @@
- .. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- .. http://www.apache.org/licenses/LICENSE-2.0
-
- .. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-
-.. NOTE TO CONTRIBUTORS:
- Please, only add notes to the Changelog just below the "Changelog" header when there are some breaking changes
- and you want to add an explanation to the users on how they are supposed to deal with them.
- The changelog is updated and maintained semi-automatically by release manager.
-
-Changelog
----------
-
-3.3.0
-.....
-
-Features
-~~~~~~~~
-
-* ``support Yandex SDK feature "endpoint" (#29635)``
-
-3.2.0
-.....
-
-This release of provider is only available for Airflow 2.3+ as explained in the
-`Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/README.md#support-for-providers>`_.
-
-Features
-~~~~~~~~
-
-* In YandexCloudBaseHook, non-prefixed extra fields are supported and are preferred (#27040). E.g. ``folder_id`` will be preferred if ``extra__yandexcloud__folder_id`` is also present.
-
-Misc
-~~~~
-
-* ``Move min airflow version to 2.3.0 for all providers (#27196)``
-
-.. Below changes are excluded from the changelog. Move them to
- appropriate section above if needed. Do not delete the lines(!):
- * ``Enable string normalization in python formatting - providers (#27205)``
- * ``Update docs for September Provider's release (#26731)``
- * ``Apply PEP-563 (Postponed Evaluation of Annotations) to non-core airflow (#26289)``
- * ``pRepare docs for November 2022 wave of Providers (#27613)``
- * ``Prepare for follow-up release for November providers (#27774)``
-
-3.1.0
-.....
-
-Features
-~~~~~~~~
-
-* ``YandexCloud provider: Support new Yandex SDK features for DataProc (#25158)``
-
-.. Below changes are excluded from the changelog. Move them to
- appropriate section above if needed. Do not delete the lines(!):
- * ``Add documentation for July 2022 Provider's release (#25030)``
- * ``Move provider dependencies to inside provider folders (#24672)``
- * ``Remove 'hook-class-names' from provider.yaml (#24702)``
-
-3.0.0
-.....
-
-Breaking changes
-~~~~~~~~~~~~~~~~
-
-* This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow
- providers support policy https://github.com/apache/airflow/blob/main/README.md#support-for-providers
-
-.. Below changes are excluded from the changelog. Move them to
- appropriate section above if needed. Do not delete the lines(!):
- * ``Migrate Yandex example DAGs to new design AIP-47 (#24082)``
- * ``Add explanatory note for contributors about updating Changelog (#24229)``
- * ``Prepare docs for May 2022 provider's release (#24231)``
- * ``Update package description to remove double min-airflow specification (#24292)``
-
-2.2.3
-.....
-
-Bug Fixes
-~~~~~~~~~
-
-* ``Fix mistakenly added install_requires for all providers (#22382)``
-
-2.2.2
-.....
-
-Misc
-~~~~~
-
-* ``Add Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)``
-
-2.2.1
-.....
-
-Misc
-~~~~
-
-* ``Support for Python 3.10``
-
-.. Below changes are excluded from the changelog. Move them to
- appropriate section above if needed. Do not delete the lines(!):
- * ``Fixed changelog for January 2022 (delayed) provider's release (#21439)``
- * ``Add documentation for January 2021 providers release (#21257)``
- * ``Add optional features in providers. (#21074)``
- * ``Remove ':type' lines now sphinx-autoapi supports typehints (#20951)``
- * ``Fix spelling (#22054)``
-
-2.2.0
-.....
-
-Features
-~~~~~~~~
-
-* ``YandexCloud provider: Support new Yandex SDK features: log_group_id, user-agent, maven packages (#20103)``
-
-
-.. Below changes are excluded from the changelog. Move them to
- appropriate section above if needed. Do not delete the lines(!):
- * ``Fix mypy for providers: elasticsearch, oracle, yandex (#20344)``
- * ``Fixup string concatenations (#19099)``
- * ``Update documentation for November 2021 provider's release (#19882)``
- * ``Prepare documentation for October Provider's release (#19321)``
- * ``Update documentation for September providers release (#18613)``
- * ``Static start_date and default arg cleanup for misc. provider example DAGs (#18597)``
- * ``Inclusive Language (#18349)``
- * ``Use typed Context EVERYWHERE (#20565)``
- * ``Fix template_fields type to have MyPy friendly Sequence type (#20571)``
- * ``Update documentation for provider December 2021 release (#20523)``
-
-2.1.0
-.....
-
-Misc
-~~~~
-
-* ``Optimise connection importing for Airflow 2.2.0``
-
-
-Features
-~~~~~~~~
-
-* ``Add autoscaling subcluster support and remove defaults (#17033)``
-
-
-.. Below changes are excluded from the changelog. Move them to
- appropriate section above if needed. Do not delete the lines(!):
- * ``Update description about the new ''connection-types'' provider meta-data (#17767)``
- * ``Import Hooks lazily individually in providers manager (#17682)``
- * ``Prepares docs for Rc2 release of July providers (#17116)``
- * ``Remove/refactor default_args pattern for miscellaneous providers (#16872)``
- * ``Prepare documentation for July release of providers. (#17015)``
- * ``Removes pylint from our toolchain (#16682)``
-
-2.0.0
-.....
-
-Breaking changes
-~~~~~~~~~~~~~~~~
-
-* ``Auto-apply apply_default decorator (#15667)``
-
-.. warning:: Due to apply_default decorator removal, this version of the provider requires Airflow 2.1.0+.
- If your Airflow version is < 2.1.0, and you want to install this provider version, first upgrade
- Airflow to at least version 2.1.0. Otherwise your Airflow package version will be upgraded
- automatically and you will have to manually run ``airflow upgrade db`` to complete the migration.
-
-.. Below changes are excluded from the changelog. Move them to
- appropriate section above if needed. Do not delete the lines(!):
- * ``Adds interactivity when generating provider documentation. (#15518)``
- * ``Prepares provider release after PIP 21 compatibility (#15576)``
- * ``Update docstrings to adhere to sphinx standards (#14918)``
- * ``Remove Backport Providers (#14886)``
- * ``Update documentation for broken package releases (#14734)``
- * ``Updated documentation for June 2021 provider release (#16294)``
- * ``Fix Sphinx Issues with Docstrings (#14968)``
- * ``More documentation update for June providers release (#16405)``
- * ``Synchronizes updated changelog after buggfix release (#16464)``
-
-1.0.1
-.....
-
-Updated documentation and readme files.
-
-1.0.0
-.....
-
-Initial version of the provider.
diff --git a/airflow/providers/yandex/__init__.py b/airflow/providers/yandex/__init__.py
deleted file mode 100644
index 13a83393a9..0000000000
--- a/airflow/providers/yandex/__init__.py
+++ /dev/null
@@ -1,16 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
diff --git a/airflow/providers/yandex/hooks/__init__.py b/airflow/providers/yandex/hooks/__init__.py
deleted file mode 100644
index 13a83393a9..0000000000
--- a/airflow/providers/yandex/hooks/__init__.py
+++ /dev/null
@@ -1,16 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
diff --git a/airflow/providers/yandex/hooks/yandex.py b/airflow/providers/yandex/hooks/yandex.py
deleted file mode 100644
index d211d79740..0000000000
--- a/airflow/providers/yandex/hooks/yandex.py
+++ /dev/null
@@ -1,176 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-from __future__ import annotations
-
-import json
-import warnings
-from typing import Any
-
-import yandexcloud
-
-from airflow.exceptions import AirflowException
-from airflow.hooks.base import BaseHook
-
-
-class YandexCloudBaseHook(BaseHook):
- """
- A base hook for Yandex.Cloud related tasks.
-
- :param yandex_conn_id: The connection ID to use when fetching connection info.
- """
-
- conn_name_attr = "yandex_conn_id"
- default_conn_name = "yandexcloud_default"
- conn_type = "yandexcloud"
- hook_name = "Yandex Cloud"
-
- @staticmethod
- def get_connection_form_widgets() -> dict[str, Any]:
- """Returns connection widgets to add to connection form"""
- from flask_appbuilder.fieldwidgets import BS3PasswordFieldWidget, BS3TextFieldWidget
- from flask_babel import lazy_gettext
- from wtforms import PasswordField, StringField
-
- return {
- "service_account_json": PasswordField(
- lazy_gettext("Service account auth JSON"),
- widget=BS3PasswordFieldWidget(),
- description="Service account auth JSON. Looks like "
- '{"id", "...", "service_account_id": "...", "private_key": "..."}. '
- "Will be used instead of OAuth token and SA JSON file path field if specified.",
- ),
- "service_account_json_path": StringField(
- lazy_gettext("Service account auth JSON file path"),
- widget=BS3TextFieldWidget(),
- description="Service account auth JSON file path. File content looks like "
- '{"id", "...", "service_account_id": "...", "private_key": "..."}. '
- "Will be used instead of OAuth token if specified.",
- ),
- "oauth": PasswordField(
- lazy_gettext("OAuth Token"),
- widget=BS3PasswordFieldWidget(),
- description="User account OAuth token. "
- "Either this or service account JSON must be specified.",
- ),
- "folder_id": StringField(
- lazy_gettext("Default folder ID"),
- widget=BS3TextFieldWidget(),
- description="Optional. This folder will be used "
- "to create all new clusters and nodes by default",
- ),
- "public_ssh_key": StringField(
- lazy_gettext("Public SSH key"),
- widget=BS3TextFieldWidget(),
- description="Optional. This key will be placed to all created Compute nodes"
- "to let you have a root shell there",
- ),
- "endpoint": StringField(
- lazy_gettext("API endpoint"),
- widget=BS3TextFieldWidget(),
- description="Optional. Specify an API endpoint. Leave blank to use default.",
- ),
- }
-
- @classmethod
- def provider_user_agent(cls) -> str | None:
- """Construct User-Agent from Airflow core & provider package versions"""
- import airflow
- from airflow.providers_manager import ProvidersManager
-
- try:
- manager = ProvidersManager()
- provider_name = manager.hooks[cls.conn_type].package_name # type: ignore[union-attr]
- provider = manager.providers[provider_name]
- return f"apache-airflow/{airflow.__version__} {provider_name}/{provider.version}"
- except KeyError:
- warnings.warn(f"Hook '{cls.hook_name}' info is not initialized in airflow.ProviderManager")
- return None
-
- @staticmethod
- def get_ui_field_behaviour() -> dict[str, Any]:
- """Returns custom field behaviour"""
- return {
- "hidden_fields": ["host", "schema", "login", "password", "port", "extra"],
- "relabeling": {},
- }
-
- def __init__(
- self,
- # Connection id is deprecated. Use yandex_conn_id instead
- connection_id: str | None = None,
- yandex_conn_id: str | None = None,
- default_folder_id: str | None = None,
- default_public_ssh_key: str | None = None,
- ) -> None:
- super().__init__()
- if connection_id:
- warnings.warn(
- "Using `connection_id` is deprecated. Please use `yandex_conn_id` parameter.",
- DeprecationWarning,
- stacklevel=2,
- )
- self.connection_id = yandex_conn_id or connection_id or self.default_conn_name
- self.connection = self.get_connection(self.connection_id)
- self.extras = self.connection.extra_dejson
- credentials = self._get_credentials()
- sdk_config = self._get_endpoint()
- self.sdk = yandexcloud.SDK(user_agent=self.provider_user_agent(), **sdk_config, **credentials)
- self.default_folder_id = default_folder_id or self._get_field("folder_id", False)
- self.default_public_ssh_key = default_public_ssh_key or self._get_field("public_ssh_key", False)
- self.client = self.sdk.client
-
- def _get_credentials(self) -> dict[str, Any]:
- service_account_json_path = self._get_field("service_account_json_path", False)
- service_account_json = self._get_field("service_account_json", False)
- oauth_token = self._get_field("oauth", False)
- if not (service_account_json or oauth_token or service_account_json_path):
- raise AirflowException(
- "No credentials are found in connection. Specify either service account "
- "authentication JSON or user OAuth token in Yandex.Cloud connection"
- )
- if service_account_json_path:
- with open(service_account_json_path) as infile:
- service_account_json = infile.read()
- if service_account_json:
- service_account_key = json.loads(service_account_json)
- return {"service_account_key": service_account_key}
- else:
- return {"token": oauth_token}
-
- def _get_endpoint(self) -> dict[str, str]:
- sdk_config = {}
- endpoint = self._get_field("endpoint", None)
- if endpoint:
- sdk_config["endpoint"] = endpoint
- return sdk_config
-
- def _get_field(self, field_name: str, default: Any = None) -> Any:
- """Get field from extra, first checking short name, then for backcompat we check for prefixed name."""
- if not hasattr(self, "extras"):
- return default
- backcompat_prefix = "extra__yandexcloud__"
- if field_name.startswith("extra__"):
- raise ValueError(
- f"Got prefixed name {field_name}; please remove the '{backcompat_prefix}' prefix "
- "when using this method."
- )
- if field_name in self.extras:
- return self.extras[field_name]
- prefixed_name = f"{backcompat_prefix}{field_name}"
- if prefixed_name in self.extras:
- return self.extras[prefixed_name]
- return default
diff --git a/airflow/providers/yandex/hooks/yandexcloud_dataproc.py b/airflow/providers/yandex/hooks/yandexcloud_dataproc.py
deleted file mode 100644
index 9b1862205e..0000000000
--- a/airflow/providers/yandex/hooks/yandexcloud_dataproc.py
+++ /dev/null
@@ -1,35 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-from __future__ import annotations
-
-from airflow.providers.yandex.hooks.yandex import YandexCloudBaseHook
-
-
-class DataprocHook(YandexCloudBaseHook):
- """
- A base hook for Yandex.Cloud Data Proc.
-
- :param yandex_conn_id: The connection ID to use when fetching connection info.
- """
-
- def __init__(self, *args, **kwargs) -> None:
- super().__init__(*args, **kwargs)
- self.cluster_id = None
- self.client = self.sdk.wrappers.Dataproc(
- default_folder_id=self.default_folder_id,
- default_public_ssh_key=self.default_public_ssh_key,
- )
diff --git a/airflow/providers/yandex/operators/__init__.py b/airflow/providers/yandex/operators/__init__.py
deleted file mode 100644
index 13a83393a9..0000000000
--- a/airflow/providers/yandex/operators/__init__.py
+++ /dev/null
@@ -1,16 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
diff --git a/airflow/providers/yandex/operators/yandexcloud_dataproc.py b/airflow/providers/yandex/operators/yandexcloud_dataproc.py
deleted file mode 100644
index 625827d109..0000000000
--- a/airflow/providers/yandex/operators/yandexcloud_dataproc.py
+++ /dev/null
@@ -1,525 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-from __future__ import annotations
-
-import warnings
-from dataclasses import dataclass
-from typing import TYPE_CHECKING, Iterable, Sequence
-
-from airflow.models import BaseOperator
-from airflow.providers.yandex.hooks.yandexcloud_dataproc import DataprocHook
-
-if TYPE_CHECKING:
- from airflow.utils.context import Context
-
-
-@dataclass
-class InitializationAction:
- """Data for initialization action to be run at start of DataProc cluster."""
-
- uri: str # Uri of the executable file
- args: Sequence[str] # Arguments to the initialization action
- timeout: int # Execution timeout
-
-
-class DataprocCreateClusterOperator(BaseOperator):
- """Creates Yandex.Cloud Data Proc cluster.
-
- :param folder_id: ID of the folder in which cluster should be created.
- :param cluster_name: Cluster name. Must be unique inside the folder.
- :param cluster_description: Cluster description.
- :param cluster_image_version: Cluster image version. Use default.
- :param ssh_public_keys: List of SSH public keys that will be deployed to created compute instances.
- :param subnet_id: ID of the subnetwork. All Data Proc cluster nodes will use one subnetwork.
- :param services: List of services that will be installed to the cluster. Possible options:
- HDFS, YARN, MAPREDUCE, HIVE, TEZ, ZOOKEEPER, HBASE, SQOOP, FLUME, SPARK, SPARK, ZEPPELIN, OOZIE
- :param s3_bucket: Yandex.Cloud S3 bucket to store cluster logs.
- Jobs will not work if the bucket is not specified.
- :param zone: Availability zone to create cluster in.
- Currently there are ru-central1-a, ru-central1-b and ru-central1-c.
- :param service_account_id: Service account id for the cluster.
- Service account can be created inside the folder.
- :param masternode_resource_preset: Resources preset (CPU+RAM configuration)
- for the primary node of the cluster.
- :param masternode_disk_size: Masternode storage size in GiB.
- :param masternode_disk_type: Masternode storage type. Possible options: network-ssd, network-hdd.
- :param datanode_resource_preset: Resources preset (CPU+RAM configuration)
- for the data nodes of the cluster.
- :param datanode_disk_size: Datanodes storage size in GiB.
- :param datanode_disk_type: Datanodes storage type. Possible options: network-ssd, network-hdd.
- :param computenode_resource_preset: Resources preset (CPU+RAM configuration)
- for the compute nodes of the cluster.
- :param computenode_disk_size: Computenodes storage size in GiB.
- :param computenode_disk_type: Computenodes storage type. Possible options: network-ssd, network-hdd.
- :param connection_id: ID of the Yandex.Cloud Airflow connection.
- :param computenode_max_count: Maximum number of nodes of compute autoscaling subcluster.
- :param computenode_warmup_duration: The warmup time of the instance in seconds. During this time,
- traffic is sent to the instance,
- but instance metrics are not collected. In seconds.
- :param computenode_stabilization_duration: Minimum amount of time in seconds for monitoring before
- Instance Groups can reduce the number of instances in the group.
- During this time, the group size doesn't decrease,
- even if the new metric values indicate that it should. In seconds.
- :param computenode_preemptible: Preemptible instances are stopped at least once every 24 hours,
- and can be stopped at any time if their resources are needed by Compute.
- :param computenode_cpu_utilization_target: Defines an autoscaling rule
- based on the average CPU utilization of the instance group.
- in percents. 10-100.
- By default is not set and default autoscaling strategy is used.
- :param computenode_decommission_timeout: Timeout to gracefully decommission nodes during downscaling.
- In seconds
- :param properties: Properties passed to main node software.
- Docs: https://cloud.yandex.com/docs/data-proc/concepts/settings-list
- :param enable_ui_proxy: Enable UI Proxy feature for forwarding Hadoop components web interfaces
- Docs: https://cloud.yandex.com/docs/data-proc/concepts/ui-proxy
- :param host_group_ids: Dedicated host groups to place VMs of cluster on.
- Docs: https://cloud.yandex.com/docs/compute/concepts/dedicated-host
- :param security_group_ids: User security groups.
- Docs: https://cloud.yandex.com/docs/data-proc/concepts/network#security-groups
- :param log_group_id: Id of log group to write logs. By default logs will be sent to default log group.
- To disable cloud log sending set cluster property dataproc:disable_cloud_logging = true
- Docs: https://cloud.yandex.com/docs/data-proc/concepts/logs
- :param initialization_actions: Set of init-actions to run when cluster starts.
- Docs: https://cloud.yandex.com/docs/data-proc/concepts/init-action
- """
-
- def __init__(
- self,
- *,
- folder_id: str | None = None,
- cluster_name: str | None = None,
- cluster_description: str | None = "",
- cluster_image_version: str | None = None,
- ssh_public_keys: str | Iterable[str] | None = None,
- subnet_id: str | None = None,
- services: Iterable[str] = ("HDFS", "YARN", "MAPREDUCE", "HIVE", "SPARK"),
- s3_bucket: str | None = None,
- zone: str = "ru-central1-b",
- service_account_id: str | None = None,
- masternode_resource_preset: str | None = None,
- masternode_disk_size: int | None = None,
- masternode_disk_type: str | None = None,
- datanode_resource_preset: str | None = None,
- datanode_disk_size: int | None = None,
- datanode_disk_type: str | None = None,
- datanode_count: int = 1,
- computenode_resource_preset: str | None = None,
- computenode_disk_size: int | None = None,
- computenode_disk_type: str | None = None,
- computenode_count: int = 0,
- computenode_max_hosts_count: int | None = None,
- computenode_measurement_duration: int | None = None,
- computenode_warmup_duration: int | None = None,
- computenode_stabilization_duration: int | None = None,
- computenode_preemptible: bool = False,
- computenode_cpu_utilization_target: int | None = None,
- computenode_decommission_timeout: int | None = None,
- connection_id: str | None = None,
- properties: dict[str, str] | None = None,
- enable_ui_proxy: bool = False,
- host_group_ids: Iterable[str] | None = None,
- security_group_ids: Iterable[str] | None = None,
- log_group_id: str | None = None,
- initialization_actions: Iterable[InitializationAction] | None = None,
- **kwargs,
- ) -> None:
- super().__init__(**kwargs)
- self.folder_id = folder_id
- self.yandex_conn_id = connection_id
- self.cluster_name = cluster_name
- self.cluster_description = cluster_description
- self.cluster_image_version = cluster_image_version
- self.ssh_public_keys = ssh_public_keys
- self.subnet_id = subnet_id
- self.services = services
- self.s3_bucket = s3_bucket
- self.zone = zone
- self.service_account_id = service_account_id
- self.masternode_resource_preset = masternode_resource_preset
- self.masternode_disk_size = masternode_disk_size
- self.masternode_disk_type = masternode_disk_type
- self.datanode_resource_preset = datanode_resource_preset
- self.datanode_disk_size = datanode_disk_size
- self.datanode_disk_type = datanode_disk_type
- self.datanode_count = datanode_count
- self.computenode_resource_preset = computenode_resource_preset
- self.computenode_disk_size = computenode_disk_size
- self.computenode_disk_type = computenode_disk_type
- self.computenode_count = computenode_count
- self.computenode_max_hosts_count = computenode_max_hosts_count
- self.computenode_measurement_duration = computenode_measurement_duration
- self.computenode_warmup_duration = computenode_warmup_duration
- self.computenode_stabilization_duration = computenode_stabilization_duration
- self.computenode_preemptible = computenode_preemptible
- self.computenode_cpu_utilization_target = computenode_cpu_utilization_target
- self.computenode_decommission_timeout = computenode_decommission_timeout
- self.properties = properties
- self.enable_ui_proxy = enable_ui_proxy
- self.host_group_ids = host_group_ids
- self.security_group_ids = security_group_ids
- self.log_group_id = log_group_id
- self.initialization_actions = initialization_actions
-
- self.hook: DataprocHook | None = None
-
- def execute(self, context: Context) -> dict:
- self.hook = DataprocHook(
- yandex_conn_id=self.yandex_conn_id,
- )
- operation_result = self.hook.client.create_cluster(
- folder_id=self.folder_id,
- cluster_name=self.cluster_name,
- cluster_description=self.cluster_description,
- cluster_image_version=self.cluster_image_version,
- ssh_public_keys=self.ssh_public_keys,
- subnet_id=self.subnet_id,
- services=self.services,
- s3_bucket=self.s3_bucket,
- zone=self.zone,
- service_account_id=self.service_account_id,
- masternode_resource_preset=self.masternode_resource_preset,
- masternode_disk_size=self.masternode_disk_size,
- masternode_disk_type=self.masternode_disk_type,
- datanode_resource_preset=self.datanode_resource_preset,
- datanode_disk_size=self.datanode_disk_size,
- datanode_disk_type=self.datanode_disk_type,
- datanode_count=self.datanode_count,
- computenode_resource_preset=self.computenode_resource_preset,
- computenode_disk_size=self.computenode_disk_size,
- computenode_disk_type=self.computenode_disk_type,
- computenode_count=self.computenode_count,
- computenode_max_hosts_count=self.computenode_max_hosts_count,
- computenode_measurement_duration=self.computenode_measurement_duration,
- computenode_warmup_duration=self.computenode_warmup_duration,
- computenode_stabilization_duration=self.computenode_stabilization_duration,
- computenode_preemptible=self.computenode_preemptible,
- computenode_cpu_utilization_target=self.computenode_cpu_utilization_target,
- computenode_decommission_timeout=self.computenode_decommission_timeout,
- properties=self.properties,
- enable_ui_proxy=self.enable_ui_proxy,
- host_group_ids=self.host_group_ids,
- security_group_ids=self.security_group_ids,
- log_group_id=self.log_group_id,
- initialization_actions=self.initialization_actions
- and [
- self.hook.sdk.wrappers.InitializationAction(
- uri=init_action.uri,
- args=init_action.args,
- timeout=init_action.timeout,
- )
- for init_action in self.initialization_actions
- ],
- )
- cluster_id = operation_result.response.id
-
- context["task_instance"].xcom_push(key="cluster_id", value=cluster_id)
- # Deprecated
- context["task_instance"].xcom_push(key="yandexcloud_connection_id", value=self.yandex_conn_id)
- return cluster_id
-
- @property
- def cluster_id(self):
- return self.output
-
-
-class DataprocBaseOperator(BaseOperator):
- """Base class for DataProc operators working with given cluster.
-
- :param connection_id: ID of the Yandex.Cloud Airflow connection.
- :param cluster_id: ID of the cluster to remove. (templated)
- """
-
- template_fields: Sequence[str] = ("cluster_id",)
-
- def __init__(self, *, yandex_conn_id: str | None = None, cluster_id: str | None = None, **kwargs) -> None:
- super().__init__(**kwargs)
- self.cluster_id = cluster_id
- self.yandex_conn_id = yandex_conn_id
-
- def _setup(self, context: Context) -> DataprocHook:
- if self.cluster_id is None:
- self.cluster_id = context["task_instance"].xcom_pull(key="cluster_id")
- if self.yandex_conn_id is None:
- xcom_yandex_conn_id = context["task_instance"].xcom_pull(key="yandexcloud_connection_id")
- if xcom_yandex_conn_id:
- warnings.warn("Implicit pass of `yandex_conn_id` is deprecated, please pass it explicitly")
- self.yandex_conn_id = xcom_yandex_conn_id
-
- return DataprocHook(yandex_conn_id=self.yandex_conn_id)
-
- def execute(self, context: Context):
- raise NotImplementedError()
-
-
-class DataprocDeleteClusterOperator(DataprocBaseOperator):
- """Deletes Yandex.Cloud Data Proc cluster.
-
- :param connection_id: ID of the Yandex.Cloud Airflow connection.
- :param cluster_id: ID of the cluster to remove. (templated)
- """
-
- def __init__(self, *, connection_id: str | None = None, cluster_id: str | None = None, **kwargs) -> None:
- super().__init__(yandex_conn_id=connection_id, cluster_id=cluster_id, **kwargs)
-
- def execute(self, context: Context) -> None:
- hook = self._setup(context)
- hook.client.delete_cluster(self.cluster_id)
-
-
-class DataprocCreateHiveJobOperator(DataprocBaseOperator):
- """Runs Hive job in Data Proc cluster.
-
- :param query: Hive query.
- :param query_file_uri: URI of the script that contains Hive queries. Can be placed in HDFS or S3.
- :param properties: A mapping of property names to values, used to configure Hive.
- :param script_variables: Mapping of query variable names to values.
- :param continue_on_failure: Whether to continue executing queries if a query fails.
- :param name: Name of the job. Used for labeling.
- :param cluster_id: ID of the cluster to run job in.
- Will try to take the ID from Dataproc Hook object if it's specified. (templated)
- :param connection_id: ID of the Yandex.Cloud Airflow connection.
- """
-
- def __init__(
- self,
- *,
- query: str | None = None,
- query_file_uri: str | None = None,
- script_variables: dict[str, str] | None = None,
- continue_on_failure: bool = False,
- properties: dict[str, str] | None = None,
- name: str = "Hive job",
- cluster_id: str | None = None,
- connection_id: str | None = None,
- **kwargs,
- ) -> None:
- super().__init__(yandex_conn_id=connection_id, cluster_id=cluster_id, **kwargs)
- self.query = query
- self.query_file_uri = query_file_uri
- self.script_variables = script_variables
- self.continue_on_failure = continue_on_failure
- self.properties = properties
- self.name = name
-
- def execute(self, context: Context) -> None:
- hook = self._setup(context)
- hook.client.create_hive_job(
- query=self.query,
- query_file_uri=self.query_file_uri,
- script_variables=self.script_variables,
- continue_on_failure=self.continue_on_failure,
- properties=self.properties,
- name=self.name,
- cluster_id=self.cluster_id,
- )
-
-
-class DataprocCreateMapReduceJobOperator(DataprocBaseOperator):
- """Runs Mapreduce job in Data Proc cluster.
-
- :param main_jar_file_uri: URI of jar file with job.
- Can be placed in HDFS or S3. Can be specified instead of main_class.
- :param main_class: Name of the main class of the job. Can be specified instead of main_jar_file_uri.
- :param file_uris: URIs of files used in the job. Can be placed in HDFS or S3.
- :param archive_uris: URIs of archive files used in the job. Can be placed in HDFS or S3.
- :param jar_file_uris: URIs of JAR files used in the job. Can be placed in HDFS or S3.
- :param properties: Properties for the job.
- :param args: Arguments to be passed to the job.
- :param name: Name of the job. Used for labeling.
- :param cluster_id: ID of the cluster to run job in.
- Will try to take the ID from Dataproc Hook object if it's specified. (templated)
- :param connection_id: ID of the Yandex.Cloud Airflow connection.
- """
-
- def __init__(
- self,
- *,
- main_class: str | None = None,
- main_jar_file_uri: str | None = None,
- jar_file_uris: Iterable[str] | None = None,
- archive_uris: Iterable[str] | None = None,
- file_uris: Iterable[str] | None = None,
- args: Iterable[str] | None = None,
- properties: dict[str, str] | None = None,
- name: str = "Mapreduce job",
- cluster_id: str | None = None,
- connection_id: str | None = None,
- **kwargs,
- ) -> None:
- super().__init__(yandex_conn_id=connection_id, cluster_id=cluster_id, **kwargs)
- self.main_class = main_class
- self.main_jar_file_uri = main_jar_file_uri
- self.jar_file_uris = jar_file_uris
- self.archive_uris = archive_uris
- self.file_uris = file_uris
- self.args = args
- self.properties = properties
- self.name = name
-
- def execute(self, context: Context) -> None:
- hook = self._setup(context)
- hook.client.create_mapreduce_job(
- main_class=self.main_class,
- main_jar_file_uri=self.main_jar_file_uri,
- jar_file_uris=self.jar_file_uris,
- archive_uris=self.archive_uris,
- file_uris=self.file_uris,
- args=self.args,
- properties=self.properties,
- name=self.name,
- cluster_id=self.cluster_id,
- )
-
-
-class DataprocCreateSparkJobOperator(DataprocBaseOperator):
- """Runs Spark job in Data Proc cluster.
-
- :param main_jar_file_uri: URI of jar file with job. Can be placed in HDFS or S3.
- :param main_class: Name of the main class of the job.
- :param file_uris: URIs of files used in the job. Can be placed in HDFS or S3.
- :param archive_uris: URIs of archive files used in the job. Can be placed in HDFS or S3.
- :param jar_file_uris: URIs of JAR files used in the job. Can be placed in HDFS or S3.
- :param properties: Properties for the job.
- :param args: Arguments to be passed to the job.
- :param name: Name of the job. Used for labeling.
- :param cluster_id: ID of the cluster to run job in.
- Will try to take the ID from Dataproc Hook object if it's specified. (templated)
- :param connection_id: ID of the Yandex.Cloud Airflow connection.
- :param packages: List of maven coordinates of jars to include on the driver and executor classpaths.
- :param repositories: List of additional remote repositories to search for the maven coordinates
- given with --packages.
- :param exclude_packages: List of groupId:artifactId, to exclude while resolving the dependencies
- provided in --packages to avoid dependency conflicts.
- """
-
- def __init__(
- self,
- *,
- main_class: str | None = None,
- main_jar_file_uri: str | None = None,
- jar_file_uris: Iterable[str] | None = None,
- archive_uris: Iterable[str] | None = None,
- file_uris: Iterable[str] | None = None,
- args: Iterable[str] | None = None,
- properties: dict[str, str] | None = None,
- name: str = "Spark job",
- cluster_id: str | None = None,
- connection_id: str | None = None,
- packages: Iterable[str] | None = None,
- repositories: Iterable[str] | None = None,
- exclude_packages: Iterable[str] | None = None,
- **kwargs,
- ) -> None:
- super().__init__(yandex_conn_id=connection_id, cluster_id=cluster_id, **kwargs)
- self.main_class = main_class
- self.main_jar_file_uri = main_jar_file_uri
- self.jar_file_uris = jar_file_uris
- self.archive_uris = archive_uris
- self.file_uris = file_uris
- self.args = args
- self.properties = properties
- self.name = name
- self.packages = packages
- self.repositories = repositories
- self.exclude_packages = exclude_packages
-
- def execute(self, context: Context) -> None:
- hook = self._setup(context)
- hook.client.create_spark_job(
- main_class=self.main_class,
- main_jar_file_uri=self.main_jar_file_uri,
- jar_file_uris=self.jar_file_uris,
- archive_uris=self.archive_uris,
- file_uris=self.file_uris,
- args=self.args,
- properties=self.properties,
- packages=self.packages,
- repositories=self.repositories,
- exclude_packages=self.exclude_packages,
- name=self.name,
- cluster_id=self.cluster_id,
- )
-
-
-class DataprocCreatePysparkJobOperator(DataprocBaseOperator):
- """Runs Pyspark job in Data Proc cluster.
-
- :param main_python_file_uri: URI of python file with job. Can be placed in HDFS or S3.
- :param python_file_uris: URIs of python files used in the job. Can be placed in HDFS or S3.
- :param file_uris: URIs of files used in the job. Can be placed in HDFS or S3.
- :param archive_uris: URIs of archive files used in the job. Can be placed in HDFS or S3.
- :param jar_file_uris: URIs of JAR files used in the job. Can be placed in HDFS or S3.
- :param properties: Properties for the job.
- :param args: Arguments to be passed to the job.
- :param name: Name of the job. Used for labeling.
- :param cluster_id: ID of the cluster to run job in.
- Will try to take the ID from Dataproc Hook object if it's specified. (templated)
- :param connection_id: ID of the Yandex.Cloud Airflow connection.
- :param packages: List of maven coordinates of jars to include on the driver and executor classpaths.
- :param repositories: List of additional remote repositories to search for the maven coordinates
- given with --packages.
- :param exclude_packages: List of groupId:artifactId, to exclude while resolving the dependencies
- provided in --packages to avoid dependency conflicts.
- """
-
- def __init__(
- self,
- *,
- main_python_file_uri: str | None = None,
- python_file_uris: Iterable[str] | None = None,
- jar_file_uris: Iterable[str] | None = None,
- archive_uris: Iterable[str] | None = None,
- file_uris: Iterable[str] | None = None,
- args: Iterable[str] | None = None,
- properties: dict[str, str] | None = None,
- name: str = "Pyspark job",
- cluster_id: str | None = None,
- connection_id: str | None = None,
- packages: Iterable[str] | None = None,
- repositories: Iterable[str] | None = None,
- exclude_packages: Iterable[str] | None = None,
- **kwargs,
- ) -> None:
- super().__init__(yandex_conn_id=connection_id, cluster_id=cluster_id, **kwargs)
- self.main_python_file_uri = main_python_file_uri
- self.python_file_uris = python_file_uris
- self.jar_file_uris = jar_file_uris
- self.archive_uris = archive_uris
- self.file_uris = file_uris
- self.args = args
- self.properties = properties
- self.name = name
- self.packages = packages
- self.repositories = repositories
- self.exclude_packages = exclude_packages
-
- def execute(self, context: Context) -> None:
- hook = self._setup(context)
- hook.client.create_pyspark_job(
- main_python_file_uri=self.main_python_file_uri,
- python_file_uris=self.python_file_uris,
- jar_file_uris=self.jar_file_uris,
- archive_uris=self.archive_uris,
- file_uris=self.file_uris,
- args=self.args,
- properties=self.properties,
- packages=self.packages,
- repositories=self.repositories,
- exclude_packages=self.exclude_packages,
- name=self.name,
- cluster_id=self.cluster_id,
- )
diff --git a/airflow/providers/yandex/provider.yaml b/airflow/providers/yandex/provider.yaml
deleted file mode 100644
index 6aa8eec209..0000000000
--- a/airflow/providers/yandex/provider.yaml
+++ /dev/null
@@ -1,70 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
----
-package-name: apache-airflow-providers-yandex
-name: Yandex
-description: |
- Yandex including `Yandex.Cloud <https://cloud.yandex.com/>`__
-suspended: false
-versions:
- - 3.3.0
- - 3.2.0
- - 3.1.0
- - 3.0.0
- - 2.2.3
- - 2.2.2
- - 2.2.1
- - 2.2.0
- - 2.1.0
- - 2.0.0
- - 1.0.1
- - 1.0.0
-
-dependencies:
- - apache-airflow>=2.3.0
- - yandexcloud>=0.173.0
-
-integrations:
- - integration-name: Yandex.Cloud
- external-doc-url: https://cloud.yandex.com/
- logo: /integration-logos/yandex/Yandex-Cloud.png
- tags: [service]
-
- - integration-name: Yandex.Cloud Dataproc
- external-doc-url: https://cloud.yandex.com/dataproc
- how-to-guide:
- - /docs/apache-airflow-providers-yandex/operators.rst
- logo: /integration-logos/yandex/Yandex-Cloud.png
- tags: [service]
-
-operators:
- - integration-name: Yandex.Cloud Dataproc
- python-modules:
- - airflow.providers.yandex.operators.yandexcloud_dataproc
-
-hooks:
- - integration-name: Yandex.Cloud
- python-modules:
- - airflow.providers.yandex.hooks.yandex
- - integration-name: Yandex.Cloud Dataproc
- python-modules:
- - airflow.providers.yandex.hooks.yandexcloud_dataproc
-
-connection-types:
- - hook-class-name: airflow.providers.yandex.hooks.yandex.YandexCloudBaseHook
- connection-type: yandexcloud
diff --git a/generated/provider_dependencies.json b/generated/provider_dependencies.json
index afd6c0e39d..ab399cfe20 100644
--- a/generated/provider_dependencies.json
+++ b/generated/provider_dependencies.json
@@ -837,14 +837,6 @@
],
"excluded-python-versions": []
},
- "yandex": {
- "deps": [
- "apache-airflow>=2.3.0",
- "yandexcloud>=0.173.0"
- ],
- "cross-providers-deps": [],
- "excluded-python-versions": []
- },
"zendesk": {
"deps": [
"apache-airflow>=2.3.0",
diff --git a/images/breeze/output-commands-hash.txt b/images/breeze/output-commands-hash.txt
index 80b78b02bc..0cb17dff19 100644
--- a/images/breeze/output-commands-hash.txt
+++ b/images/breeze/output-commands-hash.txt
@@ -2,7 +2,7 @@
# Please do not solve it but run `breeze setup regenerate-command-images`.
# This command should fix the conflict and regenerate help images that you have conflict with.
main:5bd7ecf7e8fdd34879f24ad8ac390405
-build-docs:093cf82ab80aee65c6ada9b16ac60612
+build-docs:077f48da6be5a16a82deea847130c876
ci:fix-ownership:fee2c9ec9ef19686792002ae054fecdd
ci:free-space:47234aa0a60b0efd84972e6e797379f8
ci:get-workflow-info:01ee34c33ad62fa5dc33e0ac8773223f
@@ -37,16 +37,16 @@ prod-image:verify:e7af8401f1d1fc90075d8f6e8215a9f4
prod-image:d5cb477a8ba7fa861301fa24db49d2ae
release-management:create-minor-branch:6a01066dce15e09fb269a8385626657c
release-management:generate-constraints:0c8e5c1b57b486724760d4afdfd571f7
-release-management:generate-issue-content-providers:f3c00ba74e3afc054fe29b65156740ac
+release-management:generate-issue-content-providers:20723a5f69ef3718c9b562fad302380a
release-management:install-provider-packages:b05446600134e8f0eca5cf8c6c1de856
release-management:prepare-airflow-package:3ac14ea6d2b09614959c0ec4fd564789
-release-management:prepare-provider-documentation:ef82f40591e467145085f5c955487ac4
-release-management:prepare-provider-packages:72dd7c3b19f85024bc9a4939cac5d87c
+release-management:prepare-provider-documentation:7bf85a5bc35d20d9396881e8dc442b8a
+release-management:prepare-provider-packages:39ac7c3ea29be199d47ece8e00fdd1c8
release-management:release-prod-images:dff5d068a253dcc3ac1e65a6af150e3d
release-management:start-rc-process:6aafbaceabd7b67b9a1af4c2f59abc4c
release-management:start-release:acb384d86e02ff5fde1bf971897be17c
release-management:verify-provider-packages:7da96dd515876f4a40fd63486b478d0c
-release-management:2b58afb740a9c7620373cfd71307c9d7
+release-management:65fd35264b7fe39e01af4d5cdb029026
setup:autocomplete:03343478bf1d0cf9c101d454cdb63b68
setup:check-all-params-in-groups:1a6a07dcef12a55bf42bcbcef0d86adc
setup:config:584023d1d225879a5d19e347fe9db1d3
diff --git a/images/breeze/output_build-docs.svg b/images/breeze/output_build-docs.svg
index 71c0c5b79e..380e6770f5 100644
--- a/images/breeze/output_build-docs.svg
+++ b/images/breeze/output_build-docs.svg
@@ -289,8 +289,8 @@
</text><text class="breeze-build-docs-r5" x="0" y="1215.6" textLength="12.2" clip-path="url(#breeze-build-docs-line-49)">│</text><text class="breeze-build-docs-r7" x="329.4" y="1215.6" textLength="1110.2" clip-path="url(#breeze-build-docs-line-49)">apache-airflow-providers-sqlite | apache-airflow-providers-ssh |                           </ [...]
</text><text class="breeze-build-docs-r5" x="0" y="1240" textLength="12.2" clip-path="url(#breeze-build-docs-line-50)">│</text><text class="breeze-build-docs-r7" x="329.4" y="1240" textLength="1110.2" clip-path="url(#breeze-build-docs-line-50)">apache-airflow-providers-tableau | apache-airflow-providers-tabular |                      </text><text class="breeze-buil [...]
</text><text class="breeze-build-docs-r5" x="0" y="1264.4" textLength="12.2" clip-path="url(#breeze-build-docs-line-51)">│</text><text class="breeze-build-docs-r7" x="329.4" y="1264.4" textLength="1110.2" clip-path="url(#breeze-build-docs-line-51)">apache-airflow-providers-telegram | apache-airflow-providers-trino |                       </text><text class="br [...]
-</text><text class="breeze-build-docs-r5" x="0" y="1288.8" textLength="12.2" clip-path="url(#breeze-build-docs-line-52)">│</text><text class="breeze-build-docs-r7" x="329.4" y="1288.8" textLength="1110.2" clip-path="url(#breeze-build-docs-line-52)">apache-airflow-providers-vertica | apache-airflow-providers-yandex |                       </text><text class="br [...]
-</text><text class="breeze-build-docs-r5" x="0" y="1313.2" textLength="12.2" clip-path="url(#breeze-build-docs-line-53)">│</text><text class="breeze-build-docs-r7" x="329.4" y="1313.2" textLength="1110.2" clip-path="url(#breeze-build-docs-line-53)">apache-airflow-providers-zendesk | docker-stack | helm-chart)                            [...]
+</text><text class="breeze-build-docs-r5" x="0" y="1288.8" textLength="12.2" clip-path="url(#breeze-build-docs-line-52)">│</text><text class="breeze-build-docs-r7" x="329.4" y="1288.8" textLength="1110.2" clip-path="url(#breeze-build-docs-line-52)">apache-airflow-providers-vertica | apache-airflow-providers-zendesk | docker-stack |       </text><text class="breeze-build-docs-r5" x="1451.8" y="1288.8" textLength="12.2" clip-path= [...]
+</text><text class="breeze-build-docs-r5" x="0" y="1313.2" textLength="12.2" clip-path="url(#breeze-build-docs-line-53)">│</text><text class="breeze-build-docs-r7" x="329.4" y="1313.2" textLength="1110.2" clip-path="url(#breeze-build-docs-line-53)">helm-chart)                                       [...]
</text><text class="breeze-build-docs-r5" x="0" y="1337.6" textLength="12.2" clip-path="url(#breeze-build-docs-line-54)">│</text><text class="breeze-build-docs-r4" x="24.4" y="1337.6" textLength="12.2" clip-path="url(#breeze-build-docs-line-54)">-</text><text class="breeze-build-docs-r4" x="36.6" y="1337.6" textLength="85.4" clip-path="url(#breeze-build-docs-line-54)">-github</text><text class="breeze-build-docs-r4" x="122" y="1337.6" textLength="134.2" clip-path="url(#breeze-build-docs- [...]
</text><text class="breeze-build-docs-r5" x="0" y="1362" textLength="1464" clip-path="url(#breeze-build-docs-line-55)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-build-docs-r2" x="1464" y="1362" textLength="12.2" clip-path="url(#breeze-build-docs-line-55)">
</text><text class="breeze-build-docs-r5" x="0" y="1386.4" textLength="24.4" clip-path="url(#breeze-build-docs-line-56)">╭─</text><text class="breeze-build-docs-r5" x="24.4" y="1386.4" textLength="195.2" clip-path="url(#breeze-build-docs-line-56)"> Common options </text><text class="breeze-build-docs-r5" x="219.6" y="1386.4" textLength="1220" clip-path="url(#breeze-build-docs-line-56)">──────────────────────────────────────────────────────────────────────────────────────── [...]
diff --git a/images/breeze/output_release-management_generate-issue-content-providers.svg b/images/breeze/output_release-management_generate-issue-content-providers.svg
index ace464046a..ca8c6bb826 100644
--- a/images/breeze/output_release-management_generate-issue-content-providers.svg
+++ b/images/breeze/output_release-management_generate-issue-content-providers.svg
@@ -177,7 +177,7 @@
</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="12.2" y="386" textLength="1366.4" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-15)">                                                 [...]
</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="12.2" y="410.4" textLength="1403" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-16)">                                                 [...]
</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="12.2" y="434.8" textLength="1427.4" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-17)">                                                [...]
-</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="12.2" y="459.2" textLength="1427.4" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-18)">                                                [...]
+</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="12.2" y="459.2" textLength="1317.6" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-18)">                                                [...]
</text><text class="breeze-release-management-generate-issue-content-providers-r2" x="1464" y="483.6" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-19)">
</text><text class="breeze-release-management-generate-issue-content-providers-r2" x="12.2" y="508" textLength="585.6" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-20)">Generates content for issue to test the release.</text><text class="breeze-release-management-generate-issue-content-providers-r2" x="1464" y="508" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-20)">
</text><text class="breeze-release-management-generate-issue-content-providers-r2" x="1464" y="532.4" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-21)">
diff --git a/images/breeze/output_release-management_prepare-provider-documentation.svg b/images/breeze/output_release-management_prepare-provider-documentation.svg
index 75b2769e59..9f91e0f28f 100644
--- a/images/breeze/output_release-management_prepare-provider-documentation.svg
+++ b/images/breeze/output_release-management_prepare-provider-documentation.svg
@@ -183,7 +183,7 @@
</text><text class="breeze-release-management-prepare-provider-documentation-r1" x="12.2" y="386" textLength="1439.6" clip-path="url(#breeze-release-management-prepare-provider-documentation-line-15)">                                                 [...]
</text><text class="breeze-release-management-prepare-provider-documentation-r1" x="12.2" y="410.4" textLength="1366.4" clip-path="url(#breeze-release-management-prepare-provider-documentation-line-16)">                                                & [...]
</text><text class="breeze-release-management-prepare-provider-documentation-r1" x="12.2" y="434.8" textLength="1342" clip-path="url(#breeze-release-management-prepare-provider-documentation-line-17)">                                                 [...]
-</text><text class="breeze-release-management-prepare-provider-documentation-r1" x="12.2" y="459.2" textLength="1378.6" clip-path="url(#breeze-release-management-prepare-provider-documentation-line-18)">                                                & [...]
+</text><text class="breeze-release-management-prepare-provider-documentation-r1" x="12.2" y="459.2" textLength="1268.8" clip-path="url(#breeze-release-management-prepare-provider-documentation-line-18)">                                                & [...]
</text><text class="breeze-release-management-prepare-provider-documentation-r2" x="1464" y="483.6" textLength="12.2" clip-path="url(#breeze-release-management-prepare-provider-documentation-line-19)">
</text><text class="breeze-release-management-prepare-provider-documentation-r2" x="12.2" y="508" textLength="97.6" clip-path="url(#breeze-release-management-prepare-provider-documentation-line-20)">Prepare </text><text class="breeze-release-management-prepare-provider-documentation-r4" x="109.8" y="508" textLength="109.8" clip-path="url(#breeze-release-management-prepare-provider-documentation-line-20)">CHANGELOG</text><text class="breeze-release-management-prepare-provider-documen [...]
</text><text class="breeze-release-management-prepare-provider-documentation-r2" x="1464" y="532.4" textLength="12.2" clip-path="url(#breeze-release-management-prepare-provider-documentation-line-21)">
diff --git a/images/breeze/output_release-management_prepare-provider-packages.svg b/images/breeze/output_release-management_prepare-provider-packages.svg
index d75838e262..e57bdcf16e 100644
--- a/images/breeze/output_release-management_prepare-provider-packages.svg
+++ b/images/breeze/output_release-management_prepare-provider-packages.svg
@@ -169,7 +169,7 @@
</text><text class="breeze-release-management-prepare-provider-packages-r1" x="12.2" y="337.2" textLength="1439.6" clip-path="url(#breeze-release-management-prepare-provider-packages-line-13)">                                                   [...]
</text><text class="breeze-release-management-prepare-provider-packages-r1" x="12.2" y="361.6" textLength="1427.4" clip-path="url(#breeze-release-management-prepare-provider-packages-line-14)">                                                   [...]
</text><text class="breeze-release-management-prepare-provider-packages-r1" x="12.2" y="386" textLength="1390.8" clip-path="url(#breeze-release-management-prepare-provider-packages-line-15)">                                                  & [...]
-</text><text class="breeze-release-management-prepare-provider-packages-r1" x="12.2" y="410.4" textLength="1415.2" clip-path="url(#breeze-release-management-prepare-provider-packages-line-16)">                                                   [...]
+</text><text class="breeze-release-management-prepare-provider-packages-r1" x="12.2" y="410.4" textLength="1305.4" clip-path="url(#breeze-release-management-prepare-provider-packages-line-16)">                                                   [...]
</text><text class="breeze-release-management-prepare-provider-packages-r1" x="12.2" y="434.8" textLength="854" clip-path="url(#breeze-release-management-prepare-provider-packages-line-17)">                                                  &# [...]
</text><text class="breeze-release-management-prepare-provider-packages-r2" x="1464" y="459.2" textLength="12.2" clip-path="url(#breeze-release-management-prepare-provider-packages-line-18)">
</text><text class="breeze-release-management-prepare-provider-packages-r2" x="12.2" y="483.6" textLength="585.6" clip-path="url(#breeze-release-management-prepare-provider-packages-line-19)">Prepare sdist/whl packages of Airflow Providers.</text><text class="breeze-release-management-prepare-provider-packages-r2" x="1464" y="483.6" textLength="12.2" clip-path="url(#breeze-release-management-prepare-provider-packages-line-19)">