[PR] Add alloyDB handler for google cleanup script [airflow]

["VladaZakharova (via GitHub)" <gi...@apache.org>]

VladaZakharova opened a new pull request, #74236:
URL: https://github.com/apache/airflow/pull/74236

    <!-- SPDX-License-Identifier: Apache-2.0
         https://www.apache.org/licenses/LICENSE-2.0 -->
   
   <!--
   Thank you for contributing!
   
   Please provide above a brief description of the changes made in this pull request.
   Write a good git commit message following this guide: https://chris.beams.io/posts/git-commit/
   
   Please make sure that your code changes are covered with tests.
   And in case of new features or big changes remember to adjust the documentation.
   
   For user-facing UI changes, please attach before/after screenshots (or a short
   screen recording) so reviewers can assess the visual impact.
   
   Feel free to ping (in general) for the review if you do not see reaction for a few days
   (72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
   
   In case of an existing issue, reference it using one of the following:
   
   * closes: #ISSUE
   * related: #ISSUE
   -->
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   <!--
   If generative AI tooling has been used in the process of authoring this PR, please
   change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
   -->
   
   - [ ] Yes (please specify the tool below)
   
   <!--
   Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   -->
   
   ---
   
   * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
   * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add alloyDB handler for google cleanup script [airflow]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk commented on code in PR #74236:
URL: https://github.com/apache/airflow/pull/74236#discussion_r4185757745


##########
providers/google/pyproject.toml:
##########
@@ -82,14 +82,14 @@ dependencies = [
     "google-auth-httplib2>=0.0.1",
     "google-genai>=2.8.0",
     # google-cloud-aiplatform doesn't install ray for python 3.12 (issue: https://github.com/googleapis/python-aiplatform/issues/5252).
-    # Temporarily lock in ray 2.42.0 which is compatible with python 3.12 until linked issue is solved.
+    # Temporarily require ray directly until the linked issue is solved.
     # Remove the ray dependency as well as google-cloud-bigquery-storage once linked issue is fixed
     # Floor raised to 1.164.0: earlier "evaluation" extras cap litellm below the version that
     # carries the fixes for CVE-2026-35030 and its follow-on advisories on Python 3.14.
     # - https://github.com/googleapis/python-aiplatform/issues/7057
     "google-cloud-aiplatform[evaluation]>=1.164.0",
-    "ray[default]>=2.42.0;python_version<'3.13'",
-    "ray[default]>=2.49.0;python_version>='3.13' and python_version <'3.14'",
+    "ray[default]>=2.52.0;python_version<'3.13'",

Review Comment:
   This raises the `ray[default]` minimum of the **released** Google provider: from `>=2.42.0` to `>=2.52.0` on Python < 3.13, and from `>=2.49.0` to `>=2.52.0` on 3.13. The cleanup handler doesn't need it, and there's no changelog note, so users pinned to ray 2.42–2.51 would hit resolution failures or forced upgrades on the next provider release. It also leaves two identical lines that could be merged. Could you drop it from this PR, or move it to its own PR with the reason and a changelog entry?



##########
providers/google/tests/system/google/resources_cleanup/airflow_google_provider_resource_cleanup/handlers/alloydb.py:
##########
@@ -0,0 +1,55 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+from airflow_google_provider_resource_cleanup.handlers._base import BaseDeleteHandler
+from airflow_google_provider_resource_cleanup.helpers import get_resource_path
+from google.cloud import alloydb_v1
+
+
+async def _delete_resource(resource: dict, request_type, delete_method_name: str, **request_kwargs):
+    client = alloydb_v1.AlloyDBAdminAsyncClient()
+    request = request_type(name=get_resource_path(resource), **request_kwargs)
+    operation = await getattr(client, delete_method_name)(request=request)
+    await operation.result()
+
+
+async def _delete_backup(resource: dict, log_prefix: str):
+    await _delete_resource(resource, alloydb_v1.DeleteBackupRequest, "delete_backup")
+
+
+async def _delete_instance(resource: dict, log_prefix: str):
+    await _delete_resource(resource, alloydb_v1.DeleteInstanceRequest, "delete_instance")
+
+
+async def _delete_cluster(resource: dict, log_prefix: str):
+    await _delete_resource(resource, alloydb_v1.DeleteClusterRequest, "delete_cluster", force=True)

Review Comment:
   All clusters are deleted in parallel here (one `asyncio.gather` per asset type in `BaseDeleteHandler.handle`), but AlloyDB refuses to delete a primary cluster while it still has secondary clusters. `force=True` only cascades to instances, not to secondaries. `example_alloy_db.py` creates a secondary cluster (`SECONDARY_CLUSTER_ID`), so if that test fails partway, cleanup deletes the secondary and fails on the primary with `FAILED_PRECONDITION`. The error is only printed, so the primary and its instances keep running and billing until someone reruns the tool. Could you delete secondary clusters before primaries, e.g. two passes split on `cluster_type` / `secondary_config`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org