You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "rawwar (via GitHub)" <gi...@apache.org> on 2025/05/19 06:27:22 UTC
[PR] Add endpoint for DB stats [airflow]
rawwar opened a new pull request, #50758:
URL: https://github.com/apache/airflow/pull/50758
I am adding an endpoint to retrieve DB stats. For now, I am including only the table's name, the oldest record present in the table, and the number of records in the table. This can be extended in the future.
TODO:
- [ ] additional permission for DB endpoints - GET, POST and DELETE - Only available for admins by default
- [ ] Add additional field to TableStats for size of the table in MB - Very useful to determine if a clean up is necessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add endpoint for DB stats [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on code in PR #50758:
URL: https://github.com/apache/airflow/pull/50758#discussion_r2097582278
##########
airflow-core/src/airflow/utils/db_cleanup.py:
##########
@@ -523,3 +523,12 @@ def drop_archived_tables(
session.execute(text(f"DROP TABLE {table_name}"))
dropped_count += 1
logger.info("Total dropped tables: %s", dropped_count)
+
+
+def calculate_table_size(
+ *,
+ table_name: str,
+ session: Session,
+) -> float | None:
+ """Calculate table size in megabytes."""
+ return 0
Review Comment:
Yeah. Wanted to keep few issues as good first issues for MLH internship folks to pick them up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add endpoint for DB stats [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50758:
URL: https://github.com/apache/airflow/pull/50758#issuecomment-3037464316
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add endpoint for DB stats [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #50758:
URL: https://github.com/apache/airflow/pull/50758#discussion_r2097404886
##########
airflow-core/src/airflow/utils/db_cleanup.py:
##########
@@ -523,3 +523,12 @@ def drop_archived_tables(
session.execute(text(f"DROP TABLE {table_name}"))
dropped_count += 1
logger.info("Total dropped tables: %s", dropped_count)
+
+
+def calculate_table_size(
+ *,
+ table_name: str,
+ session: Session,
+) -> float | None:
+ """Calculate table size in megabytes."""
+ return 0
Review Comment:
Looks like the implementation is not done yet.
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/db.py:
##########
@@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+from fastapi import Depends
+from sqlalchemy import func
+
+from airflow.api_fastapi.common.db.common import SessionDep
+from airflow.api_fastapi.common.parameters import QuerySkipRecordCount, QuerySkipTableSize, QueryTablesFilter
+from airflow.api_fastapi.common.router import AirflowRouter
+from airflow.api_fastapi.core_api.datamodels.db import MetadataDBStatsResponse, TableStats
+from airflow.api_fastapi.core_api.security import requires_access_db
+from airflow.utils.db import reflect_tables
+from airflow.utils.db_cleanup import _effective_table_names, calculate_table_size
+
+db_router = AirflowRouter(tags=["Maintenance"], prefix="/maintenance/db")
Review Comment:
Not sure about the `maintenance` prefix/tag. Maybe `db` is good enough ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add endpoint for DB stats [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #50758: Add endpoint for DB stats
URL: https://github.com/apache/airflow/pull/50758
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org