You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Prabhat Mishra (Jira)" <ji...@apache.org> on 2026/09/28 16:48:00 UTC
[jira] [Created] (IMPALA-15447) Iceberg scan telemetry (Stream A + B) to report Iceberg scan metrics.
Prabhat Mishra created IMPALA-15447:
---------------------------------------
Summary: Iceberg scan telemetry (Stream A + B) to report Iceberg scan metrics.
Key: IMPALA-15447
URL: https://issues.apache.org/jira/browse/IMPALA-15447
Project: IMPALA
Issue Type: Improvement
Components: Catalog, Frontend
Reporter: Prabhat Mishra
Description:
Adds two-stream Iceberg query telemetry to the Impala frontend (Java/fe):
Stream A — Scan Reports (IcebergScanPlanner.java, IcebergUtil.java):
- Wires a catalog-configured side MetricsReporter alongside the existing InMemoryMetricsReporter in IcebergScanPlanner. The existing InMemoryMetricsReporter is kept because it feeds addIcebergScanMetricsToProfile() (query profile).
- The side reporter is loaded once per JVM (double-checked locking, volatile) via CatalogUtil.loadMetricsReporter, driven by the metrics-reporter-impl JVM system property set through Cloudera Manager.
- For the fast-path where planFiles() is skipped, synthesizes a faithful ScanReport from already-computed file descriptors and posts it only to the side reporter (InMemory reporter and query profile are left unchanged).
- Stamps a query-id and engine-name onto every scan report via a new IcebergUtil.planFiles overload that passes scan options as report metadata, enabling cross-stream correlation downstream.
Stream B — Query Shape (IcebergQueryShapeCollector.java):
- New class IcebergQueryShapeCollector runs once per query, right after Planner.createPlans.
- Walks the plan tree, extracts equi-join keys and ORDER BY columns that resolve to Iceberg tables.
- POSTs a per-table query-shape event via the same telemetry library (called reflectively to avoid a compile-time dependency), tagged with the identical query-id used in Stream A.
- Hooked into Frontend.createExecRequest (Frontend.java).
Both streams are entirely best-effort: neither throws nor blocks query planning, and both no-op when no side reporter is configured (preserving today's default behavior byte-for-byte).
---
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org