You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Lisa Cao (Jira)" <ji...@apache.org> on 2026/10/13 00:11:00 UTC
[jira] [Commented] (SPARK-60206) Metric views resolve unqualified names in the reader's current namespace instead of the creation namespace
[ https://issues.apache.org/jira/browse/SPARK-60206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18126533#comment-18126533 ]
Lisa Cao commented on SPARK-60206:
----------------------------------
[~linhongliu-db] https://github.com/apache/spark/pull/59416 fixes the source: tables, SQL sources and filter subqueries now resolve in the creation namespace, and temp views aren't picked up.
Functions in dimension and measure expressions still resolve in the querying session, because those expressions are spliced into the query plan. For example, a dimension {{f(k)}} calls {{other.f}} after {{USE other}}. Resolving them inside the metric view first didn't work for SQL functions, and qualifying function names would duplicate function resolution. How would you like to handle it? Happy to do the follow-up.
> Metric views resolve unqualified names in the reader's current namespace instead of the creation namespace
> ----------------------------------------------------------------------------------------------------------
>
> Key: SPARK-60206
> URL: https://issues.apache.org/jira/browse/SPARK-60206
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: Lisa Cao
> Priority: Major
> Labels: pull-request-available
>
> A metric view resolves unqualified names against the current catalog and namespace of whoever queries it, not the ones it was created in. Regular views use the creation namespace.
> {code:sql}
> CREATE TABLE default.t USING parquet AS SELECT * FROM VALUES ('a', 1), ('b', 2) AS v(k, x);
> CREATE DATABASE other;
> CREATE TABLE other.t USING parquet AS SELECT * FROM VALUES ('z', 1000) AS v(k, x);
> USE default;
> CREATE VIEW default.mv WITH METRICS LANGUAGE YAML AS $$
> version: "0.1"
> source: t
> dimensions:
> - name: k
> expr: k
> measures:
> - name: total
> expr: sum(x)
> $$;
> CREATE VIEW default.v AS SELECT sum(x) AS total FROM t;
> USE other;
> SELECT measure(total) FROM default.mv; -- 1000, reads other.t
> SELECT total FROM default.v; -- 3
> {code}
> The same happens for:
> * SQL sources ({{source: SELECT * FROM t}})
> * functions in dimension expressions ({{expr: f(k)}} calls {{other.f}})
> * subqueries in {{filter}}
> * temp views: with {{CREATE TEMPORARY VIEW t}} (or {{df.createOrReplaceTempView("t")}}) in the reader's session, the metric view reads the temp view, even from {{default}}
> * v2 catalog metric views, including after {{USE}} of another catalog
> From a namespace without {{t}} it fails with {{TABLE_OR_VIEW_NOT_FOUND}}. Reproduced on 4.2.0 and 4.3.0-rc1, with in-memory and Hive catalogs, and over Spark Connect.
> The creation namespace is stored ({{view.catalogAndNamespace.*}}), but {{SessionCatalog.parseMetricViewDefinition}} only applies the view's SQL configs. Regular views resolve under {{AnalysisContext.withAnalysisContext(view.desc)}} ({{ViewResolution}}); metric views skip it.
> Workaround: fully qualify names in the definition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org