(spark) branch branch-4.3 updated: [SPARK-58604][SQL] Preserve viewDependencies when rebuilding a View for ALTER VIEW

[we...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

cloud-fan pushed a commit to branch branch-4.3
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.3 by this push:
     new fdd345ca7e41 [SPARK-58604][SQL] Preserve viewDependencies when rebuilding a View for ALTER VIEW
fdd345ca7e41 is described below

commit fdd345ca7e41a0b1d8050bf0463cbdcba4c88f73
Author: Szehon Ho <sz...@gmail.com>
AuthorDate: Thu Aug 6 16:05:12 2026 +0800

    [SPARK-58604][SQL] Preserve viewDependencies when rebuilding a View for ALTER VIEW
    
    ### What changes were proposed in this pull request?
    
    `CatalogV2Util.viewInfoBuilderFrom` seeds a `View.Builder` from an existing view so ALTER VIEW execs can override the one field that changes and leave everything else untouched. It copies the schema, properties, query text, SQL configs, current namespace, current catalog, query column names and schema mode — but not the typed `viewDependencies` field.
    
    This PR carries `viewDependencies` through, consistent with how the other nullable fields (`currentCatalog`, `schemaMode`) are already handled.
    
    ### Why are the changes needed?
    
    All three callers of `viewInfoBuilderFrom` are metadata-only mutations that do not change the view body:
    
    - `AlterV2ViewSetPropertiesExec` (`ALTER VIEW ... SET TBLPROPERTIES`)
    - `AlterV2ViewUnsetPropertiesExec` (`ALTER VIEW ... UNSET TBLPROPERTIES`)
    - `AlterV2ViewSchemaBindingExec` (`ALTER VIEW ... WITH SCHEMA ...`)
    
    Each rebuilds the payload and calls `ViewCatalog.replaceView`, so after any of them the catalog receives a `View` whose `viewDependencies()` is `null` and the previously recorded dependency list is silently lost. Dependency lists are a first-class field on `View` rather than an encoded string property precisely because their nested structure does not round-trip through flat properties, so a catalog has no other way to recover them.
    
    This is reachable today through metric views, which are the only producer of dependencies (`CreateV2MetricViewExec`). Creating a metric view records its source tables, and a subsequent property change drops them:
    
    ```sql
    CREATE VIEW mv WITH METRICS LANGUAGE YAML AS $$ ... $$;  -- dependencies recorded
    ALTER VIEW mv SET TBLPROPERTIES ('k' = 'v');             -- dependencies now null
    ```
    
    ### Does this PR introduce _any_ user-facing change?
    
    No, in the sense that no released version is affected — the `View` API and these ALTER VIEW execs are new in the unreleased line, so this is a fix within master rather than a change in behavior users have depended on. Catalogs that persist view lineage will now keep it across a metadata-only ALTER VIEW instead of seeing it cleared.
    
    ### How was this patch tested?
    
    Two new tests, both confirmed to fail before the fix and pass after:
    
    - `CatalogV2UtilSuite`: `viewInfoBuilderFrom` preserves a dependency list, and leaves an absent one absent (guards the null path). The first fails without the fix.
    - `MetricViewV2CatalogSuite`: end-to-end `ALTER VIEW <metric_view> SET TBLPROPERTIES` against the recording `RelationCatalog`, asserting the replacement payload still carries the source-table dependency. Without the fix this fails with `viewDependencies()` being `null`.
    
    Full suites pass locally: `CatalogV2UtilSuite` (10 tests) and `MetricViewV2CatalogSuite` (32 tests). `./dev/lint-scala` passes.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Cursor (Opus 5)
    
    Closes #57802 from szehon-ho/spark-view-deps-alter.
    
    Authored-by: Szehon Ho <sz...@gmail.com>
    Signed-off-by: Wenchen Fan <we...@databricks.com>
    (cherry picked from commit 4a471e41eb3608692ac35bd291fcc5f4797dbd96)
    Signed-off-by: Wenchen Fan <we...@databricks.com>
---
 .../sql/connector/catalog/CatalogV2Util.scala      |  1 +
 .../sql/connector/catalog/CatalogV2UtilSuite.scala | 23 +++++++++++++++++++++-
 .../sql/execution/MetricViewV2CatalogSuite.scala   | 22 +++++++++++++++++++++
 3 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala
index 28aaae4a81ff..a29fe880b915 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala
@@ -577,6 +577,7 @@ private[sql] object CatalogV2Util {
       .withQueryColumnNames(existing.queryColumnNames)
     Option(existing.currentCatalog).foreach(builder.withCurrentCatalog)
     Option(existing.schemaMode).foreach(builder.withSchemaMode)
+    Option(existing.viewDependencies).foreach(builder.withViewDependencies)
     builder
   }
 
diff --git a/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogV2UtilSuite.scala b/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogV2UtilSuite.scala
index 4b9d55be07e3..f124186903e0 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogV2UtilSuite.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogV2UtilSuite.scala
@@ -23,7 +23,7 @@ import org.mockito.Mockito.{mock, verify, when}
 import org.apache.spark.{SparkFunSuite, SparkIllegalArgumentException}
 import org.apache.spark.sql.catalyst.analysis.{AsOfTimestamp, AsOfVersion, TimeTravelSpec}
 import org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation
-import org.apache.spark.sql.types.IntegerType
+import org.apache.spark.sql.types.{IntegerType, StructType}
 import org.apache.spark.sql.util.CaseInsensitiveStringMap
 
 class CatalogV2UtilSuite extends SparkFunSuite {
@@ -113,4 +113,25 @@ class CatalogV2UtilSuite extends SparkFunSuite {
     assert(a.toString.contains("timeTravel"))
     assert(a.toString.contains("writePrivileges"))
   }
+
+  test("viewInfoBuilderFrom preserves the dependency list") {
+    val dependencies = DependencyList.of(Array(Dependency.table(Array("cat", "ns", "events"))))
+    val existing = viewWithDependencies(Some(dependencies))
+    val rebuilt = CatalogV2Util.viewInfoBuilderFrom(existing).build()
+    assert(rebuilt.viewDependencies() === dependencies)
+  }
+
+  test("viewInfoBuilderFrom leaves an absent dependency list absent") {
+    val existing = viewWithDependencies(None)
+    val rebuilt = CatalogV2Util.viewInfoBuilderFrom(existing).build()
+    assert(rebuilt.viewDependencies() === null)
+  }
+
+  private def viewWithDependencies(dependencies: Option[DependencyList]): View = {
+    val builder = new View.Builder()
+      .withSchema(new StructType().add("i", IntegerType))
+      .withQueryText("SELECT i FROM cat.ns.events")
+    dependencies.foreach(builder.withViewDependencies)
+    builder.build()
+  }
 }
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/MetricViewV2CatalogSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/MetricViewV2CatalogSuite.scala
index 9b75e89a1ad8..5036ab94c133 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/MetricViewV2CatalogSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/MetricViewV2CatalogSuite.scala
@@ -930,6 +930,28 @@ class MetricViewV2CatalogSuite extends SharedSparkSession {
     }
   }
 
+  test("ALTER VIEW <metric_view> SET TBLPROPERTIES preserves the dependency list") {
+    withTestCatalogTables {
+      val mv = MetricView(
+        "0.1",
+        AssetSource(fullSourceTableName),
+        where = None,
+        select = metricViewColumns)
+      createMetricView(fullMetricViewName, mv)
+
+      sql(s"ALTER VIEW $fullMetricViewName SET TBLPROPERTIES ('k' = 'v')")
+
+      val info = capturedViewInfo()
+      assert(info.properties().get("k") === "v")
+      val deps = info.viewDependencies()
+      assert(deps != null)
+      assert(deps.dependencies().length === 1)
+      val tableDep = deps.dependencies()(0).asInstanceOf[TableDependency]
+      assert(tableDep.nameParts().toSeq ===
+        Seq(testCatalogName, testNamespace, sourceTableName))
+    }
+  }
+
   test("SHOW TABLES on a v2 RelationCatalog lists both tables and metric views") {
     withTestCatalogTables {
       val mv = MetricView(


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org