[GitHub] [spark] maropu commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

[GitBox <gi...@apache.org>]

maropu commented on a change in pull request #31368:
URL: https://github.com/apache/spark/pull/31368#discussion_r565735036



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
##########
@@ -845,17 +845,33 @@ class SessionCatalog(
   }
 
   private def fromCatalogTable(metadata: CatalogTable, isTempView: Boolean): View = {
-    val viewText = metadata.viewText.getOrElse(sys.error("Invalid view without text."))
+    val viewText = metadata.viewText.getOrElse {
+      throw new IllegalStateException("Invalid view without text.")
+    }
     val viewConfigs = metadata.viewSQLConfigs
-    val viewPlan =
+    val parsedPlan =
       SQLConf.withExistingConf(View.effectiveSQLConf(viewConfigs, isTempView = isTempView)) {
         parser.parsePlan(viewText)
       }
-    View(
-      desc = metadata,
-      isTempView = isTempView,
-      output = metadata.schema.toAttributes,
-      child = viewPlan)
+    val viewColumnNames = metadata.viewQueryColumnNames
+    val viewPlan = if (viewColumnNames.nonEmpty) {
+      assert(viewColumnNames.length == metadata.schema.length)
+      // For view queries like `SELECT * FROM t`, the schema of the referenced table/view may
+      // change after the view has been created. We need to add an extra SELECT to pick the columns

Review comment:
       > For view queries like `SELECT * FROM t`, the schema of the referenced table/view may  change after the view has been created.
   
   We already have some tests for the case above somewhere?

##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala
##########
@@ -17,74 +17,21 @@
 
 package org.apache.spark.sql.catalyst.analysis
 
-import org.apache.spark.sql.catalyst.expressions.Alias
-import org.apache.spark.sql.catalyst.plans.logical.{LogicalPlan, Project, View}
+import org.apache.spark.sql.catalyst.plans.logical.{LogicalPlan, View}
 import org.apache.spark.sql.catalyst.rules.Rule
 
 /**
  * This file defines view types and analysis rules related to views.
  */
 
 /**
- * This rule has two goals:
- *
- * 1. Removes [[View]] operators from the plan. The operator is respected till the end of analysis
- * stage because we want to see which part of an analyzed logical plan is generated from a view.
- *
- * 2. Make sure that a view's child plan produces the view's output attributes. We try to wrap the
- * child by:
- * 1. Generate the `queryOutput` by:
- *    1.1. If the query column names are defined, map the column names to attributes in the child
- *         output by name(This is mostly for handling view queries like SELECT * FROM ..., the
- *         schema of the referenced table/view may change after the view has been created, so we
- *         have to save the output of the query to `viewQueryColumnNames`, and restore them during
- *         view resolution, in this way, we are able to get the correct view column ordering and
- *         omit the extra columns that we don't require);
- *    1.2. Else set the child output attributes to `queryOutput`.
- * 2. Map the `queryOutput` to view output by index, if the corresponding attributes don't match,
- *    try to up cast and alias the attribute in `queryOutput` to the attribute in the view output.
- * 3. Add a Project over the child, with the new output generated by the previous steps.
- *
- * Once reaches this rule, it means `CheckAnalysis` did necessary checks on number of columns
- * between the view output and the child output or the query column names. `CheckAnalysis` also
- * checked the cast from the view's child to the Project is up-cast.
- *
- * This should be only done after the batch of Resolution, because the view attributes are not
- * completely resolved during the batch of Resolution.
+ * This rule removes [[View]] operators from the plan. The operator is respected till the end of
+ * analysis stage because we want to see which part of an analyzed logical plan is generated from a
+ * view.
  */
 object EliminateView extends Rule[LogicalPlan] with CastSupport {
   override def apply(plan: LogicalPlan): LogicalPlan = plan transformUp {

Review comment:
       `transformUp` -> `transform`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org