[GitHub] [spark] allisonwang-db commented on a diff in pull request #36387: [SPARK-38918][SQL][3.1] Nested column pruning should filter out attributes that do not belong to the current relation

[GitBox <gi...@apache.org>]

allisonwang-db commented on code in PR #36387:
URL: https://github.com/apache/spark/pull/36387#discussion_r861163862


##########
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/SchemaPruningSuite.scala:
##########
@@ -597,8 +597,7 @@ abstract class SchemaPruningSuite
             |  and e.employer.name = c.employer.company.name)
             |""".stripMargin)
         checkScan(query,
-          "struct<name:struct<first:string,middle:string,last:string>," +
-            "employer:struct<id:int,company:struct<name:string,address:string>>>",
+          "struct<name:struct<first:string>,employer:struct<company:struct<name:string>>>",

Review Comment:
   Yes looks like starting from 3.2 it failed to prune some nested fields (for example in this case `name.middle` and `name.last` are not used)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org