[GitHub] [hudi] yihua commented on a diff in pull request #5379: [HUDI-3936] Fix projection for a nested field as pre-combined key

[GitBox <gi...@apache.org>]

yihua commented on code in PR #5379:
URL: https://github.com/apache/hudi/pull/5379#discussion_r854863511


##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala:
##########
@@ -336,7 +335,11 @@ abstract class HoodieBaseRelation(val sqlContext: SQLContext,
   }
 
   protected final def appendMandatoryColumns(requestedColumns: Array[String]): Array[String] = {
-    val missing = mandatoryColumns.filter(col => !requestedColumns.contains(col))
+    // For a nested field in mandatory columns, we should first get the root-level field, and then
+    // check for any missing column, as the requestedColumns should only contain root-level fields
+    // We should only append root-level field as well
+    val missing = mandatoryColumns.map(col => HoodieAvroUtils.getRootLevelFieldName(col))

Review Comment:
   If I understand it correctly, do you mean to say we can do the filtering of `mandatoryColumns` upon initialization of the class?  That's not possible since we need to do on-the-fly filtering based on the passed-in `requestedColumns` which may vary when `buildScan` is called.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org