[GitHub] dbtsai commented on issue #23474: [SPARK-26551][SQL] Fix schema pruning error when selecting one complex field and having is not null predicate on another one

[GitBox <gi...@apache.org>]

dbtsai commented on issue #23474: [SPARK-26551][SQL] Fix schema pruning error when selecting one complex field and having is not null predicate on another one
URL: https://github.com/apache/spark/pull/23474#issuecomment-452064572
 
 
   When we read `name.first` and check if `name` is not `null`, we are marking `name` as `contentAccessed = false` to avoid reading the entire `name` column.
   
   Now, the issue is when we read `name.first` and check if `name.middle` is not `null`, we are still marking `name.middle` as `contentAccessed = false` resulting `java.lang.IllegalArgumentException: middle does not exist.`.
   
   To fix the root cause, and avoid the misunderstanding of the meaning of `contentAccessed`, we might mark `contentAccessed = true` in the second case; thus, this change here will not be required. 
   
   What do you think?
   
   Thanks.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org