[GitHub] gengliangwang opened a new pull request #23387: [SPARK-26447][SQL]Allow OrcColumnarBatchReader to return less partition columns

[GitBox <gi...@apache.org>]

gengliangwang opened a new pull request #23387: [SPARK-26447][SQL]Allow OrcColumnarBatchReader to return less partition columns
URL: https://github.com/apache/spark/pull/23387
 
 
   ## What changes were proposed in this pull request?
   
   Currently OrcColumnarBatchReader returns all the partition column values in the batch read.
   In data source V2, we can improve it by returning the required partition column values only.
   
   This PR is part of https://github.com/apache/spark/pull/23383 . As @cloud-fan suggested, create a new PR to make review easier.
   
   Also, this PR doesn't improve `OrcFileFormat`, since in the method `buildReaderWithPartitionValues`, the `requiredSchema` filter out all the partition columns, so we can't know which partition column is required.
   
   ## How was this patch tested?
   
   Unit test

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org