You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@spark.apache.org by we...@apache.org on 2022/01/19 06:14:02 UTC

[spark] branch master updated (817d1d7 -> 3a45981)

This is an automated email from the ASF dual-hosted git repository.

wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.


    from 817d1d7  [SPARK-37769][SQL][FOLLOWUP] Filtering files if metadata columns are present in the data filter
     add 3a45981  [SPARK-37896][SQL] Implement a ConstantColumnVector and improve performance of the hidden file metadata

No new revisions were added by this update.

Summary of changes:
 .../execution/vectorized/ConstantColumnVector.java | 292 +++++++++++++++++++++
 .../spark/sql/execution/DataSourceScanExec.scala   |   6 +-
 .../sql/execution/datasources/FileScanRDD.scala    |  45 ++--
 .../vectorized/ConstantColumnVectorSuite.scala     | 205 +++++++++++++++
 4 files changed, 515 insertions(+), 33 deletions(-)
 create mode 100644 sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ConstantColumnVector.java
 create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ConstantColumnVectorSuite.scala

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org