You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by su...@apache.org on 2022/01/05 17:55:15 UTC
[spark] branch master updated (4741ecf -> 32a28db)
This is an automated email from the ASF dual-hosted git repository.
sunchao pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from 4741ecf [SPARK-33898][SQL][FOLLOWUP] Unify the v2 behavior with v1 for `SHOW CREATE TABLE` command
add 32a28db [SPARK-36879][SQL] Support Parquet v2 data page encoding (DELTA_BINARY_PACKED) for the vectorized path
No new revisions were added by this update.
Summary of changes:
.../parquet/VectorizedColumnReader.java | 19 +-
.../parquet/VectorizedDeltaBinaryPackedReader.java | 319 +++++++++++++++++++
.../parquet/VectorizedDeltaByteArrayReader.java | 66 ++++
.../datasources/parquet/VectorizedReaderBase.java | 156 +++++++++
.../parquet/VectorizedValuesReader.java | 18 ++
.../benchmark/DataSourceReadBenchmark.scala | 12 +
.../parquet/ParquetDeltaEncodingSuite.scala | 353 +++++++++++++++++++++
.../datasources/parquet/ParquetEncodingSuite.scala | 57 ++++
.../parquet/ParquetRebaseDatetimeSuite.scala | 288 +++++++++--------
.../datasources/parquet/ParquetTest.scala | 12 +-
10 files changed, 1161 insertions(+), 139 deletions(-)
create mode 100644 sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java
create mode 100644 sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaByteArrayReader.java
create mode 100644 sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedReaderBase.java
create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org