You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2020/05/14 03:33:43 UTC
[spark] branch master updated (42951e6 -> fd2d55c)
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from 42951e6 [SPARK-31632][CORE][WEBUI] Enrich the exception message when application information is unavailable
add fd2d55c [SPARK-31405][SQL] Fail by default when reading/writing legacy datetime values from/to Parquet/Avro files
No new revisions were added by this update.
Summary of changes:
.../scala/org/apache/spark/SparkException.scala | 2 +-
.../apache/spark/sql/avro/AvroDeserializer.scala | 37 ++++----
.../org/apache/spark/sql/avro/AvroFileFormat.scala | 10 +-
.../apache/spark/sql/avro/AvroOutputWriter.scala | 13 ++-
.../org/apache/spark/sql/avro/AvroSerializer.scala | 30 +++---
.../sql/v2/avro/AvroPartitionReaderFactory.scala | 9 +-
.../sql/avro/AvroCatalystDataConversionSuite.scala | 2 +-
.../org/apache/spark/sql/avro/AvroSuite.scala | 91 ++++++++++++------
.../spark/sql/catalyst/json/JacksonParser.scala | 1 +
.../spark/sql/catalyst/util/RebaseDateTime.scala | 4 +-
.../org/apache/spark/sql/internal/SQLConf.scala | 100 ++++++++++----------
.../parquet/VectorizedColumnReader.java | 91 ++++++++++++------
.../parquet/VectorizedParquetRecordReader.java | 12 +--
.../parquet/VectorizedPlainValuesReader.java | 29 ++++--
.../parquet/VectorizedRleValuesReader.java | 31 ++++---
.../parquet/VectorizedValuesReader.java | 4 +-
.../execution/datasources/DataSourceUtils.scala | 103 ++++++++++++++++++++-
.../sql/execution/datasources/FileScanRDD.scala | 6 +-
.../datasources/parquet/ParquetFileFormat.scala | 11 +--
.../datasources/parquet/ParquetReadSupport.scala | 7 +-
.../parquet/ParquetRecordMaterializer.scala | 9 +-
.../datasources/parquet/ParquetRowConverter.scala | 68 ++++++--------
.../datasources/parquet/ParquetWriteSupport.scala | 46 ++++-----
.../v2/parquet/ParquetPartitionReaderFactory.scala | 23 ++---
.../benchmark/DateTimeRebaseBenchmark.scala | 4 +-
.../datasources/parquet/ParquetIOSuite.scala | 89 +++++++++++-------
.../spark/sql/sources/HadoopFsRelationTest.scala | 6 +-
27 files changed, 519 insertions(+), 319 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org