You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@spark.apache.org by we...@apache.org on 2021/02/08 13:29:11 UTC

[spark] branch master updated (70ef196 -> a854906)

This is an automated email from the ASF dual-hosted git repository.

wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.


    from 70ef196  [SPARK-34157][BUILD][FOLLOW-UP] Fix Scala 2.13 compilation error via using Array.deep
     add a854906  [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read

No new revisions were added by this update.

Summary of changes:
 python/pyspark/sql/readwriter.py                   | 29 ++++++++++++-
 python/pyspark/sql/streaming.py                    | 30 ++++++++++++-
 .../org/apache/spark/sql/DataFrameReader.scala     | 23 ++++++++++
 .../datasources/parquet/ParquetFileFormat.scala    |  7 +++-
 .../datasources/parquet/ParquetOptions.scala       | 25 +++++++++++
 .../v2/parquet/ParquetPartitionReaderFactory.scala | 10 +++--
 .../datasources/v2/parquet/ParquetScan.scala       | 15 +++++--
 .../spark/sql/streaming/DataStreamReader.scala     | 23 ++++++++++
 .../parquet/ParquetRebaseDatetimeSuite.scala       | 49 +++++++++++++++-------
 9 files changed, 185 insertions(+), 26 deletions(-)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org