You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2020/07/16 15:03:06 UTC
[spark] branch master updated (6be8b93 -> c1f160e)
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from 6be8b93 [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables
add c1f160e [SPARK-30648][SQL] Support filters pushdown in JSON datasource
No new revisions were added by this update.
Summary of changes:
.../apache/spark/sql/catalyst/StructFilters.scala | 166 +++++++++++++++++++++
.../apache/spark/sql/catalyst/csv/CSVFilters.scala | 124 +++------------
.../spark/sql/catalyst/json/JacksonParser.scala | 27 ++--
.../spark/sql/catalyst/json/JsonFilters.scala | 156 +++++++++++++++++++
.../org/apache/spark/sql/internal/SQLConf.scala | 8 +
...FiltersSuite.scala => StructFiltersSuite.scala} | 38 ++---
.../spark/sql/catalyst/csv/CSVFiltersSuite.scala | 116 +-------------
.../sql/catalyst/json/JacksonParserSuite.scala | 57 +++++++
.../sql/catalyst/json/JsonFiltersSuite.scala} | 14 +-
sql/core/benchmarks/CSVBenchmark-jdk11-results.txt | 64 ++++----
sql/core/benchmarks/CSVBenchmark-results.txt | 64 ++++----
.../benchmarks/JsonBenchmark-jdk11-results.txt | 94 ++++++------
sql/core/benchmarks/JsonBenchmark-results.txt | 94 ++++++------
.../datasources/json/JsonFileFormat.scala | 6 +-
.../datasources/v2/csv/CSVScanBuilder.scala | 4 +-
.../v2/json/JsonPartitionReaderFactory.scala | 11 +-
.../execution/datasources/v2/json/JsonScan.scala | 12 +-
.../datasources/v2/json/JsonScanBuilder.scala | 26 +++-
.../sql/execution/datasources/csv/CSVSuite.scala | 30 ++++
.../execution/datasources/json/JsonBenchmark.scala | 45 +++++-
.../sql/execution/datasources/json/JsonSuite.scala | 151 ++++++++++++++++++-
21 files changed, 893 insertions(+), 414 deletions(-)
create mode 100644 sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/StructFilters.scala
create mode 100644 sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonFilters.scala
copy sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/{csv/CSVFiltersSuite.scala => StructFiltersSuite.scala} (82%)
create mode 100644 sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonParserSuite.scala
copy sql/catalyst/src/{main/scala/org/apache/spark/sql/connector/write/LogicalWriteInfoImpl.scala => test/scala/org/apache/spark/sql/catalyst/json/JsonFiltersSuite.scala} (71%)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org