You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@spark.apache.org by gu...@apache.org on 2020/12/04 10:37:33 UTC

[spark] branch master updated (976e897 -> 233a849)

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.


    from 976e897  [SPARK-33640][TESTS] Extend connection timeout to DB server for DB2IntegrationSuite and its variants
     add 233a849  [SPARK-27237][SS] Introduce State schema validation among query restart

No new revisions were added by this update.

Summary of changes:
 .../org/apache/spark/sql/internal/SQLConf.scala    |  10 +
 .../org/apache/spark/sql/types/DataType.scala      |  38 +++-
 .../sql/execution/streaming/HDFSMetadataLog.scala  |  32 +--
 .../execution/streaming/MetadataVersionUtil.scala  |  51 +++++
 .../state/StateSchemaCompatibilityChecker.scala    | 118 +++++++++++
 .../sql/execution/streaming/state/StateStore.scala |  36 +++-
 .../execution/streaming/state/StateStoreConf.scala |   3 +
 .../StateSchemaCompatibilityCheckerSuite.scala     | 230 +++++++++++++++++++++
 .../sql/streaming/StreamingAggregationSuite.scala  |  87 +++++++-
 ...reamingStateStoreFormatCompatibilitySuite.scala |  21 +-
 10 files changed, 582 insertions(+), 44 deletions(-)
 create mode 100644 sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MetadataVersionUtil.scala
 create mode 100644 sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateSchemaCompatibilityChecker.scala
 create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateSchemaCompatibilityCheckerSuite.scala


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org