You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@spark.apache.org by do...@apache.org on 2020/08/20 09:33:56 UTC

[spark] branch master updated (f793977 -> e6795cd)

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.


    from f793977  [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow
     add e6795cd  [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

No new revisions were added by this update.

Summary of changes:
 .../streaming/CompactibleFileStreamLog.scala       | 97 +++++++++++++++-------
 .../execution/streaming/FileStreamSinkLog.scala    | 10 +--
 .../execution/streaming/FileStreamSourceLog.scala  |  6 +-
 .../sql/execution/streaming/HDFSMetadataLog.scala  | 80 ++++++++++++------
 .../file-sink-log-version-2.1.0/8                  |  1 -
 .../streaming/CompactibleFileStreamLogSuite.scala  |  2 -
 .../streaming/FileStreamSinkLogSuite.scala         | 20 ++---
 7 files changed, 129 insertions(+), 87 deletions(-)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org