[spark] branch branch-3.0 updated: [SPARK-32836][SS][TESTS] Fix DataStreamReaderWriterSuite to check writer options correctly

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.0 by this push:
     new 5a81f60  [SPARK-32836][SS][TESTS] Fix DataStreamReaderWriterSuite to check writer options correctly
5a81f60 is described below

commit 5a81f600bdad7ea93e7c3116c08b3786658887b3
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Wed Sep 9 19:46:55 2020 -0700

    [SPARK-32836][SS][TESTS] Fix DataStreamReaderWriterSuite to check writer options correctly
    
    ### What changes were proposed in this pull request?
    
    This PR aims to fix the test coverage at `DataStreamReaderWriterSuite`.
    
    ### Why are the changes needed?
    
    Currently, the test case checks `DataStreamReader` options instead of `DataStreamWriter` options.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Pass the revised test case.
    
    Closes #29701 from dongjoon-hyun/SPARK-32836.
    
    Authored-by: Dongjoon Hyun <do...@apache.org>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
    (cherry picked from commit 06a994517fc3080b14f01183eeb17e56ab52eaa8)
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 .../sql/streaming/test/DataStreamReaderWriterSuite.scala   | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala
index f9fc540..b646387 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala
@@ -169,18 +169,20 @@ class DataStreamReaderWriterSuite extends StreamTest with BeforeAndAfter {
 
     LastOptions.clear()
 
-    df.writeStream
+    val query = df.writeStream
       .format("org.apache.spark.sql.streaming.test")
-      .option("opt1", "1")
-      .options(Map("opt2" -> "2"))
+      .option("opt1", "5")
+      .options(Map("opt2" -> "4"))
       .options(map)
       .option("checkpointLocation", newMetadataDir)
       .start()
-      .stop()
 
-    assert(LastOptions.parameters("opt1") == "1")
-    assert(LastOptions.parameters("opt2") == "2")
+    assert(LastOptions.parameters("opt1") == "5")
+    assert(LastOptions.parameters("opt2") == "4")
     assert(LastOptions.parameters("opt3") == "3")
+    assert(LastOptions.parameters.contains("checkpointLocation"))
+
+    query.stop()
   }
 
   test("partitioning") {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org