[spark] branch branch-2.4 updated: [SPARK-28357][CORE][TEST] Fix Flaky Test - FileAppenderSuite.rollingfile appender - size-based rolling compressed

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-2.4 by this push:
     new 094a20c  [SPARK-28357][CORE][TEST] Fix Flaky Test - FileAppenderSuite.rollingfile appender - size-based rolling compressed
094a20c is described below

commit 094a20ce9c110084f427897f95860981d1fc8212
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Fri Jul 12 18:40:07 2019 +0900

    [SPARK-28357][CORE][TEST] Fix Flaky Test - FileAppenderSuite.rollingfile appender - size-based rolling compressed
    
    ## What changes were proposed in this pull request?
    
    `SizeBasedRollingPolicy.shouldRollover` returns false when the size is equal to `rolloverSizeBytes`.
    ```scala
      /** Should rollover if the next set of bytes is going to exceed the size limit */
      def shouldRollover(bytesToBeWritten: Long): Boolean = {
        logDebug(s"$bytesToBeWritten + $bytesWrittenSinceRollover > $rolloverSizeBytes")
        bytesToBeWritten + bytesWrittenSinceRollover > rolloverSizeBytes
      }
    ```
    - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107553/testReport/org.apache.spark.util/FileAppenderSuite/rolling_file_appender___size_based_rolling__compressed_/
    ```
    org.scalatest.exceptions.TestFailedException: 1000 was not less than 1000
    ```
    
    ## How was this patch tested?
    
    Pass the Jenkins with the updated test.
    
    Closes #25125 from dongjoon-hyun/SPARK-28357.
    
    Authored-by: Dongjoon Hyun <dh...@apple.com>
    Signed-off-by: HyukjinKwon <gu...@apache.org>
    (cherry picked from commit 1c29212394adcbde2de4f4dfdc43a1cf32671ae1)
    Signed-off-by: HyukjinKwon <gu...@apache.org>
---
 core/src/test/scala/org/apache/spark/util/FileAppenderSuite.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/core/src/test/scala/org/apache/spark/util/FileAppenderSuite.scala b/core/src/test/scala/org/apache/spark/util/FileAppenderSuite.scala
index 52cd537..1a3e880 100644
--- a/core/src/test/scala/org/apache/spark/util/FileAppenderSuite.scala
+++ b/core/src/test/scala/org/apache/spark/util/FileAppenderSuite.scala
@@ -128,7 +128,7 @@ class FileAppenderSuite extends SparkFunSuite with BeforeAndAfter with Logging {
     val files = testRolling(appender, testOutputStream, textToAppend, 0, isCompressed = true)
     files.foreach { file =>
       logInfo(file.toString + ": " + file.length + " bytes")
-      assert(file.length < rolloverSize)
+      assert(file.length <= rolloverSize)
     }
   }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org