[jira] [Created] (SPARK-26940) Observed greater deviation Big Endian for SingletonReplSuite test case

["Anuja Jakhade (JIRA)" <ji...@apache.org>]

Anuja Jakhade created SPARK-26940:
-------------------------------------

             Summary: Observed greater deviation Big Endian for SingletonReplSuite test case
                 Key: SPARK-26940
                 URL: https://issues.apache.org/jira/browse/SPARK-26940
             Project: Spark
          Issue Type: Test
          Components: Tests
    Affects Versions: 2.3.2
         Environment: Ubuntu 16.04 LTS

openjdk version "1.8.0_202"
OpenJDK Runtime Environment (build 1.8.0_202-b08)
Eclipse OpenJ9 VM (build openj9-0.12.1, JRE 1.8.0 Linux (JIT enabled, AOT enabled)
OpenJ9 - 90dd8cb40
OMR - d2f4534b
JCL - d002501a90 based on jdk8u202-b08)
            Reporter: Anuja Jakhade


I have built Apache Spark v2.3.2 on Big Endian with AdoptJDK OpenJ9 1.8.0_202.

My build is successful. However while running the scala tests of "*Spark Project REPL*" module. I am facing failures at SingletonReplSuite with error log as below 

 
 - should clone and clean line object in ClosureCleaner *** FAILED ***
 isContain was true Interpreter output contained 'AssertionError':

scala> import org.apache.spark.rdd.RDD

scala>
 scala> lines: org.apache.spark.rdd.RDD[String] = pom.xml MapPartitionsRDD[46] at textFile at <console>:40

scala> defined class Data

scala> dataRDD: org.apache.spark.rdd.RDD[Data] = MapPartitionsRDD[47] at map at <console>:43

scala> res28: Long = 180

scala> repartitioned: org.apache.spark.rdd.RDD[Data] = MapPartitionsRDD[51] at repartition at <console>:41

scala> res29: Long = 180

scala>
 scala> | | getCacheSize: (rdd: org.apache.spark.rdd.RDD[_])Long

scala> cacheSize1: Long = 24608

scala> cacheSize2: Long = 17768

scala>
 scala>
 scala> deviation: Double = 0.2779583875162549

scala> | java.lang.AssertionError: assertion failed: deviation too large: 0.2779583875162549, first size: 24608, second size: 17768
 at scala.Predef$.assert(Predef.scala:170)
 ... 46 elided

scala> | _result_1550641172995: Int = 1

scala> (SingletonReplSuite.scala:121)

 

The deviation observed on big endian is greater than the acceptable deviation 0.2.

How efficient is it to increase the deviation defined in SingletonReplSuite.scala

Can this be fixed? 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org