[jira] [Created] (ZEPPELIN-158) Case classes do not persist across nodes/shuffles

["Rick Moritz (JIRA)" <ji...@apache.org>]

Rick Moritz created ZEPPELIN-158:
------------------------------------

             Summary: Case classes do not persist across nodes/shuffles
                 Key: ZEPPELIN-158
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-158
             Project: Zeppelin
          Issue Type: Bug
          Components: Core
         Environment: spark-1.2.1, YARN
            Reporter: Rick Moritz
            Priority: Critical


Load data from an object file created in spark-shell (1.2.0) using a copy-pasted case class;

defined class Transaction postETLtraining: org.apache.spark.rdd.RDD[Nothing] = FlatMappedRDD[4957] at objectFile at <console>:177 postETLtest: org.apache.spark.rdd.RDD[Nothing] = FlatMappedRDD[4959] at objectFile at <console>:176 org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 562.0 failed 4 times, most recent failure: Lost task 0.3 in stage 562.0 (TID 13208, fi00z175.fidudom.fiducia.de): java.lang.ClassNotFoundException: $iwC$$iwC$Transaction at org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:65) [etc.]

Similar errors also occur when attempting to join/orderBy case-class based schemaRDDs. The same code runs fine in a similarly set-up spark-shell (1.2.0 vs 1.2.1, but Zeppelin doesn't build against 1.2.0 due to missing REPL APIs)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)