[jira] [Updated] (TOREE-464) Failing actions on RDDs with non-primitive objects

["Luciano Resende (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TOREE-464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Luciano Resende updated TOREE-464:
----------------------------------
    Attachment: toree-464.png

> Failing actions on RDDs with non-primitive objects
> --------------------------------------------------
>
>                 Key: TOREE-464
>                 URL: https://issues.apache.org/jira/browse/TOREE-464
>             Project: TOREE
>          Issue Type: Bug
>    Affects Versions: 0.2.0
>            Reporter: Alexander Anokhin
>            Priority: Major
>             Fix For: 0.5.0
>
>         Attachments: test_case.ipynb, toree-464.png
>
>
> Running in non-local mode actions on RDDs fail when RDDs hold custom non-primitive objects. It affects current master version [0.3.0|https://github.com/apache/incubator-toree] as well as released versions [Rc1|https://github.com/apache/incubator-toree/releases/tag/v0.2.0-incubating-rc1], [Rc2|https://github.com/apache/incubator-toree/releases/tag/v0.2.0-incubating-rc2] and [Rc3|https://github.com/apache/incubator-toree/releases/tag/v0.2.0-incubating-rc3].
> Example:
> {code:scala}
> Cell 1: case class A(i: Int)
> Cell 2: val events = sc.parallelize((1 to 5).toSeq).map { i => A(i) }
> Cell 3: println(events.count())
> Results: "java.lang.NoClassDefFoundError: Could not initialize class ..."
> {code}
> However, it does work if code from the cells 1 and 2 is combined into one cell. In that case actions on such RDDs work correctly, but case class definition should be prepended to every cell with problem RDD.
> Example:
> {code:scala}
> Cell 1: case class A(i: Int); val events = sc.parallelize((1 to 5).toSeq).map { i => A(i) }
> Cell 2: println(events.count())
> Results: "5"
> {code}
>  
> Test notebook is in the attachment  [^test_case.ipynb] .



--
This message was sent by Atlassian Jira
(v8.3.4#803005)