[jira] [Resolved] (SPARK-1857) map() with lookup() causes exception

["Patrick Wendell (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Patrick Wendell resolved SPARK-1857.
------------------------------------

    Resolution: Not a Problem

I'm closing this but report for now since it's a known property of the current API.

> map() with lookup() causes exception
> ------------------------------------
>
>                 Key: SPARK-1857
>                 URL: https://issues.apache.org/jira/browse/SPARK-1857
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 0.9.0
>            Reporter: Michael Malak
>
> Using map() and lookup() in conjunction throws an exception
> {noformat}
> val a = sc.parallelize(Array(11))
> val m = sc.parallelize(Array((11,21)))
> a.map(m.lookup(_)(0)).collect
> 14/05/14 15:03:35 ERROR Executor: Exception in task ID 23
> scala.MatchError: null
> at org.apache.spark.rdd.PairRDDFunctions.lookup(PairRDDFunctions.scala:551)
> {noformat}
> A workaround is:
> {noformat}
> a.map((_,0)).join(m).map(_._2._2).collect
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)