[jira] [Resolved] (SPARK-27137) Spark captured variable is null if the code is pasted via :paste

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-27137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-27137.
----------------------------------
    Resolution: Cannot Reproduce

Seems not failing in the current master:

{code}
scala> :paste
// Entering paste mode (ctrl-D to finish)

val foo = "foo"

def f(arg: Any): Unit = {
  Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo"))
}

sc.parallelize(Seq(1, 2), 2).foreach(f)

// Exiting paste mode, now interpreting.

foo: String = foo
f: (arg: Any)Unit
{code}

It would be great if we can identify the JIRA and backport it if applicable.

> Spark captured variable is null if the code is pasted via :paste
> ----------------------------------------------------------------
>
>                 Key: SPARK-27137
>                 URL: https://issues.apache.org/jira/browse/SPARK-27137
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.4.0
>            Reporter: Osira Ben
>            Priority: Major
>
> If I execute this piece of code
> {code:java}
> val foo = "foo"
> def f(arg: Any): Unit = {
>   Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo"))
> }
> sc.parallelize(Seq(1, 2), 2).foreach(f)
> {code}
> {{in spark2-shell via :paste it throws}}
> {code:java}
> scala> :paste
> // Entering paste mode (ctrl-D to finish)
> val foo = "foo"
> def f(arg: Any): Unit = {
>   Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo"))
> }
> sc.parallelize(Seq(1, 2), 2).foreach(f)
> // Exiting paste mode, now interpreting.
> 19/03/11 15:02:06 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 1.0 (TID 2, hadoop.company.com, executor 1): java.lang.NullPointerException: foo
>         at java.util.Objects.requireNonNull(Objects.java:228)
> {code}
> However if I execute it pasting without :paste or via spark2-shell -i it doesn't.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org