You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Prashant Sharma (JIRA)" <ji...@apache.org> on 2016/03/08 07:07:40 UTC
[jira] [Comment Edited] (SPARK-3200) Class defined with reference
to external variables crashes in REPL.
[ https://issues.apache.org/jira/browse/SPARK-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184468#comment-15184468 ]
Prashant Sharma edited comment on SPARK-3200 at 3/8/16 6:06 AM:
----------------------------------------------------------------
Hi [~chrismattmann], It never worked. I have clarified above. But since no one apart from me ever ran into this and complexity of the fix was non trivial, it was "won't fix".
Actually now that we use the scala repl "as is" without much modifications. So if it needs to be fixed, there is a considerably large amount of change than it was required back then. Along with the change the maintenance overhead will also be large.
However, if the fix is in high demand. One can go ahead and fix in the scala repl too. It is also possible to work around it. Did you ran into this issue ?
[EDIT] The patch proposed in the Jira can still be merged for scala 2.10 port of scala repl that lives in Spark. But then it should be done, if this fix is highly critical.
was (Author: prashant_):
Hi [~chrismattmann], It never worked. I have clarified above. But since no one apart from me ever ran into this and complexity of the fix was non trivial, it was "won't fix".
Actually now that we use the scala repl "as is" without much modifications. So if it needs to be fixed, there is a considerably large amount of change than it was required back then. Along with the change the maintenance overhead will also be large.
However, if the fix is in high demand. One can go ahead and fix in the scala repl too. It is also possible to work around it. Did you ran into this issue ?
> Class defined with reference to external variables crashes in REPL.
> -------------------------------------------------------------------
>
> Key: SPARK-3200
> URL: https://issues.apache.org/jira/browse/SPARK-3200
> Project: Spark
> Issue Type: Bug
> Components: Spark Shell
> Affects Versions: 1.1.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
>
> Reproducer:
> {noformat}
> val a = sc.textFile("README.md").count
> case class A(i: Int) { val j = a}
> sc.parallelize(1 to 10).map(A(_)).collect()
> {noformat}
> This will happen only in distributed mode, when one refers something that refers sc and not otherwise.
> There are many ways to work around this, like directly assign a constant value instead of referring the variable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org