[jira] [Commented] (SPARK-11630) ClosureCleaner incorrectly warns for class based closures

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333366#comment-16333366 ] 

Apache Spark commented on SPARK-11630:
--------------------------------------

User 'rekhajoshm' has created a pull request for this issue:
https://github.com/apache/spark/pull/20337

> ClosureCleaner incorrectly warns for class based closures
> ---------------------------------------------------------
>
>                 Key: SPARK-11630
>                 URL: https://issues.apache.org/jira/browse/SPARK-11630
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 1.4.1
>            Reporter: Frens Jan Rumph
>            Priority: Trivial
>
> Spark's `ClosureCleaner` utility seems to check whether a function is an anonymous function: [ClosureCleaner.scala on line 49|https://github.com/apache/spark/blob/f85aa06464a10f5d1563302fd76465dded475a12/core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala#L49] If not, it warns the user.
> However, I'm using some class based functions. Something along the lines of:
> {code}
> trait FromUnreadRow[T] extends (UnreadRow => T) with Serializable
> object ToPlainRow extends FromUnreadRow[PlainRow] {
>   override def apply(row: UnreadRow): PlainRow = ???
> }
> {code}
> This works just fine. I can't really see that the warning is actually useful in this case. I appreciate checking for common 'mistakes', but in my case a user might be alarmed unnecessarily.
> Anything that can be done about this? Anything I can do?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org