You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "holdenk (JIRA)" <ji...@apache.org> on 2017/02/02 09:41:51 UTC
[jira] [Commented] (SPARK-2868) Support named accumulators in
Python
[ https://issues.apache.org/jira/browse/SPARK-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849723#comment-15849723 ]
holdenk commented on SPARK-2868:
--------------------------------
This might be a difficult issue to start of with [~heathkh] - the accumulator API has been changed a lot on the backend and the Python API is still exposing the old API. If your interested I might start by picking a smaller PySpark related issue and then coming back to this. I'd be happy to chat with you though about what the work involved might look like for this issue.
> Support named accumulators in Python
> ------------------------------------
>
> Key: SPARK-2868
> URL: https://issues.apache.org/jira/browse/SPARK-2868
> Project: Spark
> Issue Type: New Feature
> Components: PySpark
> Reporter: Patrick Wendell
>
> SPARK-2380 added this for Java/Scala. To allow this in Python we'll need to make some additional changes. One potential path is to have a 1:1 correspondence with Scala accumulators (instead of a one-to-many). A challenge is exposing the stringified values of the accumulators to the Scala code.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org