You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "holdenk (JIRA)" <ji...@apache.org> on 2016/10/08 04:41:20 UTC
[jira] [Commented] (SPARK-3513) Provide a utility for running a
function once on each executor
[ https://issues.apache.org/jira/browse/SPARK-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15557193#comment-15557193 ]
holdenk commented on SPARK-3513:
--------------------------------
This seems closely related to SPARK-650 and SPARK-636 as well. Is this general group of tasks something we want to look at enabling or are the current workaround (either mapPartitions or broadcast with custom initilization) "good enough"? (cc [~matei], [~joshrosen] & [~patrick]) who have commented on the different (related) JIRAs?
> Provide a utility for running a function once on each executor
> --------------------------------------------------------------
>
> Key: SPARK-3513
> URL: https://issues.apache.org/jira/browse/SPARK-3513
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Patrick Wendell
> Priority: Minor
>
> This is minor, but it would be nice to have a utility where you can pass a function and it will run some arbitrary function once on each each executor and return the result to you (e.g. you could perform a jstack from within the JVM). You could probably hack it together with custom locality preferences, accessing the list of live executors, and mapPartitions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org