You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:23:38 UTC
[jira] [Updated] (SPARK-7172) Worker includes ExecutorRunner and
DriverRunner in Akka messages
[ https://issues.apache.org/jira/browse/SPARK-7172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-7172:
--------------------------------
Labels: bulk-closed (was: )
> Worker includes ExecutorRunner and DriverRunner in Akka messages
> ----------------------------------------------------------------
>
> Key: SPARK-7172
> URL: https://issues.apache.org/jira/browse/SPARK-7172
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, Web UI
> Affects Versions: 1.2.2, 1.3.1
> Reporter: Jacek Lewandowski
> Priority: Minor
> Labels: bulk-closed
>
> Worker recognises {{RequestWorkerState}} messages which are sent by the UI to get the detailed information about the current state of the worker. The worker responds with {{WorkerStateResponse}} which includes the requested information.
> h5.Problem
> {{WorkerStateResponse}} includes two lists of {{ExecutorRunner}} objects and two lists of {{DriverRunner}} objects. These objects are mutable and actually performs some work. They shouldn't be used to move the information through the Akka actor system. The first of all, this is discouraged practise. The second is that when we want to get the information about the worker from some other actor system, Akka needs to serialise it it shows that these objects are not serialisable. This would be particularly useful for examining the state of Spark cluster in some test suites.
> h5.Proposed solution
> Implemented dedicated, immutable case classes to reflect the snapshot of the worker state and use them instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org