[jira] [Created] (SPARK-7172) Worker includes ExecutorRunner and DriverRunner in Akka messages

["Jacek Lewandowski (JIRA)" <ji...@apache.org>]

Jacek Lewandowski created SPARK-7172:
----------------------------------------

             Summary: Worker includes ExecutorRunner and DriverRunner in Akka messages
                 Key: SPARK-7172
                 URL: https://issues.apache.org/jira/browse/SPARK-7172
             Project: Spark
          Issue Type: Bug
          Components: Spark Core, Web UI
    Affects Versions: 1.3.1, 1.2.2, 1.4.0
            Reporter: Jacek Lewandowski


Worker recognises {{RequestWorkerState}} messages which are sent by the UI to get the detailed information about the current state of the worker. The worker responds with {{WorkerStateResponse}} which includes the requested information. 

h5.Problem
{{WorkerStateResponse}} includes two lists of {{ExecutorRunner}} objects and two lists of {{DriverRunner}} objects. These objects are mutable and actually performs some work. They shouldn't be used to move the information through the Akka actor system. The first of all, this is discouraged practise. The second is that when we want to get the information about the worker from some other actor system, Akka needs to serialise it it shows that these objects are not serialisable. This would be particularly useful for examining the state of Spark cluster in some test suites. 

h5.Proposed solution
Implemented dedicated, immutable case classes to reflect the snapshot of the worker state and use them instead.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org