You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Gengliang Wang (JIRA)" <ji...@apache.org> on 2018/12/13 14:49:00 UTC

[jira] [Created] (SPARK-26363) Remove redundant field `executorLogs` in TaskData

Gengliang Wang created SPARK-26363:
--------------------------------------

             Summary: Remove redundant field `executorLogs` in TaskData
                 Key: SPARK-26363
                 URL: https://issues.apache.org/jira/browse/SPARK-26363
             Project: Spark
          Issue Type: Improvement
          Components: Web UI
    Affects Versions: 3.0.0
            Reporter: Gengliang Wang


In https://github.com/apache/spark/pull/21688, a new filed `executorLogs` is added to `TaskData` in `api.scala`:
1. The field should not belong to `TaskData` (from the meaning of wording).
2. This is redundant with ExecutorSummary. 
3. For each row in the task table, the executor log value is lookup in KV store every time, which can be avoided for better performance in large scale.

This PR propose to reuse the executor details of request "/allexecutors" , so that we can have a cleaner api data structure, and redundant KV store queries are avoided. 





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org