You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2018/12/13 14:55:14 UTC
[GitHub] gengliangwang opened a new pull request #23310:
[SPARK-26363][WebUI] Remove redundant field `executorLogs` in TaskData
gengliangwang opened a new pull request #23310: [SPARK-26363][WebUI] Remove redundant field `executorLogs` in TaskData
URL: https://github.com/apache/spark/pull/23310
## What changes were proposed in this pull request?
In https://github.com/apache/spark/pull/21688, a new filed `executorLogs` is added to `TaskData` in `api.scala`:
1. The field should not belong to `TaskData` (from the meaning of wording).
2. This is redundant with ExecutorSummary.
3. For each row in the task table, the executor log value is lookup in KV store every time, which can be avoided for better performance.
This PR propose to reuse the executor details of request "/allexecutors" , so that we can have a cleaner api data structure, and redundant KV store queries are avoided.
(Before https://github.com/apache/spark/pull/21688 , stage page used a hash map to avoid duplicated executor logs lookup. But I think reusing the result of "allexecutors" is better.)
## How was this patch tested?
Manual check
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org