You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "shahid (JIRA)" <ji...@apache.org> on 2019/02/05 07:11:00 UTC
[jira] [Commented] (SPARK-26760) [Spark Incorrect display in SPARK
UI Executor Tab when number of cores is 4 and Active Task display as 5 in
Executor Tab of SPARK UI]
[ https://issues.apache.org/jira/browse/SPARK-26760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760521#comment-16760521 ]
shahid commented on SPARK-26760:
--------------------------------
After analyzing, following are the findings
1) Tasks related informations don't update on time, especially when the task duration is less.
2) There is a configuration related to updating the UI, "spark.ui.liveUpdate.period", default value is 100ms. So, if we reduce the period, or make it 0, this issue won't happen.
Please let me know for any issues.
> [Spark Incorrect display in SPARK UI Executor Tab when number of cores is 4 and Active Task display as 5 in Executor Tab of SPARK UI]
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-26760
> URL: https://issues.apache.org/jira/browse/SPARK-26760
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.4.0
> Environment: Spark 2.4
> Reporter: ABHISHEK KUMAR GUPTA
> Priority: Major
> Attachments: SPARK-26760.png
>
>
> Steps:
> # Launch Spark Shell
> # bin/spark-shell --master yarn --conf spark.dynamicAllocation.enabled=true --conf spark.dynamicAllocation.initialExecutors=3 --conf spark.dynamicAllocation.minExecutors=1 --conf spark.dynamicAllocation.executorIdleTimeout=60s --conf spark.dynamicAllocation.maxExecutors=5
> # Submit a Job sc.parallelize(1 to 10000,116000).count()
> # Check the YARN UI Executor Tab for the RUNNING application
> # UI display as Number of cores 4 and Active Tasks column shows as 5
> Expected:
> It Number of Active Tasks should be same as Number of Cores.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org