You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "sandeep katta (JIRA)" <ji...@apache.org> on 2018/09/11 08:57:00 UTC
[jira] [Commented] (SPARK-25392) [Spark Job History]Inconsistent
behaviour for pool details in spark web UI and history server page
[ https://issues.apache.org/jira/browse/SPARK-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610306#comment-16610306 ]
sandeep katta commented on SPARK-25392:
---------------------------------------
From the HistoryServer for the same application it throws exception as Unknown pool
Code which is throwing exception
// For now, pool information is only accessible in live UIs
val pool = parent.sc.flatMap(_.getPoolForName(poolName)).getOrElse {
throw new IllegalArgumentException(s"Unknown pool: $poolName")
}
As per code it is clear HistoryServer does not have sparkContext so it can’t get pool details.
Do you guys think it is required to support for HistoryServer or is this valid behaviour ?
cc [~sujith] [~hyukjin.kwon]
> [Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page
> ---------------------------------------------------------------------------------------------------
>
> Key: SPARK-25392
> URL: https://issues.apache.org/jira/browse/SPARK-25392
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.1
> Environment: OS: SUSE 11
> Spark Version: 2.3
> Reporter: ABHISHEK KUMAR GUPTA
> Priority: Major
>
> Steps:
> 1.Enable spark.scheduler.mode = FAIR
> 2.Submitted beeline jobs
> create database JH;
> use JH;
> create table one12( id int );
> insert into one12 values(12);
> insert into one12 values(13);
> Select * from one12;
> 3.Click on JDBC Incompleted Application ID in Job History Page
> 4. Go to Job Tab in staged Web UI page
> 5. Click on run at AccessController.java:0 under Desription column
> 6 . Click default under Pool Name column of Completed Stages table
> URL:http://blr1000023109:23020/history/application_1536399199015_0006/stages/pool/?poolname=default
> 7. It throws below error
> HTTP ERROR 400
> Problem accessing /history/application_1536399199015_0006/stages/pool/. Reason:
> Unknown pool: default
> Powered by Jetty:// x.y.z
> But under
> Yarn resource page it display the summary under Fair Scheduler Pool: default
> URL:https://blr1000023110:64323/proxy/application_1536399199015_0006/stages/pool?poolname=default
> Summary
> Pool Name Minimum Share Pool Weight Active Stages Running Tasks SchedulingMode
> default 0 1 0 0 FIFO
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org