You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2022/12/16 19:32:00 UTC
[jira] [Resolved] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions
[ https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-41365.
-----------------------------------
Fix Version/s: 3.3.2
3.4.0
Target Version/s: (was: 3.3.1)
Resolution: Fixed
> Stages UI page fails to load for proxy in some yarn versions
> -------------------------------------------------------------
>
> Key: SPARK-41365
> URL: https://issues.apache.org/jira/browse/SPARK-41365
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 3.3.1
> Environment: as above
> Reporter: Mars
> Priority: Major
> Fix For: 3.3.2, 3.4.0
>
> Attachments: image-2022-12-02-17-53-03-003.png
>
>
> My environment CDH 5.8 , click to enter the spark UI from the yarn interface
> when visit the stage URI, it fails to load, URI is
> {code:java}
> http://<yarn-url>:8088/proxy/application_1669877165233_0021/stages/stage/?id=0&attempt=0 {code}
> !image-2022-12-02-17-53-03-003.png|width=430,height=697!
> Server error stack trace:
> {code:java}
> Caused by: java.lang.NullPointerException
> at org.apache.spark.status.api.v1.StagesResource.$anonfun$doPagination$1(StagesResource.scala:207)
> at org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
> at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
> at org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
> at org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
> at org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
> at org.apache.spark.status.api.v1.StagesResource.doPagination(StagesResource.scala:206)
> at org.apache.spark.status.api.v1.StagesResource.$anonfun$taskTable$1(StagesResource.scala:161)
> at org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
> at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
> at org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
> at org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
> at org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
> at org.apache.spark.status.api.v1.StagesResource.taskTable(StagesResource.scala:145)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62){code}
>
> The issue is similar to, the final phenomenon of the issue is the same, because the parameter encode twice
> https://issues.apache.org/jira/browse/SPARK-32467
> https://issues.apache.org/jira/browse/SPARK-33611
> The two issues solve two scenarios to avoid encode twice:
> 1. https redirect proxy
> 2. set reverse proxy enabled (spark.ui.reverseProxy) in Nginx
> But if encode twice due to other reasons, such as this issue (yarn proxy), it will also fail
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org