You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/09/17 17:35:15 UTC
[spark] branch branch-3.5 updated: [SPARK-45187][CORE] Fix `WorkerPage` to use the same pattern for `logPage` urls
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new 84a053e72ac [SPARK-45187][CORE] Fix `WorkerPage` to use the same pattern for `logPage` urls
84a053e72ac is described below
commit 84a053e72ac9d9cfc91bab777cea94958d3a91da
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Sun Sep 17 10:34:23 2023 -0700
[SPARK-45187][CORE] Fix `WorkerPage` to use the same pattern for `logPage` urls
### What changes were proposed in this pull request?
This PR aims to use the same pattern for `logPage` urls of `WorkerPage` to make it work consistently when `spark.ui.reverseProxy=true`.
### Why are the changes needed?
Since Apache Spark 3.2.0 (SPARK-34635, #31753), Apache Spark adds trailing slashes to reduce redirections for `logPage`.
```scala
s"$workerUrlRef/logPage?driverId=$driverId&logType=stdout")
s"$workerUrlRef/logPage/?driverId=$driverId&logType=stdout")
...
<a href={s"$workerUrlRef/logPage?appId=${executor
<a href={s"$workerUrlRef/logPage/?appId=${executor
```
This PR aims to fix a leftover in `WorkerPage` to make it work consistently in case of the reverse proxy situation via `spark.ui.reverseProxy`. Currently, in some proxy environments, `appId` link is working but `driverId` link is broken due to the redirections. This inconsistent behavior makes the users confused.
```
- <a href={s"$workerUrlRef/logPage?driverId=${driver.driverId}&logType=stdout"}>stdout</a>
- <a href={s"$workerUrlRef/logPage?driverId=${driver.driverId}&logType=stderr"}>stderr</a>
+ <a href={s"$workerUrlRef/logPage/?driverId=${driver.driverId}&logType=stdout"}>stdout</a>
+ <a href={s"$workerUrlRef/logPage/?driverId=${driver.driverId}&logType=stderr"}>stderr</a>
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual tests because it requires a reverse proxy.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #42959 from dongjoon-hyun/SPARK-45187.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
(cherry picked from commit f8f2735426ee7ad3d7a1f5bd07e72643516f4a35)
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala b/core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala
index 3171d3f16e8..e740b328dd7 100644
--- a/core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala
@@ -212,8 +212,8 @@ private[ui] class WorkerPage(parent: WorkerWebUI) extends WebUIPage("") {
</td>
<td>{formatResourcesAddresses(driver.resources)}</td>
<td>
- <a href={s"$workerUrlRef/logPage?driverId=${driver.driverId}&logType=stdout"}>stdout</a>
- <a href={s"$workerUrlRef/logPage?driverId=${driver.driverId}&logType=stderr"}>stderr</a>
+ <a href={s"$workerUrlRef/logPage/?driverId=${driver.driverId}&logType=stdout"}>stdout</a>
+ <a href={s"$workerUrlRef/logPage/?driverId=${driver.driverId}&logType=stderr"}>stderr</a>
</td>
<td>
{driver.finalException.getOrElse("")}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org