You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by zs...@apache.org on 2026/10/14 01:04:04 UTC
(spark) branch master updated: [SPARK-60019][UI] Build the executor thread dump download in the browser
This is an automated email from the ASF dual-hosted git repository.
zsxwing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c5e7769433ce [SPARK-60019][UI] Build the executor thread dump download in the browser
c5e7769433ce is described below
commit c5e7769433ce23abcee1f914a39695d7d62bc123
Author: Shixiong Zhu <zs...@gmail.com>
AuthorDate: Tue Oct 13 18:03:50 2026 -0700
[SPARK-60019][UI] Build the executor thread dump download in the browser
### What changes were proposed in this pull request?
The Download button on the executor thread dump page inlines the whole dump into a `data:` URI `href`:
```scala
href={"data:text/plain;charset=utf-8," + threadDump.map(_.toString).mkString}
```
This PR stops putting the dump in a URL. Like the SQL plan Download button (`downloadPlanBlob` in `spark-sql-viz.js`), the button now builds the file in the browser:
- `ExecutorThreadDumpPage` renders the Download link without an `href` (`data-action="downloadThreadDump"`, `data-filename=...`) and embeds the dump, unchanged, in a hidden `<pre id="thread-dump-text" class="d-none">`.
- `table.js` gets a `downloadThreadDump` function that builds a `Blob` from that element and downloads it, wired through the same delegated `data-action` listeners as Expand All / Collapse All.
A new internal config, `spark.ui.threadDump.downloadInBrowser.enabled` (default `true`), is a kill switch. When it is `false`, the page renders the original `data:` URI link.
### Why are the changes needed?
When a browser materializes a `data:` URI download, it excludes the URL fragment (everything from the first `#`) before decoding the body. See step 2 of the WHATWG "data: URL processor": https://url.spec.whatwg.org/#data-url-processor. Chromium (`GURL::GetContentPiece`) and Firefox (`nsDataHandler::ParseURI`) both implement this.
Since https://github.com/apache/spark/pull/53634 (SPARK-54861), idle task threads are named `Executor task idle worker#<threadId>`, so nearly every executor thread dump contains a `#`. The downloaded file is silently truncated at the first such thread, losing it and every thread after it.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
New tests:
- `ui-test/tests/table.test.js` (jsdom, run by the "Run Spark UI tests" job): clicking Download saves a `Blob` whose text is the complete embedded dump, including the thread after one whose name contains `#`, with the expected file name. Clicking the legacy `data:` URI link does not trigger the in-browser download.
- `ExecutorThreadDumpPageSuite`: renders the page with a fixed thread dump and checks the markup for each value of `spark.ui.threadDump.downloadInBrowser.enabled`. By default, Download is a link with no `href` and the hidden `thread-dump-text` element holds the complete dump; with the kill switch off, it is the legacy `data:` URI link. It also checks that `table.js` uses the `data-action`, `data-filename` and element id the page renders, so a change on either side fails.
These tests check that the dump is never put in a URL. They do not reproduce the browser's `data:` URI truncation itself, which needs a real browser download.
Manual test:
1. Start `bin/spark-shell --master 'local-cluster[2,1,1024]'`.
2. Run `spark.range(10).repartition(20).show()` so the executors have idle task threads (`Executor task idle worker#N`).
3. Open the Executors tab, click Thread Dump for an executor, then click Download.
4. The saved `threaddump_<executorId>.txt` contains every thread on the page, including the `Executor task idle worker#N` threads and their stack traces.
5. Set `spark.ui.threadDump.downloadInBrowser.enabled=false` and repeat: the old `data:` link is rendered.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)
Closes #59265 from zsxwing/thread-dump-download-in-browser.
Authored-by: Shixiong Zhu <zs...@gmail.com>
Signed-off-by: Shixiong Zhu <zs...@gmail.com>
---
.../resources/org/apache/spark/ui/static/table.js | 18 +++
.../org/apache/spark/internal/config/UI.scala | 13 ++
.../spark/ui/exec/ExecutorThreadDumpPage.scala | 19 ++-
.../ui/exec/ExecutorThreadDumpPageSuite.scala | 124 ++++++++++++++++++
ui-test/tests/table.test.js | 138 +++++++++++++++++++++
5 files changed, 311 insertions(+), 1 deletion(-)
diff --git a/core/src/main/resources/org/apache/spark/ui/static/table.js b/core/src/main/resources/org/apache/spark/ui/static/table.js
index 493acd989323..b70a6a627f4c 100644
--- a/core/src/main/resources/org/apache/spark/ui/static/table.js
+++ b/core/src/main/resources/org/apache/spark/ui/static/table.js
@@ -63,6 +63,19 @@ function expandAllThreadStackTrace(toggleButton) {
}
/* eslint-enable no-unused-vars */
+// Builds the file in the browser (as the SQL plan download does) rather than linking to a data:
+// URI, which browsers truncate at the first '#'.
+function downloadThreadDump(fileName) {
+ const text = document.getElementById("thread-dump-text").textContent;
+ const url = URL.createObjectURL(new Blob([text], { type: "text/plain;charset=utf-8" }));
+ const link = document.createElement("a");
+ link.href = url;
+ link.download = fileName;
+ link.click();
+ // Release the Blob once the browser has started the download, as exportSparkProperties does.
+ setTimeout(function () { URL.revokeObjectURL(url); }, 100);
+}
+
function collapseAllThreadStackTrace(toggleButton) {
$('.accordion-body').each(function() {
$(this).remove()
@@ -120,6 +133,11 @@ $(function() {
collapseAllThreadStackTrace($(this).data("toggle-button") !== false);
});
+ // downloadThreadDump
+ $(document).on("click", "[data-action=downloadThreadDump]", function() {
+ downloadThreadDump($(this).data("filename"));
+ });
+
// onMouseOverAndOut
$(document).on("mouseenter mouseleave", "tr.accordion-heading[data-thread-id]", function() {
onMouseOverAndOut($(this).data("thread-id"));
diff --git a/core/src/main/scala/org/apache/spark/internal/config/UI.scala b/core/src/main/scala/org/apache/spark/internal/config/UI.scala
index 4c2e328bf0c0..bc3d4f13b7cd 100644
--- a/core/src/main/scala/org/apache/spark/internal/config/UI.scala
+++ b/core/src/main/scala/org/apache/spark/internal/config/UI.scala
@@ -133,6 +133,19 @@ private[spark] object UI {
.booleanConf
.createWithDefault(true)
+ val UI_THREAD_DUMP_DOWNLOAD_IN_BROWSER_ENABLED =
+ ConfigBuilder("spark.ui.threadDump.downloadInBrowser.enabled")
+ .internal()
+ .doc("Whether the Download button on the executor thread dump page builds the file in " +
+ "the browser from the dump embedded in the page. When disabled, the button is a link " +
+ "with the whole dump inlined into a data: URI, which browsers truncate at the first " +
+ "'#' (a character that thread names contain). This is a kill switch to restore the " +
+ "old link if building the file in the browser causes a regression.")
+ .version("4.4.0")
+ .withBindingPolicy(ConfigBindingPolicy.NOT_APPLICABLE)
+ .booleanConf
+ .createWithDefault(true)
+
val UI_HEAP_HISTOGRAM_ENABLED = ConfigBuilder("spark.ui.heapHistogramEnabled")
.doc("Whether to show a link for executor heap histogram in Executor page.")
.version("3.5.0")
diff --git a/core/src/main/scala/org/apache/spark/ui/exec/ExecutorThreadDumpPage.scala b/core/src/main/scala/org/apache/spark/ui/exec/ExecutorThreadDumpPage.scala
index 0f6c8cacd420..216295e0e518 100644
--- a/core/src/main/scala/org/apache/spark/ui/exec/ExecutorThreadDumpPage.scala
+++ b/core/src/main/scala/org/apache/spark/ui/exec/ExecutorThreadDumpPage.scala
@@ -23,6 +23,7 @@ import jakarta.servlet.http.HttpServletRequest
import org.apache.spark.SparkContext
import org.apache.spark.internal.config.UI.UI_FLAMEGRAPH_ENABLED
+import org.apache.spark.internal.config.UI.UI_THREAD_DUMP_DOWNLOAD_IN_BROWSER_ENABLED
import org.apache.spark.status.api.v1.ThreadStackTrace
import org.apache.spark.ui.{CspNonce, SparkUITab, UIUtils, WebUIPage}
import org.apache.spark.ui.UIUtils.{formatImportJavaScript, prependBaseUri}
@@ -68,6 +69,22 @@ private[ui] class ExecutorThreadDumpPage(
</tr>
}
+ // A data: URI link with the dump inlined is truncated by browsers at the first '#' (the
+ // URL fragment delimiter), which thread names such as "Executor task idle worker#292"
+ // contain. So by default the button has no href: it builds the file in the browser from
+ // the hidden element (see downloadThreadDump in table.js), the same way the SQL plan
+ // download does.
+ val downloadButton: Seq[Node] =
+ if (sc.forall(_.getReadOnlyConf.get(UI_THREAD_DUMP_DOWNLOAD_IN_BROWSER_ENABLED))) {
+ <a class="downloadbutton" data-action="downloadThreadDump"
+ data-filename={"threaddump_" + executorId + ".txt"}>Download</a>
+ <pre id="thread-dump-text" class="d-none">{threadDump.map(_.toString).mkString}</pre>
+ } else {
+ <a class="downloadbutton"
+ href={"data:text/plain;charset=utf-8," + threadDump.map(_.toString).mkString}
+ download={"threaddump_" + executorId + ".txt"}>Download</a>
+ }
+
<div class="row">
<div class="col-12">
<p>Updated at {UIUtils.formatDate(time)}</p>
@@ -97,7 +114,7 @@ private[ui] class ExecutorThreadDumpPage(
<div class="thead-stack-trace-table-button d-flex align-items-center">
<a class="expandbutton" data-action="expandAllThreadStackTrace">Expand All</a>
<a class="expandbutton d-none" data-action="collapseAllThreadStackTrace">Collapse All</a>
- <a class="downloadbutton" href={"data:text/plain;charset=utf-8," + threadDump.map(_.toString).mkString} download={"threaddump_" + executorId + ".txt"}>Download</a>
+ {downloadButton}
<div class="d-flex">
<div class="bs-example" data-example-id="simple-d-flex">
<div class="mb-3">
diff --git a/core/src/test/scala/org/apache/spark/ui/exec/ExecutorThreadDumpPageSuite.scala b/core/src/test/scala/org/apache/spark/ui/exec/ExecutorThreadDumpPageSuite.scala
new file mode 100644
index 000000000000..da422a1a175c
--- /dev/null
+++ b/core/src/test/scala/org/apache/spark/ui/exec/ExecutorThreadDumpPageSuite.scala
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ui.exec
+
+import java.nio.charset.StandardCharsets
+
+import scala.xml.{Elem, Group, Node, NodeSeq}
+
+import jakarta.servlet.http.HttpServletRequest
+import org.mockito.Mockito.{mock, when}
+
+import org.apache.spark.{SparkConf, SparkContext, SparkFunSuite}
+import org.apache.spark.internal.config.UI.UI_THREAD_DUMP_DOWNLOAD_IN_BROWSER_ENABLED
+import org.apache.spark.status.api.v1.{StackTrace, ThreadStackTrace}
+import org.apache.spark.ui.SparkUITab
+import org.apache.spark.util.Utils
+
+class ExecutorThreadDumpPageSuite extends SparkFunSuite {
+
+ // A thread whose name contains '#' is followed by another thread. A data: URI link is truncated
+ // by browsers at the first '#', which would drop the rest of the dump.
+ private val threadDump = Array(
+ threadStackTrace(55, "Executor task idle worker#55",
+ "java.base/java.lang.Thread.run(Thread.java:840)\n"),
+ threadStackTrace(1, "main",
+ "org.apache.spark.SparkContext.<init>(SparkContext.scala:100)\n"))
+
+ private val threadDumpText = threadDump.map(_.toString).mkString
+
+ private def threadStackTrace(id: Long, name: String, frame: String): ThreadStackTrace =
+ ThreadStackTrace(id, name, Thread.State.RUNNABLE, StackTrace(Seq(frame)), None, "",
+ Seq.empty, Seq.empty, Seq.empty, None, None, false, false, false, 5)
+
+ private def renderPage(downloadInBrowserEnabled: Boolean): NodeSeq = {
+ val conf = new SparkConf(false)
+ .set(UI_THREAD_DUMP_DOWNLOAD_IN_BROWSER_ENABLED, downloadInBrowserEnabled)
+ val sc = mock(classOf[SparkContext])
+ when(sc.conf).thenReturn(conf)
+ when(sc.getReadOnlyConf).thenReturn(conf)
+ when(sc.getExecutorThreadDump("1")).thenReturn(Some(threadDump))
+
+ val tab = mock(classOf[SparkUITab])
+ when(tab.appName).thenReturn("test")
+ when(tab.basePath).thenReturn("")
+ when(tab.headerTabs).thenReturn(Seq.empty)
+
+ val request = mock(classOf[HttpServletRequest])
+ when(request.getParameter("executorId")).thenReturn("1")
+ NodeSeq.fromSeq(new ExecutorThreadDumpPage(tab, Some(sc)).render(request))
+ }
+
+ // All elements of the page. NodeSeq's `\\` cannot be used: it fails on the page's <xml:group>.
+ private def elements(nodes: Seq[Node]): Seq[Elem] = nodes.flatMap {
+ case group: Group => elements(group.nodes)
+ case elem: Elem => elem +: elements(elem.child)
+ case _ => Nil
+ }
+
+ private def downloadControl(page: NodeSeq): Elem = {
+ val controls =
+ elements(page).filter(elem => (elem \@ "class").split(" ").contains("downloadbutton"))
+ assert(controls.length === 1)
+ controls.head
+ }
+
+ private def embeddedThreadDump(page: NodeSeq): Option[String] =
+ elements(page).find(e => e.label == "pre" && e \@ "id" == "thread-dump-text").map(_.text)
+
+ test("Download builds the file in the browser from the embedded thread dump") {
+ val page = renderPage(downloadInBrowserEnabled = true)
+
+ val download = downloadControl(page)
+ assert(download.label === "a")
+ assert(download.attribute("href").isEmpty)
+ assert(download \@ "data-action" === "downloadThreadDump")
+ assert(download \@ "data-filename" === "threaddump_1.txt")
+ // The hidden element holds the complete dump, including every thread after the '#'.
+ assert(embeddedThreadDump(page) === Some(threadDumpText))
+ }
+
+ test("table.js handles the Download markup the page renders") {
+ // Only a browser runs table.js against the rendered page, so check that the click listener
+ // and the download helper use the data-action, data-filename and element id the page emits.
+ val page = renderPage(downloadInBrowserEnabled = true)
+ val download = downloadControl(page)
+ val dumpElement = elements(page).find(e => e.label == "pre" && e.text == threadDumpText)
+ assert(dumpElement.isDefined, "the page must embed the thread dump")
+ assert(download.attribute("data-filename").isDefined)
+
+ val tableJs = Utils.tryWithResource(
+ getClass.getResourceAsStream("/org/apache/spark/ui/static/table.js")) { in =>
+ new String(in.readAllBytes(), StandardCharsets.UTF_8)
+ }
+ assert(tableJs.contains(s"[data-action=${download \@ "data-action"}]"))
+ assert(tableJs.contains(""".data("filename")"""))
+ assert(tableJs.contains(s"""getElementById("${dumpElement.get \@ "id"}")"""))
+ }
+
+ test("Download is the legacy data: URI link when the kill switch is off") {
+ val page = renderPage(downloadInBrowserEnabled = false)
+
+ val download = downloadControl(page)
+ assert(download.label === "a")
+ assert(download \@ "href" === "data:text/plain;charset=utf-8," + threadDumpText)
+ assert(download \@ "download" === "threaddump_1.txt")
+ assert(download.attribute("data-action").isEmpty)
+ assert(embeddedThreadDump(page).isEmpty)
+ }
+}
diff --git a/ui-test/tests/table.test.js b/ui-test/tests/table.test.js
new file mode 100644
index 000000000000..4926fa73b794
--- /dev/null
+++ b/ui-test/tests/table.test.js
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * @jest-environment jsdom
+ */
+
+import { readFileSync } from 'fs';
+import { join, dirname } from 'path';
+import { fileURLToPath } from 'url';
+import '../../core/src/main/resources/org/apache/spark/ui/static/jquery.min.js';
+
+/* global $ */
+
+// table.js is a classic (non-module) script. Running it registers its delegated click listeners
+// on the document once jQuery is ready.
+const __dirname = dirname(fileURLToPath(import.meta.url));
+const tablePath = join(
+ __dirname, '../../core/src/main/resources/org/apache/spark/ui/static/table.js');
+new Function(readFileSync(tablePath, 'utf8'))();
+
+// A thread name containing '#' is followed by more threads. A data: URI download is truncated at
+// the first '#', which would drop everything after it.
+const THREAD_DUMP =
+ '"Executor task idle worker#55" daemon prio=5 Id=55 TIMED_WAITING\n' +
+ ' at java.base/java.lang.Thread.run(Thread.java:840)\n' +
+ '\n' +
+ '"main" prio=5 Id=1 RUNNABLE\n' +
+ ' at org.apache.spark.SparkContext.<init>(SparkContext.scala:100)\n';
+
+// The page embeds the dump HTML-escaped, as Scala XML renders it.
+function escapeHtml(text) {
+ return text.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>')
+ .replace(/"/g, '"');
+}
+
+function readBlob(blob) {
+ return new Promise((resolve, reject) => {
+ const reader = new FileReader();
+ reader.onload = () => resolve(reader.result);
+ reader.onerror = () => reject(reader.error);
+ reader.readAsText(blob);
+ });
+}
+
+// jsdom implements neither URL.createObjectURL nor downloads, so record the Blob and the link
+// the helper clicks to start the download, and the Blob URLs it revokes.
+let downloads;
+let revokedUrls;
+let originalAnchorClick;
+
+beforeAll(() => new Promise(resolve => $(resolve)));
+
+beforeEach(() => {
+ downloads = [];
+ revokedUrls = [];
+ let lastBlob;
+ URL.createObjectURL = blob => {
+ lastBlob = blob;
+ return 'blob:thread-dump';
+ };
+ URL.revokeObjectURL = url => revokedUrls.push(url);
+ originalAnchorClick = HTMLAnchorElement.prototype.click;
+ HTMLAnchorElement.prototype.click = function () {
+ downloads.push({
+ blob: lastBlob, href: this.href, fileName: this.download, revokedAtClick: revokedUrls.length
+ });
+ };
+});
+
+afterEach(async () => {
+ // Let a pending Blob URL revoke run before the stubs are restored.
+ await new Promise(resolve => setTimeout(resolve, 200));
+ HTMLAnchorElement.prototype.click = originalAnchorClick;
+ delete URL.createObjectURL;
+ delete URL.revokeObjectURL;
+});
+
+test('Download saves the complete thread dump, including threads after a "#"', async function () {
+ document.body.innerHTML =
+ '<a class="downloadbutton" data-action="downloadThreadDump" ' +
+ 'data-filename="threaddump_1.txt">Download</a>' +
+ `<pre id="thread-dump-text" class="d-none">${escapeHtml(THREAD_DUMP)}</pre>`;
+
+ // Dispatch the click: HTMLAnchorElement.prototype.click is replaced to record downloads.
+ document.querySelector('a.downloadbutton')
+ .dispatchEvent(new MouseEvent('click', { bubbles: true, cancelable: true }));
+
+ expect(downloads.length).toBe(1);
+ const { blob, href, fileName } = downloads[0];
+ expect(fileName).toBe('threaddump_1.txt');
+ expect(href).toBe('blob:thread-dump');
+ expect(blob.type).toBe('text/plain;charset=utf-8');
+ expect(await readBlob(blob)).toBe(THREAD_DUMP);
+});
+
+test('Download releases the Blob URL after starting the download', async function () {
+ document.body.innerHTML =
+ '<a class="downloadbutton" data-action="downloadThreadDump" ' +
+ 'data-filename="threaddump_1.txt">Download</a>' +
+ `<pre id="thread-dump-text" class="d-none">${escapeHtml(THREAD_DUMP)}</pre>`;
+
+ document.querySelector('a.downloadbutton')
+ .dispatchEvent(new MouseEvent('click', { bubbles: true, cancelable: true }));
+ await new Promise(resolve => setTimeout(resolve, 200));
+
+ expect(downloads.length).toBe(1);
+ // Revoking before the click would cancel the download.
+ expect(downloads[0].revokedAtClick).toBe(0);
+ expect(revokedUrls).toEqual(['blob:thread-dump']);
+});
+
+test('the legacy data: URI link is left to the browser', function () {
+ document.body.innerHTML =
+ '<a class="downloadbutton" href="data:text/plain;charset=utf-8,dump" ' +
+ 'download="threaddump_1.txt">Download</a>';
+ const link = document.querySelector('a.downloadbutton');
+ // jsdom does not implement navigation, so cancel the link's default action.
+ link.addEventListener('click', event => event.preventDefault());
+
+ link.dispatchEvent(new MouseEvent('click', { bubbles: true, cancelable: true }));
+
+ expect(downloads.length).toBe(0);
+});
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org