You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zsxwing (via GitHub)" <gi...@apache.org> on 2026/10/07 05:06:36 UTC
[PR] [SPARK-60019][UI] Build the executor thread dump download in the browser [spark]
zsxwing opened a new pull request, #59265:
URL: https://github.com/apache/spark/pull/59265
### What changes were proposed in this pull request?
The Download button on the executor thread dump page inlines the whole dump into a `data:` URI `href`:
```scala
href={"data:text/plain;charset=utf-8," + threadDump.map(_.toString).mkString}
```
This PR stops putting the dump in a URL. Like the SQL plan Download button (`downloadPlanBlob` in `spark-sql-viz.js`), the button now builds the file in the browser:
- `ExecutorThreadDumpPage` renders the Download link without an `href` (`data-action="downloadThreadDump"`, `data-filename=...`) and embeds the dump, unchanged, in a hidden `<pre id="thread-dump-text" class="d-none">`.
- `table.js` gets a `downloadThreadDump` function that builds a `Blob` from that element and downloads it, wired through the same delegated `data-action` listeners as Expand All / Collapse All.
A new internal config, `spark.ui.threadDump.downloadInBrowser.enabled` (default `true`), is a kill switch. When it is `false`, the page renders the original `data:` URI link.
### Why are the changes needed?
When a browser materializes a `data:` URI download, it excludes the URL fragment (everything from the first `#`) before decoding the body. See step 2 of the WHATWG "data: URL processor": https://url.spec.whatwg.org/#data-url-processor. Chromium (`GURL::GetContentPiece`) and Firefox (`nsDataHandler::ParseURI`) both implement this.
Since https://github.com/apache/spark/pull/53634 (SPARK-54861), idle task threads are named `Executor task idle worker#<threadId>`, so nearly every executor thread dump contains a `#`. The downloaded file is silently truncated at the first such thread, losing it and every thread after it.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
No automated test: the download is a browser-side `Blob` click, which the HtmlUnit driver used by `UISeleniumSuite` cannot run, and `ChromeUISeleniumSuite` is `@Ignore`d.
Manual test:
1. Start `bin/spark-shell --master 'local-cluster[2,1,1024]'`.
2. Run `spark.range(10).repartition(20).show()` so the executors have idle task threads (`Executor task idle worker#N`).
3. Open the Executors tab, click Thread Dump for an executor, then click Download.
4. The saved `threaddump_<executorId>.txt` contains every thread on the page, including the `Executor task idle worker#N` threads and their stack traces.
5. Set `spark.ui.threadDump.downloadInBrowser.enabled=false` and repeat: the old `data:` link is rendered.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60019][UI] Build the executor thread dump download in the browser [spark]
Posted by "zsxwing (via GitHub)" <gi...@apache.org>.
zsxwing closed pull request #59265: [SPARK-60019][UI] Build the executor thread dump download in the browser
URL: https://github.com/apache/spark/pull/59265
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60019][UI] Build the executor thread dump download in the browser [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #59265:
URL: https://github.com/apache/spark/pull/59265#discussion_r4225634009
##########
core/src/main/resources/org/apache/spark/ui/static/table.js:
##########
@@ -63,6 +63,16 @@ function expandAllThreadStackTrace(toggleButton) {
}
/* eslint-enable no-unused-vars */
+// Builds the file in the browser (as the SQL plan download does) rather than linking to a data:
+// URI, which browsers truncate at the first '#'.
+function downloadThreadDump(fileName) {
Review Comment:
**Nit (P3):** Nothing in `core/src/test` or `ui-test/tests` exercises this branch or the `downloadThreadDump` helper. If it regressed, the silent truncation this PR fixes could come back without CI noticing. Examples: this condition gets inverted, `thread-dump-text` is renamed on only one side, or the helper reads a truncated source.
The PR notes that HtmlUnit can't run the Blob click, but neither half needs a real download:
- **Helper:** `ui-test/tests` is a jest/jsdom suite that runs in CI. It already loads classic static scripts, as `spark-sql-viz.test.js` does, so a test can load `table.js`, stub `URL.createObjectURL` to capture the Blob, and check it against a `<pre id="thread-dump-text">` that contains a `...worker#N` thread followed by more threads.
- **Rendering:** `UISeleniumSuite` (through `goToUi`), or a direct `render()` call as in `DriverLogPageSuite`, can check the markup each config value produces.
**Verification:**
- **Regression:** Calling the download helper on embedded text that contains 'worker#N' followed by more threads yields a Blob whose text equals the complete embedded text and uses the requested file name.
- **Behavior:** Rendering the executor thread dump page with the default config produces the hidden dump element holding every thread and a Download control with no data: URI.
- **Compatibility:** Rendering the page with the kill switch disabled reproduces the legacy data: URI Download link.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60019][UI] Build the executor thread dump download in the browser [spark]
Posted by "zsxwing (via GitHub)" <gi...@apache.org>.
zsxwing commented on PR #59265:
URL: https://github.com/apache/spark/pull/59265#issuecomment-6138277324
> What weighs the other way: Expand All / Collapse All on the same page are already href-less anchors, and I found no stated Spark UI accessibility policy.
Yep, I'd prefer to stay consistent with the existing links on this page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org