You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/08/11 21:29:40 UTC
(datafusion) branch gh-readonly-queue/main/pr-24275-30ca39bef9ea2e746e143727f9de48b1608edc2a created (now 7e015b74af)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-24275-30ca39bef9ea2e746e143727f9de48b1608edc2a
in repository https://gitbox.apache.org/repos/asf/datafusion.git
at 7e015b74af fix(physical-plan): CTAS panic on wasm32-unknown-unknown (#24275)
This branch includes the following new commits:
new 7e015b74af fix(physical-plan): CTAS panic on wasm32-unknown-unknown (#24275)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion) 01/01: fix(physical-plan): CTAS panic on wasm32-unknown-unknown (#24275)
Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-24275-30ca39bef9ea2e746e143727f9de48b1608edc2a
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit 7e015b74af0928187a77cdf9df23b587e349ebf9
Author: Kent Wu <ke...@gmail.com>
AuthorDate: Tue Aug 11 17:29:17 2026 -0400
fix(physical-plan): CTAS panic on wasm32-unknown-unknown (#24275)
## Which issue does this PR close?
- Closes #24274
## Rationale for this change
`CREATE TABLE ... AS SELECT` and `MemTable::load` panic on
`wasm32-unknown-unknown` in browser hosts using `wasm-bindgen-futures`.
Both route through `collect_partitioned`, which unconditionally calls
`JoinSet::spawn` which needs an active tokio reactor.
`wasm32-unknown-unknown` builds typically don't have one.
## What changes are included in this PR?
- Adds a single-partition fast path to `collect_partitioned` that drains
the stream directly instead of using `JoinSet::spawn`
- Adds a `test_create_table_as_select` regression test to
`datafusion/wasmtest`, exercising the CTAS path both natively (via
`tokio::test`) and in the browser (via `wasm-pack test`).
Behavior for `partition_count != 1` is unchanged.
## Are these changes tested?
Yes
- Adds a CTAS query-level regression test to the `linux-wasm-pack` CI
job.
- Ran wasm-pack test suite on chrome:
- `RUSTFLAGS='--cfg getrandom_backend="wasm_js"' wasm-pack test
--headless --chrome datafusion/wasmtest`
- `cargo fmt --all`
- `cargo clippy --all-targets --all-features -- -D warnings`
- Full workspace extended-tests suite
## Are there any user-facing changes?
No API changes. On `wasm32-unknown-unknown`, `CREATE TABLE ... AS
SELECT` and `MemTable::load` no longer panic.
---
datafusion/physical-plan/src/execution_plan.rs | 7 +++++++
datafusion/wasmtest/src/lib.rs | 28 ++++++++++++++++++++++++++
2 files changed, 35 insertions(+)
diff --git a/datafusion/physical-plan/src/execution_plan.rs b/datafusion/physical-plan/src/execution_plan.rs
index b1d7c32882..ff803746aa 100644
--- a/datafusion/physical-plan/src/execution_plan.rs
+++ b/datafusion/physical-plan/src/execution_plan.rs
@@ -1655,6 +1655,13 @@ pub async fn collect_partitioned(
plan: Arc<dyn ExecutionPlan>,
context: Arc<TaskContext>,
) -> Result<Vec<Vec<RecordBatch>>> {
+ // Avoid `JoinSet::spawn` for single partition
+ if plan.output_partitioning().partition_count() == 1 {
+ let stream = plan.execute(0, context)?;
+ let batches: Vec<RecordBatch> = stream.try_collect().await?;
+ return Ok(vec![batches]);
+ }
+
let streams = execute_stream_partitioned(plan, context)?;
let mut join_set = JoinSet::new();
diff --git a/datafusion/wasmtest/src/lib.rs b/datafusion/wasmtest/src/lib.rs
index f545ccf193..d8da5d4b4f 100644
--- a/datafusion/wasmtest/src/lib.rs
+++ b/datafusion/wasmtest/src/lib.rs
@@ -206,6 +206,34 @@ mod test {
let _ = collect(physical_plan, task_ctx).await.unwrap();
}
+ #[wasm_bindgen_test(unsupported = tokio::test)]
+ async fn test_create_table_as_select() {
+ let ctx = get_ctx();
+ ctx.sql("CREATE TABLE t AS SELECT 1 AS a, 'x' AS b")
+ .await
+ .unwrap()
+ .collect()
+ .await
+ .unwrap();
+
+ let result = ctx
+ .sql("SELECT * FROM t")
+ .await
+ .unwrap()
+ .collect()
+ .await
+ .unwrap();
+
+ assert_eq!(
+ batches_to_string(&result),
+ "+---+---+\n\
+ | a | b |\n\
+ +---+---+\n\
+ | 1 | x |\n\
+ +---+---+"
+ );
+ }
+
#[wasm_bindgen_test(unsupported = tokio::test)]
async fn test_parquet_write() {
let (schema, batch) = create_test_data();
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org