You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2026/08/25 05:07:00 UTC
[jira] [Updated] (SPARK-58983) [spark-connect-rust] Fix CI so all workflows run and pass
[ https://issues.apache.org/jira/browse/SPARK-58983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-58983:
---------------------------------
Description:
All GitHub Actions workflows on apache/spark-connect-rust failed with "startup_failure"
("This run likely failed because of a workflow file issue") on pushes to master and on
pull requests, so there was effectively no CI. Fixing that surfaced several genuine
build/test failures that the default build never exercised. This issue tracks getting CI
to both run *and* pass.
h2. Root cause of startup_failure
The ASF org restricts GitHub Actions to actions/* plus a vetted allowlist; the workflows
referenced non-allowlisted third-party actions (actions-rust-lang/setup-rust-toolchain,
dtolnay/rust-toolchain, arduino/setup-protoc, PyO3/maturin-action,
pypa/gh-action-pypi-publish), so the runs failed to start.
h2. Fixes
* Replace all non-allowlisted actions with GitHub-authored actions (actions/checkout,
actions/setup-python, actions/setup-java, actions/cache, actions/upload-artifact,
actions/download-artifact) plus shell steps (rustup for the toolchain, apt/brew/choco
for protoc, pip for maturin/twine).
* Fix a pre-existing crate-name bug in build_python_connect.yml's rust-checks job, which
ran {{cargo test -p spark-connect -p spark-connect-core}} using the old crate names
(renamed to apache-spark-connect / apache-spark-connect-core).
* Fix a wasm-udf compile error: a unit test named parse_python_version_output shadowed the
module function of the same name (via {{use super::*}}); rename the test.
* Run the WASM packer against the reference pyspark: isolate pyspark_wasm_udf on the packer
import path so {{pyspark.cloudpickle}} and {{pyspark.sql.types._parse_datatype_json_value}}
resolve to the reference pyspark. The Rust-backed pyspark skin lacks them and cannot emit
an executor-compatible DataType.
* Bump the transitive dependency ethnum 1.5.0 -> 1.5.3: 1.5.0 does
{{mem::transmute::<(), TryFromIntError>(())}}, which no longer compiles now that std's
TryFromIntError is not zero-sized (E0512).
* Move the example crates and datasets under examples/ and update the workflow paths.
was:
All GitHub Actions workflows on apache/spark-connect-rust fail with "startup_failure"
("This run likely failed because of a workflow file issue") on pushes to master and on
pull requests, so there is effectively no CI.
Root cause: the workflows use third-party GitHub Actions that are not on the ASF Actions
allowlist (the ASF org restricts Actions to actions/* plus a vetted allowlist), so the
runs fail to start:
- actions-rust-lang/setup-rust-toolchain (rust.yml, lint.yml, build_python_connect.yml)
- dtolnay/rust-toolchain, arduino/setup-protoc, PyO3/maturin-action,
pypa/gh-action-pypi-publish (release.yml)
Fix: replace all non-allowlisted actions with GitHub-authored actions (actions/checkout,
actions/setup-python, actions/setup-java, actions/cache, actions/upload-artifact,
actions/download-artifact) plus shell steps (rustup for the toolchain, apt/brew/choco for
protoc, pip for maturin/twine). Also fix a pre-existing bug in build_python_connect.yml's
rust-checks job, which ran `cargo test -p spark-connect -p spark-connect-core` using the
old crate names (renamed to apache-spark-connect / apache-spark-connect-core).
Summary: [spark-connect-rust] Fix CI so all workflows run and pass (was: [spark-connect-rust] CI workflows fail to start: replace non-allowlisted GitHub Actions)
> [spark-connect-rust] Fix CI so all workflows run and pass
> ---------------------------------------------------------
>
> Key: SPARK-58983
> URL: https://issues.apache.org/jira/browse/SPARK-58983
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
> Affects Versions: 5.0.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
>
> All GitHub Actions workflows on apache/spark-connect-rust failed with "startup_failure"
> ("This run likely failed because of a workflow file issue") on pushes to master and on
> pull requests, so there was effectively no CI. Fixing that surfaced several genuine
> build/test failures that the default build never exercised. This issue tracks getting CI
> to both run *and* pass.
> h2. Root cause of startup_failure
> The ASF org restricts GitHub Actions to actions/* plus a vetted allowlist; the workflows
> referenced non-allowlisted third-party actions (actions-rust-lang/setup-rust-toolchain,
> dtolnay/rust-toolchain, arduino/setup-protoc, PyO3/maturin-action,
> pypa/gh-action-pypi-publish), so the runs failed to start.
> h2. Fixes
> * Replace all non-allowlisted actions with GitHub-authored actions (actions/checkout,
> actions/setup-python, actions/setup-java, actions/cache, actions/upload-artifact,
> actions/download-artifact) plus shell steps (rustup for the toolchain, apt/brew/choco
> for protoc, pip for maturin/twine).
> * Fix a pre-existing crate-name bug in build_python_connect.yml's rust-checks job, which
> ran {{cargo test -p spark-connect -p spark-connect-core}} using the old crate names
> (renamed to apache-spark-connect / apache-spark-connect-core).
> * Fix a wasm-udf compile error: a unit test named parse_python_version_output shadowed the
> module function of the same name (via {{use super::*}}); rename the test.
> * Run the WASM packer against the reference pyspark: isolate pyspark_wasm_udf on the packer
> import path so {{pyspark.cloudpickle}} and {{pyspark.sql.types._parse_datatype_json_value}}
> resolve to the reference pyspark. The Rust-backed pyspark skin lacks them and cannot emit
> an executor-compatible DataType.
> * Bump the transitive dependency ethnum 1.5.0 -> 1.5.3: 1.5.0 does
> {{mem::transmute::<(), TryFromIntError>(())}}, which no longer compiles now that std's
> TryFromIntError is not zero-sized (E0512).
> * Move the example crates and datasets under examples/ and update the workflow paths.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org