You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/09/30 20:50:00 UTC
[jira] [Updated] (SPARK-59906) Run relevant tests when JVM-launched Python worker scripts change under smart test selection
[ https://issues.apache.org/jira/browse/SPARK-59906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated SPARK-59906:
-----------------------------------
Labels: pull-request-available (was: )
> Run relevant tests when JVM-launched Python worker scripts change under smart test selection
> --------------------------------------------------------------------------------------------
>
> Key: SPARK-59906
> URL: https://issues.apache.org/jira/browse/SPARK-59906
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 5.0.0
> Reporter: Tian Gao
> Priority: Major
> Labels: pull-request-available
>
> PySpark smart test selection (PySparkBaseTestCase, driven by PYSPARK_CHANGED_FILES) keeps a test only when the changed file's module is reachable from the test module in the static import graph built by grimp. Scripts that the JVM launches as separate processes (pyspark.worker, pyspark.daemon, pyspark.worker_util, and the pyspark.sql.worker.* runners) are never imported by any test module, so a change touching only those files is treated as irrelevant to every test and all of them are skipped -- including the UDF, pandas, Arrow, UDTF, and Python data source tests that actually exercise the changed code. This maps each such entry-point script to the API module whose use triggers it (the worker/daemon files are treated as relevant to every test), so the corresponding tests are selected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org