Re: [PR] [WIP][INFRA] Add smart class-level test selection for post-merge CI [spark]

["zhengruifeng (via GitHub)" <gi...@apache.org>]

zhengruifeng commented on PR #57461:
URL: https://github.com/apache/spark/pull/57461#issuecomment-5087027472

   The head intentionally includes a few small non-infrastructure changes to exercise smart test selection end to end. The expected candidates are:
   
   - `SparkConf.getAllWithPrefix` has a behavior-preserving Scala implementation refactor. `SparkConfSuite` directly calls this API, so `JVM: org.apache.spark.SparkConfSuite` is the expected JVM candidate.
   - `pyspark.sql.classic.DataFrame.sort` has a behavior-preserving Python source change. `pyspark.sql.tests.test_dataframe` contains direct `DataFrame.sort` call sites, so it is an expected Python candidate.
   - `DataFrameTestsMixin` gains a no-op test. The selector should select both the direct module, `pyspark.sql.tests.test_dataframe`, and its Spark Connect consumer, `pyspark.sql.tests.connect.test_parity_dataframe`, which imports and subclasses that mixin.
   - The `SparkThrowableSuite` change is ScalaDoc-only, and the `SQLTestsMixin` change is a test-file docstring-only hunk. Neither hunk should select a target by itself.
   
   The workflow does not hard-code these mappings. Copilot ranks candidates from the pushed-tip change, then `dev/smart_test_selection.py` accepts only exact catalog entries. If targets remain after validation, `precompile` runs once and the JVM and Python jobs run in parallel; each accepted suite is executed with SBT `testOnly` and each accepted Python module with `python/run-tests --testnames`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org