You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2021/11/14 20:34:53 UTC
[spark] branch branch-3.2 updated: [SPARK-37322][TESTS] `run_scala_tests` should respect test module order
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new 21d901b [SPARK-37322][TESTS] `run_scala_tests` should respect test module order
21d901b is described below
commit 21d901bb362ac735bed34a0b9731a5d2c6aa20ae
Author: William Hyun <wi...@apache.org>
AuthorDate: Sun Nov 14 12:32:36 2021 -0800
[SPARK-37322][TESTS] `run_scala_tests` should respect test module order
### What changes were proposed in this pull request?
This PR aims to make `run_scala_tests` respect test module order
### Why are the changes needed?
Currently the execution order is random.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Manually through the following and check if the catalyst module runs first.
```
$ SKIP_MIMA=1 SKIP_UNIDOC=1 ./dev/run-tests --parallelism 1 --modules "catalyst,hive-thriftserver"
```
Closes #34590 from williamhyun/order.
Authored-by: William Hyun <wi...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit c2221a8e3d95ce22d76208c705179c5954318567)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
dev/run-tests.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/dev/run-tests.py b/dev/run-tests.py
index c54042c..545905b 100755
--- a/dev/run-tests.py
+++ b/dev/run-tests.py
@@ -469,7 +469,8 @@ def run_scala_tests(build_tool, extra_profiles, test_modules, excluded_tags, inc
`determine_test_suites` function"""
set_title_and_block("Running Spark unit tests", "BLOCK_SPARK_UNIT_TESTS")
- test_modules = set(test_modules)
+ # Remove duplicates while keeping the test module order
+ test_modules = list(dict.fromkeys(test_modules))
test_profiles = extra_profiles + \
list(set(itertools.chain.from_iterable(m.build_profile_flags for m in test_modules)))
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org