You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2023/08/07 11:46:50 UTC
[spark] branch master updated: [SPARK-44701][PYTHON][TESTS] Skip `ClassificationTestsOnConnect` when `torch` is not installed
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 433fb2af8a3 [SPARK-44701][PYTHON][TESTS] Skip `ClassificationTestsOnConnect` when `torch` is not installed
433fb2af8a3 is described below
commit 433fb2af8a3ca239958cb7b006e2924ecfac0d56
Author: Ruifeng Zheng <ru...@apache.org>
AuthorDate: Mon Aug 7 20:46:38 2023 +0900
[SPARK-44701][PYTHON][TESTS] Skip `ClassificationTestsOnConnect` when `torch` is not installed
### What changes were proposed in this pull request?
Skip `ClassificationTestsOnConnect` when `torch` is not installed
### Why are the changes needed?
we moved torch on connect tests to `pyspark_ml_connect`, so module `pyspark_connect` won't have `torch`
to fix https://github.com/apache/spark/actions/runs/5776211318/job/15655104006 in 3.5 daily GA:
```
Starting test(python3.9): pyspark.ml.tests.connect.test_connect_classification (temp output: /__w/spark/spark/python/target/fbb6a495-df65-4334-8c04-4befc9ee81df/python3.9__pyspark.ml.tests.connect.test_connect_classification__jp1htw6f.log)
Traceback (most recent call last):
File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/__w/spark/spark/python/pyspark/ml/tests/connect/test_connect_classification.py", line 21, in <module>
from pyspark.ml.tests.connect.test_legacy_mode_classification import ClassificationTestsMixin
File "/__w/spark/spark/python/pyspark/ml/tests/connect/test_legacy_mode_classification.py", line 22, in <module>
from pyspark.ml.connect.classification import (
File "/__w/spark/spark/python/pyspark/ml/connect/classification.py", line 46, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
```
### Does this PR introduce _any_ user-facing change?
no, test-only
### How was this patch tested?
CI
Closes #42375 from zhengruifeng/torch_skip.
Authored-by: Ruifeng Zheng <ru...@apache.org>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
python/pyspark/ml/tests/connect/test_connect_classification.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/python/pyspark/ml/tests/connect/test_connect_classification.py b/python/pyspark/ml/tests/connect/test_connect_classification.py
index 6ad47322234..f3e621c19f0 100644
--- a/python/pyspark/ml/tests/connect/test_connect_classification.py
+++ b/python/pyspark/ml/tests/connect/test_connect_classification.py
@@ -20,7 +20,14 @@ import unittest
from pyspark.sql import SparkSession
from pyspark.ml.tests.connect.test_legacy_mode_classification import ClassificationTestsMixin
+have_torch = True
+try:
+ import torch # noqa: F401
+except ImportError:
+ have_torch = False
+
+@unittest.skipIf(not have_torch, "torch is required")
class ClassificationTestsOnConnect(ClassificationTestsMixin, unittest.TestCase):
def setUp(self) -> None:
self.spark = (
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org