You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2023/11/30 00:46:59 UTC
[PR] [SPARK-46178][CONNECT][TESTS] Move MockDF under PlanOnlyTestFixture [spark]
HyukjinKwon opened a new pull request, #44087:
URL: https://github.com/apache/spark/pull/44087
### What changes were proposed in this pull request?
This PR proposes to move `MockDF` under `PlanOnlyTestFixture` for a proper error messages when you run the tests.
### Why are the changes needed?
When you don't have the dependencies for Spark Connect, and you run the tests:
```
./python/run-tests --python-executables=python3 --testnames 'pyspark.pandas.tests.connect.test_parity_ops_on_diff_frames_groupby_rolling'
```
it shows the error as below:
```
...
File "/.../spark/python/pyspark/pandas/tests/connect/test_parity_ops_on_diff_frames_groupby_rolling.py", line 22, in <module>
from pyspark.testing.connectutils import ReusedConnectTestCase
File "/.../spark/python/pyspark/testing/connectutils.py", line 92, in <module>
class PlanOnlyTestFixture(unittest.TestCase, PySparkErrorTestUtils):
File "/.../spark/python/pyspark/testing/connectutils.py", line 93, in PlanOnlyTestFixture
class MockDF(DataFrame):
^^^^^^^^^
NameError: name 'DataFrame' is not defined
```
which isn't actionable.
After this change:
```
...
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/.../spark/python/pyspark/pandas/tests/connect/test_parity_ops_on_diff_frames_groupby_rolling.py", line 22, in <module>
from pyspark.testing.connectutils import ReusedConnectTestCase
File "/.../spark/python/pyspark/testing/connectutils.py", line 91, in <module>
class PlanOnlyTestFixture(unittest.TestCase, PySparkErrorTestUtils):
File "/.../spark/python/pyspark/testing/connectutils.py", line 92, in PlanOnlyTestFixture
from pyspark.sql.connect.dataframe import DataFrame
File "/.../spark/python/pyspark/sql/connect/dataframe.py", line 20, in <module>
check_dependencies(__name__)
File "/.../spark/python/pyspark/sql/connect/utils.py", line 36, in check_dependencies
require_minimum_grpc_version()
File "/.../spark/python/pyspark/sql/connect/utils.py", line 48, in require_minimum_grpc_version
raise ImportError(
ImportError: grpcio >= 1.48.1 must be installed; however, it was not found.
```
### Does this PR introduce _any_ user-facing change?
No, test-only.
### How was this patch tested?
Manually as described above.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46178][CONNECT][TESTS] Move MockDF under PlanOnlyTestFixture [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #44087: [SPARK-46178][CONNECT][TESTS] Move MockDF under PlanOnlyTestFixture
URL: https://github.com/apache/spark/pull/44087
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46178][CONNECT][TESTS] Move MockDF under PlanOnlyTestFixture [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #44087:
URL: https://github.com/apache/spark/pull/44087#issuecomment-1832929625
Build: https://github.com/HyukjinKwon/spark/actions/runs/7040411745
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org