You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by yi...@apache.org on 2022/12/09 14:13:33 UTC
[spark] branch branch-3.3 updated: [SPARK-40270][PS][FOLLOWUP][3.3] Skip test_style when pandas <1.3.0
This is an automated email from the ASF dual-hosted git repository.
yikun pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new b6c6526e3b1 [SPARK-40270][PS][FOLLOWUP][3.3] Skip test_style when pandas <1.3.0
b6c6526e3b1 is described below
commit b6c6526e3b1c5bd32b010a38cb0f4faeba678e22
Author: Yikun Jiang <yi...@gmail.com>
AuthorDate: Fri Dec 9 22:13:09 2022 +0800
[SPARK-40270][PS][FOLLOWUP][3.3] Skip test_style when pandas <1.3.0
### What changes were proposed in this pull request?
According to https://pandas.pydata.org/docs/reference/api/pandas.io.formats.style.Styler.to_latex.html:
`pandas.io.formats.style.Styler.to_latex` introduced since 1.3.0, so before panda 1.3.0, should skip the check
```
ERROR [0.180s]: test_style (pyspark.pandas.tests.test_dataframe.DataFrameTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/__w/spark/spark/python/pyspark/pandas/tests/test_dataframe.py", line 5795, in test_style
check_style()
File "/__w/spark/spark/python/pyspark/pandas/tests/test_dataframe.py", line 5793, in check_style
self.assert_eq(pdf_style.to_latex(), psdf_style.to_latex())
AttributeError: 'Styler' object has no attribute 'to_latex'
```
Related: https://github.com/apache/spark/commit/58375a86e6ff49c5bcee49939fbd98eb848ae59f
### Why are the changes needed?
This test break the 3.2 branch pyspark test (with python 3.6 + pandas 1.1.x), so I think better add the `skipIf` it.
See also https://github.com/apache/spark/pull/38982#issuecomment-1343923114
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- CI passed
Closes #39007 from Yikun/branch-3.3-check.
Authored-by: Yikun Jiang <yi...@gmail.com>
Signed-off-by: Yikun Jiang <yi...@gmail.com>
---
python/pyspark/pandas/tests/test_dataframe.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/python/pyspark/pandas/tests/test_dataframe.py b/python/pyspark/pandas/tests/test_dataframe.py
index 0a7eda77564..0c23bf07a69 100644
--- a/python/pyspark/pandas/tests/test_dataframe.py
+++ b/python/pyspark/pandas/tests/test_dataframe.py
@@ -6375,6 +6375,10 @@ class DataFrameTest(ComparisonTestBase, SQLTestUtils):
psdf = ps.from_pandas(pdf)
self.assert_eq(pdf.cov(), psdf.cov())
+ @unittest.skipIf(
+ LooseVersion(pd.__version__) < LooseVersion("1.3.0"),
+ "pandas support `Styler.to_latex` since 1.3.0",
+ )
def test_style(self):
# Currently, the `style` function returns a pandas object `Styler` as it is,
# processing only the number of rows declared in `compute.max_rows`.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org