You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2021/10/20 03:14:40 UTC
[spark] branch master updated: [SPARK-36348][TEST] Complete
test_astype for index
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b07dd1a [SPARK-36348][TEST] Complete test_astype for index
b07dd1a is described below
commit b07dd1aacab9cf2df4ba3e88842a408e3c5c26a8
Author: Yikun Jiang <yi...@gmail.com>
AuthorDate: Wed Oct 20 12:14:06 2021 +0900
[SPARK-36348][TEST] Complete test_astype for index
### What changes were proposed in this pull request?
Before 3.2, there was a bug:
```
pidx = pd.Index([10, 20, 15, 30, 45, None], name="x")
psidx = ps.Index(pidx)
self.assert_eq(psidx.astype(str), pidx.astype(str))
[left pandas on spark]: Index(['10.0', '20.0', '15.0', '30.0', '45.0', 'nan'], dtype='object', name='x')
[right pandas]: Index(['10', '20', '15', '30', '45', 'None'], dtype='object', name='x')
```
So, we didn't add any test on [test_base.py int_with_nan]https://github.com/apache/spark/blob/bcc595c112a23d8e3024ace50f0dbc7eab7144b2/python/pyspark/pandas/tests/indexes/test_base.py#L2249
Now, the bug had been resolved, we complete the testcase in here.
### Why are the changes needed?
regression for SPARK-36348 and complete testcase.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Test only
Closes #34335 from Yikun/SPARK-36348.
Authored-by: Yikun Jiang <yi...@gmail.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
python/pyspark/pandas/tests/indexes/test_base.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/python/pyspark/pandas/tests/indexes/test_base.py b/python/pyspark/pandas/tests/indexes/test_base.py
index 4003998..a7f19a7 100644
--- a/python/pyspark/pandas/tests/indexes/test_base.py
+++ b/python/pyspark/pandas/tests/indexes/test_base.py
@@ -2243,12 +2243,14 @@ class IndexesTest(PandasOnSparkTestCase, TestUtils):
pidx = pd.Index([10, 20, 15, 30, 45, None], name="x")
psidx = ps.Index(pidx)
+ self.assert_eq(psidx.astype(bool), pidx.astype(bool))
+ self.assert_eq(psidx.astype(str), pidx.astype(str))
pidx = pd.Index(["hi", "hi ", " ", " \t", "", None], name="x")
psidx = ps.Index(pidx)
self.assert_eq(psidx.astype(bool), pidx.astype(bool))
- self.assert_eq(psidx.astype(str).to_numpy(), ["hi", "hi ", " ", " \t", "", "None"])
+ self.assert_eq(psidx.astype(str), pidx.astype(str))
pidx = pd.Index([True, False, None], name="x")
psidx = ps.Index(pidx)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org