(spark) branch master updated: [SPARK-46571][PS][TEST] Re-enable TODOs that are resolved from recent Pandas

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new b74b1592c9ec [SPARK-46571][PS][TEST] Re-enable TODOs that are resolved from recent Pandas
b74b1592c9ec is described below

commit b74b1592c9ec07b3d29b6d4d900b1d3ba1417cd1
Author: Haejoon Lee <ha...@databricks.com>
AuthorDate: Wed Jan 3 13:15:43 2024 +0900

    [SPARK-46571][PS][TEST] Re-enable TODOs that are resolved from recent Pandas
    
    ### What changes were proposed in this pull request?
    
    This PR proposes to re-enable some tests that we can run our current build.
    
    ### Why are the changes needed?
    
    Because the previous bugs are resolved from recent Pandas, so we can re-enable the tests.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No, it's test-only.
    
    ### How was this patch tested?
    
    Manually tests from local build.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #44568 from itholic/SPARK-46571.
    
    Authored-by: Haejoon Lee <ha...@databricks.com>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 .../pyspark/pandas/tests/diff_frames_ops/test_groupby_rolling_adv.py   | 1 -
 python/pyspark/pandas/tests/groupby/test_stat_func.py                  | 3 +--
 python/pyspark/pandas/tests/test_namespace.py                          | 2 +-
 3 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/python/pyspark/pandas/tests/diff_frames_ops/test_groupby_rolling_adv.py b/python/pyspark/pandas/tests/diff_frames_ops/test_groupby_rolling_adv.py
index 48d75fbcaf89..2f8b150e0a3e 100644
--- a/python/pyspark/pandas/tests/diff_frames_ops/test_groupby_rolling_adv.py
+++ b/python/pyspark/pandas/tests/diff_frames_ops/test_groupby_rolling_adv.py
@@ -33,7 +33,6 @@ class GroupByRollingAdvMixin(GroupByRollingTestingFuncMixin):
         super().tearDownClass()
 
     def test_groupby_rolling_std(self):
-        # TODO: `std` now raise error in pandas 1.0.0
         self._test_groupby_rolling_func("std")
 
     def test_groupby_rolling_var(self):
diff --git a/python/pyspark/pandas/tests/groupby/test_stat_func.py b/python/pyspark/pandas/tests/groupby/test_stat_func.py
index 257394b59f51..4dc55a7beefc 100644
--- a/python/pyspark/pandas/tests/groupby/test_stat_func.py
+++ b/python/pyspark/pandas/tests/groupby/test_stat_func.py
@@ -91,8 +91,7 @@ class FuncTestsMixin(GroupbyStatTestingFuncMixin):
             check_exact=False,
         )
 
-        # TODO: fix bug of `sum` and re-enable the test below
-        # self._test_stat_func(lambda groupby_obj: groupby_obj.sum(), check_exact=False)
+        self._test_stat_func(lambda groupby_obj: groupby_obj.sum(), check_exact=False)
         self.assert_eq(
             psdf.groupby("A").sum().sort_index(),
             pdf.groupby("A").sum().sort_index(),
diff --git a/python/pyspark/pandas/tests/test_namespace.py b/python/pyspark/pandas/tests/test_namespace.py
index 8c91f0b0d90a..a78fedd3ece4 100644
--- a/python/pyspark/pandas/tests/test_namespace.py
+++ b/python/pyspark/pandas/tests/test_namespace.py
@@ -393,7 +393,7 @@ class NamespaceTestsMixin:
         psdf3 = psdf.copy()
 
         columns = pd.MultiIndex.from_tuples([("X", "A"), ("X", "B"), ("Y", "C")])
-        # TODO: colums.names = ["XYZ", "ABC"]
+        columns.names = ["XYZ", "ABC"]
         pdf3.columns = columns
         psdf3.columns = columns
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org