[GitHub] [spark] Yikun commented on a diff in pull request #36356: [SPARK-38821][PYTHON] Skip nsmall/nlarge nan test under pandas 1.4.[0,1,2]

[GitBox <gi...@apache.org>]

Yikun commented on code in PR #36356:
URL: https://github.com/apache/spark/pull/36356#discussion_r858401383


##########
python/pyspark/pandas/tests/test_dataframe.py:
##########
@@ -1814,8 +1814,12 @@ def test_nlargest(self):
             index=np.random.rand(7),
         )
         psdf = ps.from_pandas(pdf)
-        self.assert_eq(psdf.nlargest(5, columns="a"), pdf.nlargest(5, columns="a"))
-        self.assert_eq(psdf.nlargest(5, columns=["a", "b"]), pdf.nlargest(5, columns=["a", "b"]))
+        # see also: https://github.com/pandas-dev/pandas/issues/46589
+        if not (LooseVersion("1.4.0") <= LooseVersion(pd.__version__) <= LooseVersion("1.4.2")):
+            self.assert_eq(psdf.nlargest(5, columns="a"), pdf.nlargest(5, columns="a"))
+            self.assert_eq(
+                psdf.nlargest(5, columns=["a", "b"]), pdf.nlargest(5, columns=["a", "b"])
+            )

Review Comment:
   
   If you still think I need to compare with real results rather than skip, I'd also like to change. We need to change `index=np.random.rand(7)` to a certain range, and construct a result df.
   
   Because this is only failed with panda 1.4.0~1.4.2, so I thought skip is enough.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org