You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "asl3 (via GitHub)" <gi...@apache.org> on 2023/07/16 19:44:57 UTC
[GitHub] [spark] asl3 opened a new pull request, #42023: [SPARK-44446][PYTHON] Add checks for expected list type special cases
asl3 opened a new pull request, #42023:
URL: https://github.com/apache/spark/pull/42023
### What changes were proposed in this pull request?
This PR adds handling for special cases when `expected` is type list.
### Why are the changes needed?
The change is needed to handle all cases for when `expected` is type list.
### Does this PR introduce _any_ user-facing change?
Yes, the PR makes modifications to the user-facing function `assertDataFrameEqual`
### How was this patch tested?
Added tests to `runtime/python/pyspark/sql/tests/test_utils.py` and `runtime/python/pyspark/sql/tests/connect/test_utils.py`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xinrong-meng closed pull request #42023: [SPARK-44446][PYTHON] Add checks for expected list type special cases
Posted by "xinrong-meng (via GitHub)" <gi...@apache.org>.
xinrong-meng closed pull request #42023: [SPARK-44446][PYTHON] Add checks for expected list type special cases
URL: https://github.com/apache/spark/pull/42023
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ueshin commented on a diff in pull request #42023: [SPARK-44446][PYTHON] Add checks for expected list type special cases
Posted by "ueshin (via GitHub)" <gi...@apache.org>.
ueshin commented on code in PR #42023:
URL: https://github.com/apache/spark/pull/42023#discussion_r1278081871
##########
python/pyspark/sql/tests/test_utils.py:
##########
@@ -1119,6 +1119,30 @@ def test_empty_no_column(self):
assertDataFrameEqual(df1, df2, checkRowOrder=False)
assertDataFrameEqual(df1, df2, checkRowOrder=True)
+ def test_empty_expected_list(self):
+ df1 = self.spark.range(0, 10).drop("id")
+
+ df2 = []
Review Comment:
This should be:
```py
df2 = [Row(), Row(), ... Row()] # 10 `Row()`s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xinrong-meng commented on pull request #42023: [SPARK-44446][PYTHON] Add checks for expected list type special cases
Posted by "xinrong-meng (via GitHub)" <gi...@apache.org>.
xinrong-meng commented on PR #42023:
URL: https://github.com/apache/spark/pull/42023#issuecomment-1638681917
Merged to master, thank!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] asl3 commented on a diff in pull request #42023: [SPARK-44446][PYTHON] Add checks for expected list type special cases
Posted by "asl3 (via GitHub)" <gi...@apache.org>.
asl3 commented on code in PR #42023:
URL: https://github.com/apache/spark/pull/42023#discussion_r1278085441
##########
python/pyspark/sql/tests/test_utils.py:
##########
@@ -1119,6 +1119,30 @@ def test_empty_no_column(self):
assertDataFrameEqual(df1, df2, checkRowOrder=False)
assertDataFrameEqual(df1, df2, checkRowOrder=True)
+ def test_empty_expected_list(self):
+ df1 = self.spark.range(0, 10).drop("id")
+
+ df2 = []
Review Comment:
thanks, fixing in [#42196](https://github.com/apache/spark/pull/42196)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org