You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/09/29 05:50:16 UTC
[GitHub] [spark] zhengruifeng opened a new pull request, #38042: [SPARK-40606][PS][TEST] Eliminate `to_pandas` warnings in test
zhengruifeng opened a new pull request, #38042:
URL: https://github.com/apache/spark/pull/38042
### What changes were proposed in this pull request?
Eliminate `to_pandas` warnings in test, by changing `to_pandas()` to `_to_pandas()`
### Why are the changes needed?
when a test containing `to_pandas` failed, it may print tons of warnings, for example, in https://github.com/zhengruifeng/spark/actions/runs/3142284988/jobs/5106178199
it print following warning from line 1243 to line 3985:
<img width="1170" alt="image" src="https://user-images.githubusercontent.com/7322292/192949507-ae1d3677-6ba4-4d80-84b1-44884fb1988b.png">
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
updated tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #38042: [SPARK-40606][PS][TEST] Eliminate `to_pandas` warnings in test
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on PR #38042:
URL: https://github.com/apache/spark/pull/38042#issuecomment-1263128738
Merged into master, thanks all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng closed pull request #38042: [SPARK-40606][PS][TEST] Eliminate `to_pandas` warnings in test
Posted by GitBox <gi...@apache.org>.
zhengruifeng closed pull request #38042: [SPARK-40606][PS][TEST] Eliminate `to_pandas` warnings in test
URL: https://github.com/apache/spark/pull/38042
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a diff in pull request #38042: [SPARK-40606][PS][TEST] Eliminate `to_pandas` warnings in test
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on code in PR #38042:
URL: https://github.com/apache/spark/pull/38042#discussion_r984143444
##########
python/pyspark/pandas/tests/data_type_ops/test_binary_ops.py:
##########
@@ -152,7 +152,7 @@ def test_from_to_pandas(self):
data = [b"1", b"2", b"3"]
pser = pd.Series(data)
psser = ps.Series(data)
- self.assert_eq(pser, psser.to_pandas())
+ self.assert_eq(pser, psser._to_pandas())
Review Comment:
the only difference between `to_pandas()` and `_to_pandas()` is the [warning](https://github.com/apache/spark/blob/7e5a87f0128a7679490b89294ac75e8af0856d1f/python/pyspark/pandas/frame.py#L5406-L5409).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a diff in pull request #38042: [SPARK-40606][PS][TEST] Eliminate `to_pandas` warnings in test
Posted by GitBox <gi...@apache.org>.
srowen commented on code in PR #38042:
URL: https://github.com/apache/spark/pull/38042#discussion_r983901486
##########
python/pyspark/pandas/tests/data_type_ops/test_binary_ops.py:
##########
@@ -152,7 +152,7 @@ def test_from_to_pandas(self):
data = [b"1", b"2", b"3"]
pser = pd.Series(data)
psser = ps.Series(data)
- self.assert_eq(pser, psser.to_pandas())
+ self.assert_eq(pser, psser._to_pandas())
Review Comment:
Just for my info, what is the difference in the old and new method, anything functional?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #38042: [SPARK-40606][PS][TEST] Eliminate `to_pandas` warnings in test
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on PR #38042:
URL: https://github.com/apache/spark/pull/38042#issuecomment-1263016279
also cc @HyukjinKwon @itholic @xinrong-meng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org