You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "itholic (via GitHub)" <gi...@apache.org> on 2023/12/05 06:21:28 UTC
[PR] [SPARK-46262][PS] Enable test for `np.left_shift` for Pandas-on-Spark object. [spark]
itholic opened a new pull request, #44178:
URL: https://github.com/apache/spark/pull/44178
### What changes were proposed in this pull request?
This PR proposes to enable test for `np.left_shift` for Pandas-on-Spark object.
### Why are the changes needed?
Because we completely support the PyArrow>=4.0.0, so the behavior difference within PyArrow<0.15 will not appear.
Now the behavior always same within the supported version as below:
```python
>>> pdf = pd.DataFrame({"A": [1, 2, 3, 4]})
>>> psdf = ps.DataFrame({"A": [1, 2, 3, 4]})
>>> np.left_shift(pdf, 1)
A
0 2
1 4
2 6
3 8
>>> np.left_shift(psdf, 1)
A
0 2
1 4
2 6
3 8
```
### Does this PR introduce _any_ user-facing change?
No, this just enables the test case.
### How was this patch tested?
The existing CI especially `NumPyCompatTests` should pass.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46262][PS] Enable test for `np.left_shift` for Pandas-on-Spark object. [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #44178:
URL: https://github.com/apache/spark/pull/44178#issuecomment-1840475960
merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46262][PS] Enable test for `np.left_shift` for Pandas-on-Spark object. [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #44178: [SPARK-46262][PS] Enable test for `np.left_shift` for Pandas-on-Spark object.
URL: https://github.com/apache/spark/pull/44178
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46262][PS] Enable test for `np.left_shift` for Pandas-on-Spark object. [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #44178:
URL: https://github.com/apache/spark/pull/44178#discussion_r1414970208
##########
python/pyspark/pandas/tests/test_numpy_compat.py:
##########
@@ -41,10 +41,6 @@ class NumPyCompatTestsMixin:
"log1p", # flaky
"modf",
"floor_divide", # flaky
- # Results seem inconsistent in a different version of, I (Hyukjin) suspect, PyArrow.
Review Comment:
:-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org