You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2023/05/26 02:01:32 UTC
[GitHub] [spark] zhengruifeng opened a new pull request, #41324: [SPARK-43603][PS][CONNECT][TEST] Reorg `ps.DataFrame` unit tests
zhengruifeng opened a new pull request, #41324:
URL: https://github.com/apache/spark/pull/41324
### What changes were proposed in this pull request?
reorganize `ps.DataFrame` unit tests, to follow the categories in
https://spark.apache.org/docs/latest/api/python/reference/pyspark.pandas/frame.html
and https://pandas.pydata.org/docs/reference/frame.html
### Why are the changes needed?
1, even after https://github.com/apache/spark/commit/58b6535cbf76f2b26fc08b94905a57dcc4d955f6, pyspark.pandas.tests.test_dataframe and pyspark.pandas.tests.test_dataframe_slow are still too slow, each one normally takes >5 minutes, and sometimes >10 minutes.
2, the tests were not well organized
### Does this PR introduce _any_ user-facing change?
no test-only
### How was this patch tested?
updated CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #41324: [WIP][SPARK-43603][PS][CONNECT][TEST] Reorganize `ps.DataFrame` unit tests
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #41324:
URL: https://github.com/apache/spark/pull/41324#issuecomment-1565048093
some groups are too slow in this PR, I am closing it in favor of https://github.com/apache/spark/pull/41330
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng closed pull request #41324: [WIP][SPARK-43603][PS][CONNECT][TEST] Reorganize `ps.DataFrame` unit tests
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #41324: [WIP][SPARK-43603][PS][CONNECT][TEST] Reorganize `ps.DataFrame` unit tests
URL: https://github.com/apache/spark/pull/41324
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #41324: [WIP][SPARK-43603][PS][CONNECT][TEST] Reorganize `ps.DataFrame` unit tests
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #41324:
URL: https://github.com/apache/spark/pull/41324#issuecomment-1563897174
@HyukjinKwon i'm splitting some tests a bit further, since they are much slower than others
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org