You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/11/15 19:55:56 UTC
[GitHub] [spark] xinrong-databricks opened a new pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
xinrong-databricks opened a new pull request #34608:
URL: https://github.com/apache/spark/pull/34608
### What changes were proposed in this pull request?
The PR is proposed to rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api.
### Why are the changes needed?
Currently, (Spark)DataFrame.to_pandas_on_spark is too long to memorize and inconvenient to call.
The PR is proposed to rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api.
We may improve the user experience and make APIs more developer-friendly by implementing that.
### Does this PR introduce _any_ user-facing change?
Yes.
(Spark)DataFrame.to_pandas_on_spark is renamed to (Spark)DataFrame.pandas_api
```py
>>> sdf = spark.createDataFrame([{'name': 'Alice', 'age': 1}])
>>> sdf.pandas_api()
age name
0 1 Alice
>>> sdf.to_pandas_on_spark()
Traceback (most recent call last):
...
AttributeError: 'DataFrame' object has no attribute 'to_pandas_on_spark'
```
### How was this patch tested?
Existing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969326273
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49719/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xinrong-databricks commented on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
xinrong-databricks commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969757204
Thanks @itholic! I removed the deprecation warning of `to_koalas`. I will update the PR title and description accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xinrong-databricks commented on a change in pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
xinrong-databricks commented on a change in pull request #34608:
URL: https://github.com/apache/spark/pull/34608#discussion_r749863901
##########
File path: python/pyspark/sql/dataframe.py
##########
@@ -3198,7 +3198,7 @@ def writeTo(self, table: str) -> DataFrameWriterV2:
"""
return DataFrameWriterV2(self, table)
- def to_pandas_on_spark(
+ def pandas_api(
Review comment:
Makes sense! Deprecated `to_pandas_on_spark`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969872541
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49734/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969797202
**[Test build #145264 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145264/testReport)** for PR 34608 at commit [`5ec279a`](https://github.com/apache/spark/commit/5ec279ac94cdc66683cf6f0802124a89c60435d0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969834784
**[Test build #145264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145264/testReport)** for PR 34608 at commit [`5ec279a`](https://github.com/apache/spark/commit/5ec279ac94cdc66683cf6f0802124a89c60435d0).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ueshin edited a comment on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
ueshin edited a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-971912666
> Just came to think of it, maybe we could rather restoring the `to_koalas()` and use both way (`to_pandas_on_spark` and `to_koalas`) as valid ??
This is not a strong opinion on undeprecating `to_koalas`, but I don't think we will keep `to_koalas` in a long run, so we might want to keep it deprecated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969872541
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49734/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-970559782
**[Test build #145289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145289/testReport)** for PR 34608 at commit [`f53eba0`](https://github.com/apache/spark/commit/f53eba07f4db6a0551e3ae8d4f52f443f19de282).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ueshin commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
ueshin commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-971912666
> Just came to think of it, maybe we could rather restoring the `to_koalas()` and use both way (`to_pandas_on_spark` and `to_koalas`) as valid ??
This is not a strong opinion on undeprecating `to_koalas`, but I don't think we will keep `to_koalas` in a long run, so we should keep it deprecated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #34608: [SPARK-37337][PYTHON] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #34608:
URL: https://github.com/apache/spark/pull/34608
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969797202
**[Test build #145264 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145264/testReport)** for PR 34608 at commit [`5ec279a`](https://github.com/apache/spark/commit/5ec279ac94cdc66683cf6f0802124a89c60435d0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969335588
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145249/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969842648
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49734/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969847044
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145264/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969291232
**[Test build #145249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145249/testReport)** for PR 34608 at commit [`f91e18f`](https://github.com/apache/spark/commit/f91e18ff279d8297564ae525550143252153e6df).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] holdenk commented on a change in pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
holdenk commented on a change in pull request #34608:
URL: https://github.com/apache/spark/pull/34608#discussion_r751640100
##########
File path: python/docs/source/getting_started/quickstart_ps.ipynb
##########
@@ -14486,4 +14486,4 @@
},
"nbformat": 4,
"nbformat_minor": 1
-}
+}
Review comment:
Could be, I don't have strong feels about this whitespace change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ueshin commented on a change in pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
ueshin commented on a change in pull request #34608:
URL: https://github.com/apache/spark/pull/34608#discussion_r749643160
##########
File path: python/pyspark/sql/dataframe.py
##########
@@ -3198,7 +3198,7 @@ def writeTo(self, table: str) -> DataFrameWriterV2:
"""
return DataFrameWriterV2(self, table)
- def to_pandas_on_spark(
+ def pandas_api(
Review comment:
We can't just rename the API because we already have a release.
We should keep or deprecate the original function and add the new API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969847044
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145264/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-970583364
**[Test build #145289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145289/testReport)** for PR 34608 at commit [`f53eba0`](https://github.com/apache/spark/commit/f53eba07f4db6a0551e3ae8d4f52f443f19de282).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-970652750
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49759/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-970597738
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49759/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-970639134
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49759/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969868456
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49734/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] itholic edited a comment on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
itholic edited a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969631888
+1 for shorten way.
Just came to think of it, maybe we could rather restoring the `to_koalas()` and use both way (`to_pandas_on_spark` and `to_koalas`) as valid ??
WDYT, @HyukjinKwon @ueshin ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969360406
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49719/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-970606152
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145289/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xinrong-databricks commented on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
xinrong-databricks commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969268600
CC @HyukjinKwon @ueshin @itholic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-970559782
**[Test build #145289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145289/testReport)** for PR 34608 at commit [`f53eba0`](https://github.com/apache/spark/commit/f53eba07f4db6a0551e3ae8d4f52f443f19de282).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34608: [SPARK-37337][PYTHON] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-976007989
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969335588
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145249/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969379933
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49719/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969379933
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49719/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] itholic commented on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
itholic commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969631888
Just came to think of it, maybe we could rather restoring the `to_koalas()` and use both way (`to_pandas_on_spark` and `to_koalas`) as valid ??
WDYT, @HyukjinKwon @ueshin ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-970652750
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49759/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xinrong-databricks commented on a change in pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
xinrong-databricks commented on a change in pull request #34608:
URL: https://github.com/apache/spark/pull/34608#discussion_r749639639
##########
File path: python/docs/source/getting_started/quickstart_ps.ipynb
##########
@@ -14486,4 +14486,4 @@
},
"nbformat": 4,
"nbformat_minor": 1
-}
+}
Review comment:
Is the newline removed automatically because I edited the file in IDE?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969311149
**[Test build #145249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145249/testReport)** for PR 34608 at commit [`f91e18f`](https://github.com/apache/spark/commit/f91e18ff279d8297564ae525550143252153e6df).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34608: [SPARK-37338] Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-969291232
**[Test build #145249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145249/testReport)** for PR 34608 at commit [`f91e18f`](https://github.com/apache/spark/commit/f91e18ff279d8297564ae525550143252153e6df).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34608: [SPARK-37337] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34608:
URL: https://github.com/apache/spark/pull/34608#issuecomment-970606152
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145289/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org