You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/11/17 04:27:59 UTC

[GitHub] [spark] HyukjinKwon opened a new pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

HyukjinKwon opened a new pull request #34624:
URL: https://github.com/apache/spark/pull/34624


   ### What changes were proposed in this pull request?
   
   This PR documents `DataFrame.mapInArrow` in PySpark (https://github.com/apache/spark/pull/34505).
   I initially planned to document like `DataFrame.mapInPandas` (https://github.com/apache/spark/pull/25025) but decided to don't publicly document like that because this is an unstable developer API.
   
   ### Why are the changes needed?
   
   For users to refer to, and use.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes, it documents `DataFrame.mapInArrow` as below:
   
   <img width="1205" alt="Screen Shot 2021-11-17 at 1 27 18 PM" src="https://user-images.githubusercontent.com/6477701/142134690-05ececab-c2c4-46a3-a57e-17c91f3d31b3.png">
   
   ### How was this patch tested?
   
   Manually tested by building PySpark documentation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971225277


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49785/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971204986


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145314/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971271253


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49784/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-972389499


   Merged to master.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon closed pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #34624:
URL: https://github.com/apache/spark/pull/34624


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971176381


   **[Test build #145314 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145314/testReport)** for PR 34624 at commit [`79ec116`](https://github.com/apache/spark/commit/79ec1162f5886067dda5421a30c4c3c100893e53).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971310406






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971172606


   **[Test build #145313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145313/testReport)** for PR 34624 at commit [`79ec116`](https://github.com/apache/spark/commit/79ec1162f5886067dda5421a30c4c3c100893e53).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971172606


   **[Test build #145313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145313/testReport)** for PR 34624 at commit [`79ec116`](https://github.com/apache/spark/commit/79ec1162f5886067dda5421a30c4c3c100893e53).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971176381


   **[Test build #145314 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145314/testReport)** for PR 34624 at commit [`79ec116`](https://github.com/apache/spark/commit/79ec1162f5886067dda5421a30c4c3c100893e53).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] viirya commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-972386777


   lgtm


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971310406






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971174633


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145313/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971225174


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49784/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971304795


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49785/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971197353


   **[Test build #145314 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145314/testReport)** for PR 34624 at commit [`79ec116`](https://github.com/apache/spark/commit/79ec1162f5886067dda5421a30c4c3c100893e53).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-972375528


   cc @viirya mind taking a quick look please?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-972389316


   Thanks! Merged to master.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971174633


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145313/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971174606


   **[Test build #145313 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145313/testReport)** for PR 34624 at commit [`79ec116`](https://github.com/apache/spark/commit/79ec1162f5886067dda5421a30c4c3c100893e53).
    * This patch **fails to build**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971175866


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34624:
URL: https://github.com/apache/spark/pull/34624#issuecomment-971204986


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145314/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org