You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by phalodi <gi...@git.apache.org> on 2016/07/08 09:23:08 UTC
[GitHub] spark pull request #14104: Add Asynchronous Actions documentation
GitHub user phalodi opened a pull request:
https://github.com/apache/spark/pull/14104
Add Asynchronous Actions documentation
## What changes were proposed in this pull request?
Add Asynchronous Actions documentation inside action of programming guide
## How was this patch tested?
check the documentation indentation and formatting with md preview.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/phalodi/spark SPARK-16438
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14104.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14104
----
commit aba737f9be520ec33d0841d718f8250fbfeb09e1
Author: sandy <ph...@gmail.com>
Date: 2016-07-08T09:15:12Z
Add Asynchronous Actions documentation
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14104: [SPARK-16438] Add Asynchronous Actions documentat...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14104#discussion_r70180403
--- Diff: docs/programming-guide.md ---
@@ -1099,6 +1099,34 @@ for details.
</tr>
</table>
+#### Asynchronous Actions
+Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking each other.
--- End diff --
"without blocking each other": actions never block each other anyway. They don't block the calling thread.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: Add Asynchronous Actions documentation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14104
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14104
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14104
**[Test build #61979 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61979/consoleFull)** for PR 14104 at commit [`aba737f`](https://github.com/apache/spark/commit/aba737f9be520ec33d0841d718f8250fbfeb09e1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14104
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14104: [SPARK-16438] Add Asynchronous Actions documentat...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14104#discussion_r70180404
--- Diff: docs/programming-guide.md ---
@@ -1099,6 +1099,34 @@ for details.
</tr>
</table>
+#### Asynchronous Actions
+Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking each other.
+The following table lists some of the asynchronous actions supported by Spark. Refer to the RDD API doc ([Scala](api/scala/index.html#org.apache.spark.rdd.AsyncRDDActions),[Java](api/java/org/apache/spark/rdd/AsyncRDDActions.html))
+
+<table class="table">
+<tr><th>Asynchronous Action</th><th>Meaning</th></tr>
+<tr>
+ <td> <b>collectAsync</b>() </td>
+ <td> Returns a future for retrieving all the elements of the dataset as an array at the driver program. This is usually useful after a filter or other operation that returns a sufficiently small subset of the data. </td>
+</tr>
+<tr>
+ <td> <b>countAsync</b>() </td>
+ <td> Returns a future for counting the number of elements in the RDD. </td>
+</tr>
+<tr>
+ <td> <b>foreachAsync</b>(<i>func</i>) </td>
+ <td> Applies a function f to all elements of this RDD. </td>
--- End diff --
This isn't specific to the async version.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14104
@phalodi no need to ping. We don't merge things immediately, especially when non-essential. It gives more time for review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61979/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen ok no problem so what you suggest should i remove the table and just add the line below the actions table and give link to scala and java docs? And the line "Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking each other." is not correct so should i add that to "return future to non-block calling thread?" even in above statement my intention is also talk about calling thread because when we call action it will always in calling thread so calling thread will block but when we use async actions it will not block so we are targeted here non-blocking for calling thread because actions are called in calling thread. so i make changes and push it again as u suggest just add single line at end of actions and give link for scala and java doc api. Its cool?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14104
Although I don't think it's essential to mention this API (it's in the API doc) it wouldn't hurt to point to it here. The mention should be accurate though. I think it's correct to say that the Spark RDD API also exposes asynchronous versions of some actions like foreach, which return a handle immediately to the caller which can be used to wait for its completion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen Below is final screen shot as you suggested i hope its looks good please review it and merge it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14104: [SPARK-16438] Add Asynchronous Actions documentat...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14104#discussion_r70240934
--- Diff: docs/programming-guide.md ---
@@ -1099,6 +1099,9 @@ for details.
</tr>
</table>
+Spark RDD API also exposes asynchronous versions of some actions like foreach, collect, count etc. which return a handle immediately to the caller which can be used to wait for its completion.
--- End diff --
I have a few edits to suggest so let me just propose some text directly:
The Spark RDD API also exposes asynchronous versions of some actions, like `foreachAsync` for `foreach`, which immediately return a `FutureAction` to the caller instead of blocking on completion of the action. This can be used to manage or wait for the asynchronous execution of the action.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14104
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen yeah i agree with you its same like non async actions but yeah we should list it because its more useful in real life application when single application running multiple jobs of different users. so we should list it so user will more familiar with it and also know which are the async actions. i think its helpful for users still if you think its not a valid point then just tell me i change pull request and add just reference for async actions but i want user should know because when i first time see it i m surprise that in whole documentation no where mention about async actions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen I put back-tick-quote for foreach, foreachAsync and FutureAction.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14104
**[Test build #61979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61979/consoleFull)** for PR 14104 at commit [`aba737f`](https://github.com/apache/spark/commit/aba737f9be520ec33d0841d718f8250fbfeb09e1).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen Is this is correct if it is then please merge it otherwise give comment if needed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen i know its nothing major just call normal functions in future but still naive user first time who learn scala and spark dont know what are future and all so at least we should add reference of it. if you suggest i simple add reference link without adding any statement of blocking and non blocking. so atleast user know there is some api like this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14104
I fixed my comment above, sorry. They _do_ block the calling thread, but actions triggered in different threads on one RDD do not block each other. The first sentence isn't correct. My suggestion is to omit the table, not all of the text you added.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen so what do you think we should not add this api reference in document?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14104: [SPARK-16438] Add Asynchronous Actions documentat...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/14104
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen @rxin Please review it and give your comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen now its perfect i think at last from table to one line but yeah you are right it good to just mention that spark provide it and not duplicate table of actions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@rxin Below are screen shot of Asynchronous Actions inside actions and screen shot of index of programming guide.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14104
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen Thanks man i know i am not giving correct language but my intention is also same i hope its final commit :) if its done can you merge it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14104
**[Test build #61980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61980/consoleFull)** for PR 14104 at commit [`aba737f`](https://github.com/apache/spark/commit/aba737f9be520ec33d0841d718f8250fbfeb09e1).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen sure i will make appropriate changes and push it again
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen yeah you are right its not blocking calling thread but they execute sequentially right but in async action its return future so its running on different threads soo not run sequentially soo its fully non blocking example if i have action in that we have some transformation like map n inside map i have blocking code then how its work its block or not?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14104
**[Test build #61980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61980/consoleFull)** for PR 14104 at commit [`aba737f`](https://github.com/apache/spark/commit/aba737f9be520ec33d0841d718f8250fbfeb09e1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@AmplabJenkins I dont know i just create the issue on JIRA and make changes and make pull request for merge , I think it is valid pull request because in real world all application need non blocking execution. So if its is valid then merge it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61980/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14104
Mostly this just duplicates the non-async documentation. I think it's worth mentioning the existence of the async methods with a pointer to the API docs, but, I don't think this table adds value.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14104: [SPARK-16438] Add Asynchronous Actions documentat...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14104#discussion_r70242186
--- Diff: docs/programming-guide.md ---
@@ -1099,6 +1099,9 @@ for details.
</tr>
</table>
+The Spark RDD API also exposes asynchronous versions of some actions, like foreachAsync for foreach, which immediately return a FutureAction to the caller instead of blocking on completion of the action. This can be used to manage or wait for the asynchronous execution of the action.
--- End diff --
Last nit pick: back-tick-quote the code nouns like I did.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14104
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen i make changes as you suggest i remove the table and add a single below actions table and also change statement to be more clear about non blocking.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62231/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14104
@phalodi it's a bot, not a person.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by phalodi <gi...@git.apache.org>.
Github user phalodi commented on the issue:
https://github.com/apache/spark/pull/14104
@srowen @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14104
Can you paste a screenshot of the doc?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14104
**[Test build #62231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62231/consoleFull)** for PR 14104 at commit [`845a123`](https://github.com/apache/spark/commit/845a1233a0b458cdf056b8957afd99ea74be02da).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14104
**[Test build #62231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62231/consoleFull)** for PR 14104 at commit [`845a123`](https://github.com/apache/spark/commit/845a1233a0b458cdf056b8957afd99ea74be02da).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14104: [SPARK-16438] Add Asynchronous Actions documentat...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14104#discussion_r70222276
--- Diff: docs/programming-guide.md ---
@@ -1099,6 +1099,9 @@ for details.
</tr>
</table>
+Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking calling thread. Refer to the RDD API doc for asynchronous actions ([Scala](api/scala/index.html#org.apache.spark.rdd.AsyncRDDActions),[Java](api/java/org/apache/spark/rdd/AsyncRDDActions.html))
--- End diff --
I still don't think this is accurate. Spark can execute actions concurrently without this API. This merely makes the call non-blocking for the caller. It really adds very little beyond calling the normal API and wrapping in a Future (though it's more useful for the Java API where that's harder). Hence why I thought it was unofficially deprecated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org