You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by surq <gi...@git.apache.org> on 2014/12/10 11:52:47 UTC
[GitHub] spark pull request: Print the specified number of data and handle ...
GitHub user surq opened a pull request:
https://github.com/apache/spark/pull/3662
Print the specified number of data and handle all of the elements in RDD
Dstream.print function:Print 10 elements and handle 11 elements.
A new function based on Dstream.print function is presented:
the new function:
Print the specified number of data and handle all of the elements in RDD.
there is a work scene:
val dstream = stream.map->filter->mapPartitions->print
the data after filter need update database in mapPartitions,but don't need print each data,only need to print the top 20 for view the data processing.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/surq/spark SPARK-4817
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3662.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3662
----
commit 4e3f715941f94cb2467ca68b205a5fa3630130a3
Author: surq <su...@asiainfo.com>
Date: 2014-12-10T10:49:54Z
Print the specified number of data and handle all of the elements in RDD
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-4817[STREAMING]Print the specified numbe...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/3662#issuecomment-68017411
As I commented on the JIRA, this isnt right way to do what you are trying to achieve. The JIRA has already been closed, so mind closing this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4817][streaming]Print the specified num...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3662#issuecomment-66435798
[Test build #24306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24306/consoleFull) for PR 3662 at commit [`4e3f715`](https://github.com/apache/spark/commit/4e3f715941f94cb2467ca68b205a5fa3630130a3).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-4817[STREAMING]Print the specified numbe...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3662#issuecomment-66586675
[Test build #24358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24358/consoleFull) for PR 3662 at commit [`411b287`](https://github.com/apache/spark/commit/411b28709b55cfa94ebd04ced6d67df997ebf467).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-4817[STREAMING]Print the specified numbe...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/3662
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4817][streaming]Print the specified num...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3662#issuecomment-66444342
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24306/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-4817[STREAMING]Print the specified numbe...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3662#issuecomment-66595143
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24358/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4817][streaming]Print the specified num...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/3662#issuecomment-66434968
You should put `SPARK-XXXX [STREAMING]` in the title. But your original JIRA was a duplicate of https://issues.apache.org/jira/browse/SPARK-3325 so perhaps you can connect this to that JIRA.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-4817[STREAMING]Print the specified numbe...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3662#issuecomment-66595135
[Test build #24358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24358/consoleFull) for PR 3662 at commit [`411b287`](https://github.com/apache/spark/commit/411b28709b55cfa94ebd04ced6d67df997ebf467).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4817][streaming]Print the specified num...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3662#issuecomment-66444334
[Test build #24306 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24306/consoleFull) for PR 3662 at commit [`4e3f715`](https://github.com/apache/spark/commit/4e3f715941f94cb2467ca68b205a5fa3630130a3).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org