You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by Jimmy-Newtron <gi...@git.apache.org> on 2016/12/08 17:39:28 UTC
[GitHub] spark pull request #16215: Update PairDStreamFunctions.scala
GitHub user Jimmy-Newtron opened a pull request:
https://github.com/apache/spark/pull/16215
Update PairDStreamFunctions.scala
## What changes were proposed in this pull request?
Added methods to retrieve keys or values of a pair DStream
## How was this patch tested?
NONE
Added two methods that are available into the PairRDDFunctions.scala
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Jimmy-Newtron/spark patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16215.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16215
----
commit c51e047252fbcf602886d5b555d8aace22158fb2
Author: Jimmy-Newtron <en...@protonmail.com>
Date: 2016-12-08T17:36:30Z
Update PairDStreamFunctions.scala
Added two methods that are available into the PairRDDFunctions.scala
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16215: [Streaming] Update PairDStreamFunctions.scala
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/16215
I get it, it still seems not so worth it. The `DStream` API has always been mostly redundant with `foreachRDD` + the RDD API, and is itself maybe not something being actively updated anyway.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16215: [Streaming] Update PairDStreamFunctions.scala
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/16215
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16215: [Streaming] Update PairDStreamFunctions.scala
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/16215
I don't think this worth new API methods since it's so easy to do this directly. Please see http://spark.apache.org/contributing.html too. I'd close this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16215: [Streaming] Update PairDStreamFunctions.scala
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16215
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16215: [Streaming] Update PairDStreamFunctions.scala
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16215
This is actually parity with PairRDDFunctions so I think it's OK to have them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16215: [Streaming] Update PairDStreamFunctions.scala
Posted by Jimmy-Newtron <gi...@git.apache.org>.
Github user Jimmy-Newtron commented on the issue:
https://github.com/apache/spark/pull/16215
I think that this new API methods is going to bring consistency between PairRDDFunctions and PairDStreamFunctions by providing the same API to access the same data.
I see the use case of a streaming application that is consuming a Kafka topic to retrieve the values as the simplest example of usage of this new API.
I accept the PR rejection, even if the new API is matching only positive aspects of the Code Review Criteria.
Positives:
Adds functionality needed by a large number of users
Simple, targeted
Negatives, Risks: NONE
Best regards
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org