You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by Satyajitv <gi...@git.apache.org> on 2018/08/02 05:40:25 UTC
[GitHub] spark pull request #21961: Spark 20597
GitHub user Satyajitv opened a pull request:
https://github.com/apache/spark/pull/21961
Spark 20597
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
Default topic first checks if TOPIC_OPTION_KEY is provided,
if TOPIC_OPTION_KEY is provided then
defaulttopic=TOPIC_OPTION_KEY
else TOPIC_OPTION_KEY is not provided then
defaulttopic=PATH_OPTION_KEY
## How was this patch tested?
Have tested the code in local, but would start writing tests once the approach is confirmed by @jaceklaskowski, as I am expecting change requests.
PF more details on https://issues.apache.org/jira/browse/SPARK-20597
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Satyajitv/spark SPARK-20597
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21961.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21961
----
commit d4e1ed0c25121ad5bf24cfe137e2ee1bff430c94
Author: Satyajit Vegesna <sa...@...>
Date: 2018-08-02T05:12:57Z
SPARK-20597 KafkaSourceProvider falls back on path as synonym for topic
commit 381e66fa0bdd14b5754d8d81710021714e5fc031
Author: Satyajit Vegesna <sa...@...>
Date: 2018-08-02T05:22:07Z
Added parameters that were mistakenly taken out in previous commit
commit 6dc893a681721b51e61a9df099ae8f2c865c38c1
Author: Satyajit Vegesna <sa...@...>
Date: 2018-08-02T05:24:13Z
Added parameters that were mistakenly taken out in previous commit
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21961: Spark 20597
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21961
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21961: Spark 20597
Posted by mahmoudmahdi24 <gi...@git.apache.org>.
Github user mahmoudmahdi24 commented on the issue:
https://github.com/apache/spark/pull/21961
Hello @Satyajitv, please rename the title of this PR properly.
The PR title should be of the form [SPARK-xxxx][COMPONENT] Title, where SPARK-xxxx is the relevant JIRA number, COMPONENT is one of the PR categories shown at spark-prs.appspot.com and Title may be the JIRA’s title or a more specific title describing the PR itself.
Take a look to this helpful document : https://spark.apache.org/contributing.html
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21961: Spark 20597
Posted by holdensmagicalunicorn <gi...@git.apache.org>.
Github user holdensmagicalunicorn commented on the issue:
https://github.com/apache/spark/pull/21961
@Satyajitv, thanks! I am a bot who has found some folks who might be able to help with the review:@tdas, @zsxwing and @cloud-fan
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21961: Spark 20597
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21961
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21961: Spark 20597
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21961
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21961: Spark 20597
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21961#discussion_r210800762
--- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala ---
@@ -231,7 +231,13 @@ private[kafka010] class KafkaSourceProvider extends DataSourceRegister
parameters: Map[String, String],
partitionColumns: Seq[String],
outputMode: OutputMode): Sink = {
- val defaultTopic = parameters.get(TOPIC_OPTION_KEY).map(_.trim)
+ // Picks the defaulttopicname from "path" key, an entry in "parameters" Map,
+ // if no topic key is present in the "parameters" Map and is provided with key "path".
+ val defaultTopic = parameters.get(TOPIC_OPTION_KEY) match {
--- End diff --
Isn't this simpler as something like
```
val defaultTopic = parameters.getOrElse(TOPIC_OPTION_KEY, parameters.get(PATH_OPTION_KEY)).map(_.trim)
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21961: Spark 20597
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21961
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org