[kudu-CR] KUDU-2640: Add Spark Structured Streaming Sink

["Andrew Wong (Code Review)" <ge...@cloudera.org>]

Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12087 )

Change subject: KUDU-2640: Add Spark Structured Streaming Sink
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/12087/2/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala:

http://gerrit.cloudera.org:8080/#/c/12087/2/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala@216
PS2, Line 216:   private def getOperationType(parameters: Map[String, String]): OperationType = {
             :     parameters.get(OPERATION).map(stringToOperationType).getOrElse(Upsert)
             :   }
Hrm, I get why this is the case for KuduSink, but should it be the case for the source in general? Seems like it might make misconfiguring and subsequently incorrectly upserting rows pretty easy. Could we just add a default arg to KuduSink instead?


http://gerrit.cloudera.org:8080/#/c/12087/2/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala@466
PS2, Line 466: batchId: Long
May be obvious, but mind adding a small note on why we shouldn't use this? E.g. what it's used for in Spark and why we don't care.



-- 
To view, visit http://gerrit.cloudera.org:8080/12087
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I731e35f82c8cca7d911e4d879aa6853112132b17
Gerrit-Change-Number: 12087
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Wed, 02 Jan 2019 18:25:47 +0000
Gerrit-HasComments: Yes