[GitHub] marmbrus commented on a change in pull request #23702: [SPARK-26785][SQL] data source v2 API refactor: streaming write

[GitBox <gi...@apache.org>]

marmbrus commented on a change in pull request #23702: [SPARK-26785][SQL] data source v2 API refactor: streaming write
URL: https://github.com/apache/spark/pull/23702#discussion_r258165039
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
 ##########
 @@ -513,13 +514,16 @@ class MicroBatchExecution(
 
     val triggerLogicalPlan = sink match {
       case _: Sink => newAttributePlan
-      case s: StreamingWriteSupportProvider =>
-        val writer = s.createStreamingWriteSupport(
-          s"$runId",
-          newAttributePlan.schema,
-          outputMode,
-          new DataSourceOptions(extraOptions.asJava))
-        WriteToDataSourceV2(new MicroBatchWrite(currentBatchId, writer), newAttributePlan)
+      case s: SupportsStreamingWrite =>
+        // TODO: we should translate OutputMode to concrete write actions like truncate, but
+        // the truncate action is being developed in SPARK-26666.
 
 Review comment:
   Sorry for being unclear. My comment was in response to wenchen's suggestion that we "remove UPDATE mode completely and re-add it when we have a detailed design?".
   
   The feature has utility even without passing the key.  That said, I agree it would be better if the key is available in the V2 API.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org