[GitHub] HeartSaVioR edited a comment on issue #23859: [SPARK-26956][SQL] remove streaming output mode from data source v2 APIs

[GitBox <gi...@apache.org>]

HeartSaVioR edited a comment on issue #23859: [SPARK-26956][SQL] remove streaming output mode from data source v2 APIs
URL: https://github.com/apache/spark/pull/23859#issuecomment-466273566
 
 
   I'm sorry but I can't find mentioning of complete mode from neither the doc nor the PR you linked.
   
   Btw, here's explanation of `Complete Mode` in doc on Structured Streaming http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html :
   
   > Complete Mode - The entire updated Result Table will be written to the external storage. It is up to the storage connector to decide how to handle writing of the entire table.
   
   So does the final result for complete mode depend on sink side how to handle truncation - if sink ignores truncation it will have full of data, and if sink does truncation it will only have new data? 
   
   Maybe I'm not clear about "new data". Does new data mean output rows from current batch, or all rows in full of result table?
   
   Previously I would only think final result always contain every rows the query outputs. I would like to be clear that we keep this as it is, or let sink decide it.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org