[GitHub] [spark] HeartSaVioR commented on pull request #34220: [SPARK-36954][SQL] Fast fail when withWatermark called on non-streaming dataset

[GitBox <gi...@apache.org>]

HeartSaVioR commented on pull request #34220:
URL: https://github.com/apache/spark/pull/34220#issuecomment-941792480


   Sorry for the late. Technically, the result between batch and streaming should be same if the query makes the perfect watermark. We should try to fix if there is a case not respecting this and the operation is supported on both batch and streaming.
   
   Batch assumes perfect watermark because all inputs are available when running the query, so it's legitimate to consider withWatermark as no-op in the batch query. And as @zsxwing said, we are trying to make ease of switching the query between batch and streaming.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org