You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/03/29 05:04:12 UTC

[GitHub] [spark] HeartSaVioR commented on pull request #31989: [WIP][SPARK-34891][SS] Introduce state store manager for session window in streaming query

HeartSaVioR commented on pull request #31989:
URL: https://github.com/apache/spark/pull/31989#issuecomment-809067233


   Except the test suite, one more thing worths to address here is write amplification; we "blindly" replace all start times and all sessions. This could bring unnecessary writes on "unmodified" existing sessions. In many cases we expect the new inputs will be bound and expanding to the existing sessions, but with very long watermark gap and old inputs which have various timestamps, the case could still happen.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org