[GitHub] [spark] HeartSaVioR commented on pull request #33691: [SPARK-36465][SS] Dynamic gap duration in session window

[GitBox <gi...@apache.org>]

HeartSaVioR commented on pull request #33691:
URL: https://github.com/apache/spark/pull/33691#issuecomment-895764462


   Thanks for the contribution!
   
   For the functionality of dynamic gap duration in session window, there are some proposals in other frameworks and I think these proposals make sense despite the difference - these proposals are based on non-SQL API whereas this is based on SQL API.
   
   https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/windows/
   https://cwiki.apache.org/confluence/display/KAFKA/KIP-362%3A+Support+dynamic+gap+session+window
   
   Basically end users should be able to apply some logic against the row to extract the gap duration, and we should ensure the usage is simple enough. For example, suppose a use case we apply gap duration to 30 minutes for working hours (based on event time column) and 10 minutes for non-working hours. (It could also be based on event type column or something like.) How to represent this query?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org