Trigger on GroupStateTimeout with no new data in group

[Abhishek Gupta <ab...@gmail.com>]

Hi All,

I had a question about modeling a user session kind of analytics use-case
in Spark Structured Streaming. Is there a way to model something like this
using Arbitrary stateful Spark streaming

User session -> reads a few FAQS on a website and then decides to create a
ticket or not
FAQ Deflection Metrics:
i) Successful Deflection: No issues created within 5 mins of reading the
last FAQ
ii) Failed Deflection: Issue is created within 5 mins of reading FAQ

There are 3 cases here, 2 of which can be done using FlatMapGroupWithState,
not sure about the 3rd i.e
i) Maintain user's last action state, if issue create event happens and
last state is FAQ view within 5 mins -> Failed deflection
ii) Maintain user's last state, if issue create and last state is FAQ view
beyond 5 mins -> Successful deflection
iii) Maintain user's last state with maybe a Processing Time timeout of 5
mins i.e FAQ viewed at T1, no issue creation event from user but time now
is T1 + 5 mins, so we should increment Successful deflection->

Can we do it using Spark GroupStateTimeout? I was confused if a timeout
trigger can happen with no data coming in the group