How to timeout a statement automatically after a given interval

[sanchay javeria <sa...@gmail.com>]

Hey all,

I have a use-case where I want to timeout a statement automatically after a
given interval (using interactive session and SQL interpreter). This would
mean that, say, after 1 minute, the executing statement is automatically
cancelled and an exception is thrown. The underlying Session and the Spark
Application however would still be up. I'm having some troubles with this
but this is what my first few tries entailed -


   - I implemented my own custom driver class by inheriting
   from repl/src/main/scala/org/apache/livy/repl/ReplDriver.scala and overrode
   the `def handle(ctx: ChannelHandlerContext, msg:
   BaseProtocol.ReplJobRequest): Int` RPC handler. Here, I tried to wrap
   the returned value in a `Future` object with a timeout but that didn't work
   since the call to `Session.execute` also creates a non-blocking `Future`
   object which calls `SQLInterpreter.execute()` to run the query.



   - My other approach was to simply add a timeout to `Future.get()` in `def
   executeStatement(content: ExecuteRequest): Statement` and `def
getStatement(stmtId:
   Int): Option[Statement]` inside
org/apache/livy/server/interactive/InteractiveSession.scala
   but that also didn't work because these methods simply return a JSON where
   the output field isn't populated.

How do I go about implementing a statement execution timeout inside Livy
itself? Implementing this on the client side is NOT an option for me due to
some constraints.

Please let me know if I can get some help on this, I'll really appreciate
it!

Thank you,
Sanchay