You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "juliuszsompolski (via GitHub)" <gi...@apache.org> on 2023/11/20 18:09:18 UTC
[PR] [WIP][SPARK-46011][CONNECT] Session heartbeat/keepalive using ExtendSession RPC [spark]
juliuszsompolski opened a new pull request, #43913:
URL: https://github.com/apache/spark/pull/43913
### What changes were proposed in this pull request?
Recently, `ReleaseSession` RPC was added for the clients to be able to stop the session, and release resources on the server. However, if the client dies without sending it, the session will still linger on the server for 1 hour. In this PR, an `ExtendSession` RPC is added. Active client using it will actively send `ExtendSession` "ping" requests from a background thread. A server receiving those pings can be more aggressive in timing out the session once they stop. If a client does not send `ExtendSession`, the old behavior is preserved by the server.
TODO details.
### Why are the changes needed?
Better control of session lifecycle.
### Does this PR introduce _any_ user-facing change?
Changes the internal controls of keeping the Spark Connect session alive. The default should be close to what it was previously.
### How was this patch tested?
WIP
### Was this patch authored or co-authored using generative AI tooling?
Github Copilot was assisting in some boilerplate auto-completion.
Generated-by: Github Copilot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [WIP][SPARK-46011][CONNECT] Session heartbeat/keepalive using ExtendSession RPC [spark]
Posted by "juliuszsompolski (via GitHub)" <gi...@apache.org>.
juliuszsompolski closed pull request #43913: [WIP][SPARK-46011][CONNECT] Session heartbeat/keepalive using ExtendSession RPC
URL: https://github.com/apache/spark/pull/43913
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [WIP][SPARK-46011][CONNECT] Session heartbeat/keepalive using ExtendSession RPC [spark]
Posted by "juliuszsompolski (via GitHub)" <gi...@apache.org>.
juliuszsompolski commented on PR #43913:
URL: https://github.com/apache/spark/pull/43913#issuecomment-1824802232
Upon discussions with @hvanhovell @grundprinzip @rangadi, we don't want to change the semantics of the session extension with an extra ExtendSession RPC at this point, but we do want to have some extra flexibility in e.g. setting per session timeouts that the guava cache does not give. I will close this PR and take pieces of it that refactor the session manager into a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org