[jira] [Resolved] (TEZ-1086) Allow Tez sessions to decay smoothly instead of timeouts

["Gopal V (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gopal V resolved TEZ-1086.
--------------------------
    Resolution: Not a Problem

min-held containers fix provides a cleaner fix to the this problem.

> Allow Tez sessions to decay smoothly instead of timeouts
> --------------------------------------------------------
>
>                 Key: TEZ-1086
>                 URL: https://issues.apache.org/jira/browse/TEZ-1086
>             Project: Apache Tez
>          Issue Type: New Feature
>            Reporter: Gopal V
>
> With hiveserver2's session sharing, we hit a problem with session container delays.
> If the timeout is too small, like 2 seconds, there is no real reuse of containers, even for small queries which hit the server.
> If the timeout is too big, like 2 minutes, then a big query will hold onto idle containers for way too long causing smaller queries to stall.
> An ideal middle ground would be to keep some containers around, while getting rid of a majority of containers owned by the session.
> A simple heuristic could be to set the fractional-life of a session container to 2 seconds.
> If the fraction is 0.5 (i.e half-life), then 50% of the idle capacity would become available for the small query in 2 seconds.
> This could work better for the JIT as the mechanism can sort containers by their launchtime and kill off the newer ones always (or pick 1 per node to keep).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)