[jira] [Resolved] (HIVE-10092) LLAP: improve how buffers are locked for split

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sergey Shelukhin resolved HIVE-10092.
-------------------------------------
       Resolution: Fixed
    Fix Version/s: llap

[~gopalv] fyi

> LLAP: improve how buffers are locked for split
> ----------------------------------------------
>
>                 Key: HIVE-10092
>                 URL: https://issues.apache.org/jira/browse/HIVE-10092
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>             Fix For: llap
>
>         Attachments: HIVE-10092.patch
>
>
> Right now, for simplicity, entire split of decompressed buffers is locked in cache, in case some buffers are shared between RGs, to avoid dealing with situations where we uncompress some data, pass it on to processor for RG N, then processor processes and unlocks it, and before we can pass it on for RG N+1 it's evicted. 
> However, if split is too big, and cache is small, or many splits are processed at the same time, this can result in a deadlock as entire cache is locked. We need to improve locking to be more granular and probably also try to avoid deadlocks in general (bypass cache?)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)