[jira] [Commented] (HIVE-17482) External LLAP client: acquire locks for tables queried directly by LLAP

["Eugene Koifman (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219623#comment-16219623 ] 

Eugene Koifman commented on HIVE-17482:
---------------------------------------

[~jdere]
It occurred to me that this may lead to deadlocks.
Suppose Spark is running a query (S join S).

So the 1st fragment will get a Shared lock on S.  
Then some other query will try an X lock on S and block.
Then the 2nd fragment will try to get an S lock and will block behind the X lock.
Will the 1st frag ever release its' S lock?

> External LLAP client: acquire locks for tables queried directly by LLAP
> -----------------------------------------------------------------------
>
>                 Key: HIVE-17482
>                 URL: https://issues.apache.org/jira/browse/HIVE-17482
>             Project: Hive
>          Issue Type: Sub-task
>          Components: llap
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>             Fix For: 3.0.0
>
>         Attachments: HIVE-17482.1.patch, HIVE-17482.2.patch, HIVE-17482.3.patch, HIVE-17482.4.patch, HIVE-17482.5.patch, HIVE-17482.6.patch
>
>
> When using the LLAP external client with simple queries (filter/project of single table), the appropriate locks should be taken on the table being read like they are for normal Hive queries. This is important in the case of transactional tables being queried, since the compactor relies on the presence of table locks to determine whether it can safely delete old versions of compacted files without affecting currently running queries.
> This does not have to happen in the complex query case, since a query is used (with the appropriate locking mechanisms) to create/populate the temp table holding the results to the complex query.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)