You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Ádám Szita (Jira)" <ji...@apache.org> on 2021/02/04 14:39:00 UTC
[jira] [Commented] (HIVE-22492) Amortize lock contention due to
LRFU accounting
[ https://issues.apache.org/jira/browse/HIVE-22492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278872#comment-17278872 ]
Ádám Szita commented on HIVE-22492:
-----------------------------------
Hey [~bslim] - can you please elaborate on what was the justification to set the lambda to 0.1 as part of this change?
( [https://github.com/apache/hive/commit/0781cf2c5104dafd0c5496631cafabac9d59df67#diff-b7bbe8545a21ec7d7e9cfe40ef66444789e332996aaa9e7f1430dbe4822a2c9cR4123] )
> Amortize lock contention due to LRFU accounting
> -----------------------------------------------
>
> Key: HIVE-22492
> URL: https://issues.apache.org/jira/browse/HIVE-22492
> Project: Hive
> Issue Type: Improvement
> Reporter: Slim Bouguerra
> Assignee: Slim Bouguerra
> Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22492.2.patch, HIVE-22492.patch, llap-lock-contention.svg
>
>
> LRFU eviction policy can be a major source of contention under high load.
> This can be see on the following profiles.
> To fix this the idea is to use a batching wrapper to amortize the locking contention.
> The trick i a common way to amortize locking as explained here http://www.ece.eng.wayne.edu/~sjiang/pubs/papers/ding-09-BP-Wrapper.pdf
--
This message was sent by Atlassian Jira
(v8.3.4#803005)