You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/07/27 14:59:00 UTC
[jira] [Work logged] (HIVE-26428) Limit usage of LLAP BPWrapper to threads of IO threadpools
[ https://issues.apache.org/jira/browse/HIVE-26428?focusedWorklogId=795696&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795696 ]
ASF GitHub Bot logged work on HIVE-26428:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Jul/22 14:58
Start Date: 27/Jul/22 14:58
Worklog Time Spent: 10m
Work Description: szlta opened a new pull request, #3481:
URL: https://github.com/apache/hive/pull/3481
BPWrapper is used in LRFU cache eviction policy to decrease the time spent waiting for lock on the heap. This is done by adding a buffer as threadlocal and accumulating CacheableBuffer instances there before trying to acquire a lock. This works well when we have threads from pools such as IO-Elevator threads or OrcEncode threads.
For ephemeral threads there's no advantage of doing this as the buffers in threadlocals may never reach the heap or list structures of LRFU, thereby also making evictions less efficient. This can happen e.g. LLAPCacheAwareFS is used with Parquet, where we're using the Tez threads for both execution and IO.
We should disable BPWrappers for such cases.
Issue Time Tracking
-------------------
Worklog Id: (was: 795696)
Remaining Estimate: 0h
Time Spent: 10m
> Limit usage of LLAP BPWrapper to threads of IO threadpools
> ----------------------------------------------------------
>
> Key: HIVE-26428
> URL: https://issues.apache.org/jira/browse/HIVE-26428
> Project: Hive
> Issue Type: Improvement
> Reporter: Ádám Szita
> Assignee: Ádám Szita
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> BPWrapper is used in LRFU cache eviction policy to decrease the time spent waiting for lock on the heap. This is done by adding a buffer as threadlocal and accumulating CacheableBuffer instances there before trying to acquire a lock. This works well when we have threads from pools such as IO-Elevator threads or OrcEncode threads.
> For ephemeral threads there's no advantage of doing this as the buffers in threadlocals may never reach the heap or list structures of LRFU, thereby also making evictions less efficient. This can happen e.g. LLAPCacheAwareFS is used with Parquet, where we're using the Tez threads for both execution and IO.
> We should disable BPWrappers for such cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)