You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/02/08 11:29:20 UTC
[GitHub] [hive] szlta commented on a change in pull request #1951: HIVE-24736
szlta commented on a change in pull request #1951:
URL: https://github.com/apache/hive/pull/1951#discussion_r571973578
##########
File path: llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelLrfuCachePolicy.java
##########
@@ -836,13 +912,33 @@ public String description() {
* @return long array with LRFU stats
*/
public long[] getUsageStats() {
- long dataOnHeap = 0L; // all non-meta related buffers on min-heap
- long dataOnList = 0L; // all non-meta related buffers on eviction list
- long metaOnHeap = 0L; // meta data buffers on min-heap
- long metaOnList = 0L; // meta data buffers on eviction list
- long listSize = 0L; // number of entries on eviction list
- long lockedData = 0L; // number of bytes in locked data buffers
- long lockedMeta = 0L; // number of bytes in locked metadata buffers
+ long dataOnHeap = 0L; // all non-meta related buffers on min-heap
+ long dataOnList = 0L; // all non-meta related buffers on eviction list
+ long metaOnHeap = 0L; // meta data buffers on min-heap
+ long metaOnList = 0L; // meta data buffers on eviction list
+ long listSize = 0L; // number of entries on eviction list
+ long lockedData = 0L; // number of bytes in locked data buffers
+ long lockedMeta = 0L; // number of bytes in locked metadata buffers
+ long bpWrapCount = 0L; // number of buffers in BP wrapper threadlocals
+ long bpWrapDistinct = 0L; // number of distinct buffers in BP wrapper threadlocals
+ long bpWrapData = 0L; // number of bytes stored in BP wrapper data buffers
+ long bpWrapMeta = 0L; // number of bytes stored in BP wrapper metadata buffers
+
+ // Using set to produce result of distinct buffers only
+ // (same buffer may be present in multiple thread local bp wrappers, or even inside heap/list, but ultimately
+ // it uses the same cache space)
+ Set<LlapCacheableBuffer> bpWrapperBuffers = new HashSet<>();
+ for (BPWrapper bpWrapper : bpWrappers.values()) {
+ bpWrapper.lock.lock();
Review comment:
It is only called when someone (e.g. cluster admin) asks for memory stats through web UI. Hence it is okay to block all structures for these cases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org