You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org> on 2016/09/01 19:44:48 UTC
[Impala-CR] IMPALA-3611: track unused Disk IO buffer memory
Hello Internal Jenkins, Dan Hecht,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/4208
to review the following change.
Change subject: IMPALA-3611: track unused Disk IO buffer memory
......................................................................
IMPALA-3611: track unused Disk IO buffer memory
Track I/O buffers against separate MemTrackers. This gives us better
visibility into memory consumption from the debug webpage and from
MemTracker consumption dumps. The immediate motivation was in trying to
determine whether idle memory consumption of an impalad was caused by a
memory leak.
We add two trackers: for buffers cached in DiskIoMgr's free list,
and another for clients that don't provide a MemTracker (the only
one is BufferedBlockMgr, which will be removed at some point).
The previous code "tracked" the buffers against the process-wide
tracker, but it was a no-op outside of ASAN builds since the
process-wide tracker took its value from TCMalloc.
The test code required fixing because it assumed that buffers were
always credited against the DiskIoMgr's tracker. This only made sense
when the DiskIoMgr's tracker is the root process-wide tracker.
Fix backend test logging for disk-io-mgr-test.
Testing:
Ran exhaustive tests.
Change-Id: I8777cf76f04d34a46f53d53005412e0f1d63b5b7
Reviewed-on: http://gerrit.cloudera.org:8080/3799
Reviewed-by: Dan Hecht <dh...@cloudera.com>
Tested-by: Internal Jenkins
(cherry picked from commit 17bf14417e3438d772b19111431453bdd537742a)
---
M be/src/runtime/disk-io-mgr-scan-range.cc
M be/src/runtime/disk-io-mgr-test.cc
M be/src/runtime/disk-io-mgr.cc
M be/src/runtime/disk-io-mgr.h
M be/src/runtime/row-batch.cc
5 files changed, 242 insertions(+), 168 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/08/4208/1
--
To view, visit http://gerrit.cloudera.org:8080/4208
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8777cf76f04d34a46f53d53005412e0f1d63b5b7
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: master
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-CR] IMPALA-3611: track unused Disk IO buffer memory
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has abandoned this change.
Change subject: IMPALA-3611: track unused Disk IO buffer memory
......................................................................
Abandoned
--
To view, visit http://gerrit.cloudera.org:8080/4208
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: abandon
Gerrit-Change-Id: I8777cf76f04d34a46f53d53005412e0f1d63b5b7
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: master
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>