You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Bikramjeet Vig (JIRA)" <ji...@apache.org> on 2017/12/14 18:38:00 UTC
[jira] [Resolved] (IMPALA-5848) Account for TCMalloc overhead and
client cache buffers in MemTracker
[ https://issues.apache.org/jira/browse/IMPALA-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bikramjeet Vig resolved IMPALA-5848.
------------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.12.0
https://github.com/apache/impala/commit/9a6e5fa99650c4d916aa577742a7758af53f14ee
IMPALA-5848: Account for TCMalloc overhead in MemTracker
This patch adds a new MemTracker under the Process MemTracker called
"TCMalloc Overhead" which accounts for different cache freelists
maintained by TCMalloc. This added accounting also helps bring down
the amount of untracked memory.
An example dump of the Process MemTracker now looks like:
Process: Limit=8.34 GB Total=119.10 MB Peak=119.10 MB
Buffer Pool: Free Buffers: Total=0
Buffer Pool: Clean Pages: Total=0
Buffer Pool: Unused Reservation: Total=0
TCMalloc Overhead: Total=11.42 MB
Untracked Memory: Total=107.69 MB
Testing:
Tested manually by checking the memz webpage.
Change-Id: I602e9d5e8e8d7470dcfe4addde3265057c16263a
Reviewed-on: http://gerrit.cloudera.org:8080/8782
Reviewed-by: Bikramjeet Vig <bi...@cloudera.com>
Tested-by: Impala Public Jenkins
> Account for TCMalloc overhead and client cache buffers in MemTracker
> --------------------------------------------------------------------
>
> Key: IMPALA-5848
> URL: https://issues.apache.org/jira/browse/IMPALA-5848
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.10.0
> Reporter: Tim Armstrong
> Assignee: Bikramjeet Vig
> Labels: resource-management
> Fix For: Impala 2.12.0
>
>
> On our stress test cluster I see a significant amount of Untracked memory. I think we can account for a chunk of it, which will further help understanding the state of a cluster.
> I see:
> {code}
> Untracked Memory: Total=4.30 GB
> {code}
> I think some of this memory is TCMalloc caches and metadata (I'm not sure exactly which is counted)
> {code}
> MALLOC: 61821970968 (58958.0 MiB) Bytes in use by application
> MALLOC: + 0 ( 0.0 MiB) Bytes in page heap freelist
> MALLOC: + 808683664 ( 771.2 MiB) Bytes in central cache freelist
> MALLOC: + 3794688 ( 3.6 MiB) Bytes in transfer cache freelist
> MALLOC: + 520586840 ( 496.5 MiB) Bytes in thread cache freelists
> MALLOC: + 182366400 ( 173.9 MiB) Bytes in malloc metadata
> MALLOC: ------------
> MALLOC: = 63337402560 (60403.3 MiB) Actual memory used (physical + swap)
> MALLOC: + 45341532160 (43241.1 MiB) Bytes released to OS (aka unmapped)
> MALLOC: ------------
> MALLOC: = 108678934720 (103644.3 MiB) Virtual address space used
> MALLOC:
> MALLOC: 368033 Spans in use
> MALLOC: 3511 Thread heaps in use
> MALLOC: 8192 Tcmalloc page size
> ------------------------------------------------
> Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).
> Bytes released to the OS take up virtual address space but no physical memory.
> {code}
> The client cache also is likely taking up a significant amount of memory, since I believe the clients all have buffers attached:
> {code}
> impala-server.backends.client-cache.total-clients 1006
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)