[jira] [Resolved] (IMPALA-3259) Codegen is not cancellable and can use a lot of CPU and memory

["Tim Armstrong (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-3259.
-----------------------------------
    Resolution: Fixed

This has been mitigated by the various fixes that we've done to improve this. Will call it done for now.

> Codegen is not cancellable and can use a lot of CPU and memory
> --------------------------------------------------------------
>
>                 Key: IMPALA-3259
>                 URL: https://issues.apache.org/jira/browse/IMPALA-3259
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 2.5.0
>            Reporter: Huaisi Xu
>            Priority: Major
>              Labels: codegen
>         Attachments: screenshot-1.png
>
>
> memz stucks at 
> {code:java}
> Memory Usage
> Memory consumption / limit: 10.43 GB / 20.00 GB
> tcmalloc
> ------------------------------------------------
> MALLOC:    10489890000 (10003.9 MiB) Bytes in use by application
> MALLOC: +            0 (    0.0 MiB) Bytes in page heap freelist
> MALLOC: +    674292952 (  643.1 MiB) Bytes in central cache freelist
> MALLOC: +     15303600 (   14.6 MiB) Bytes in transfer cache freelist
> MALLOC: +     15594152 (   14.9 MiB) Bytes in thread cache freelists
> MALLOC: +     51245248 (   48.9 MiB) Bytes in malloc metadata
> MALLOC:   ------------
> MALLOC: =  11246325952 (10725.3 MiB) Actual memory used (physical + swap)
> MALLOC: +   6235471872 ( 5946.6 MiB) Bytes released to OS (aka unmapped)
> MALLOC:   ------------
> MALLOC: =  17481797824 (16671.9 MiB) Virtual address space used
> MALLOC:
> MALLOC:         565828              Spans in use
> MALLOC:            245              Thread heaps in use
> MALLOC:           8192              Tcmalloc page size
> ------------------------------------------------
> Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).
> Bytes released to the OS take up virtual address space but no physical memory.
> {code}
> this query is cancelled because of unreachable impalad because I attached gdb..
> I have flushed the query archiving cache by issuing ~30 "show tables" queries. (this works in IMPALA-3254) but the memory in this case stayed high.
> I will try to reproduce and get a heap profile.
> There is no query running at this moment.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)