You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/06/12 22:41:00 UTC
[jira] [Commented] (IMPALA-7145) Leak of memory from OpenSSL when
spill-to-disk encryption is enabled
[ https://issues.apache.org/jira/browse/IMPALA-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510332#comment-16510332 ]
ASF subversion and git services commented on IMPALA-7145:
---------------------------------------------------------
Commit 501dc6abfb360575613eb2210c0c87f3e37235ce in impala's branch refs/heads/master from [~tarmstrong@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=501dc6a ]
IMPALA-7145: fix leak of OpenSSL context when spilling
Add a RAII wrapper for the OpenSSL context that automatically frees
on all exit paths from the function.
Add a backend test wrapper that enables LeakSanitizer for an individual
test. This is a step towards IMPALA-2746.
Fix version check bug in asan.h.
Testing:
Enable LeakSanitizer for openssl-util-test. This reliably found the bug.
Ran core tests under ASAN.
Change-Id: I98760ed8f31b18b489a156f945c29c95c9bf3184
Reviewed-on: http://gerrit.cloudera.org:8080/10666
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Leak of memory from OpenSSL when spill-to-disk encryption is enabled
> --------------------------------------------------------------------
>
> Key: IMPALA-7145
> URL: https://issues.apache.org/jira/browse/IMPALA-7145
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.0, Impala 2.3.0, Impala 2.5.0, Impala 2.4.0, Impala 2.6.0, Impala 2.7.0, Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0, Impala 2.12.0, Impala 2.13.0, Impala 3.1.0, 2.2.0, 2.1.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Blocker
> Labels: resource-management
>
> I tried looping this test for a while to flush out a different issue and discovered a memory leak:
> {noformat}
> while impala-py.test tests/query_test/test_spilling.py -k naaj -n4 --verbose; do date; done
> {noformat}
> Eventually the test fails with "Memory Limit Exceeded" and there is a huge amount of untracked memory reported on /memz.
> I tried running the heap growth profiler and the only suspicious thing I saw was CRYPTO_malloc, similar to IMPALA-6792. My current hypothesis is that it's something related to disk spill encryption and we haven't noticed it before because we had disk spill encryption off.
> If I disable encryption "start-impala-cluster.py --impalad_args=--disk_spill_encryption=false" then I don't see the leak.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org