You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Weston Pace (Jira)" <ji...@apache.org> on 2021/05/04 11:25:00 UTC
[jira] [Commented] (ARROW-9530) [C++] Add option to disable
jemalloc background thread on Linux
[ https://issues.apache.org/jira/browse/ARROW-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338920#comment-17338920 ]
Weston Pace commented on ARROW-9530:
------------------------------------
I've submitted a PR. It doesn't allow you to prevent the background threads from starting up at all but you can shut them down which should address both the valgrind issue and any concerns about performance impact of those threads.
> [C++] Add option to disable jemalloc background thread on Linux
> ---------------------------------------------------------------
>
> Key: ARROW-9530
> URL: https://issues.apache.org/jira/browse/ARROW-9530
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Rob Ambalu
> Assignee: Weston Pace
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We noticed that after we upgraded pyarrow to version 0.17.1 from 0.9.0 we now get two unwanted side effects just by linking in arrow libs into our c++ build. We link arrow libs into one of our low level core libraries so now all of our applications are affected by these side effects:
> 1) a "jemalloc_bg_thd" thread is spawned on dlinit before we ever hit main
> 2) all our apps are now hitting valgrind leak warnings due to a (potential) leak in jemalloc code:
> ==33515== 656 bytes in 1 blocks are possibly lost in loss record 1 of 1
> ==33515== at 0x402E9EA: calloc (vg_replace_malloc.c:752)
> ==33515== by 0x4011F44: _dl_allocate_tls (in /usr/lib64/ld-2.17.so)
> ==33515== by 0x5DFF9C0: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
> ==33515== by 0x589186B: je_arrow_private_je_pthread_create_wrapper (background_thread.c:48)
> ==33515== by 0x589186B: background_thread_create_signals_masked (background_thread.c:365)
> ==33515== by 0x589186B: background_thread_create_locked (background_thread.c:573)
> ==33515== by 0x5891A47: je_arrow_private_je_background_thread_create (background_thread.c:598)
> ==33515== by 0x400F502: _dl_init (in /usr/lib64/ld-2.17.so)
> ==33515== by 0x40011A9: ??? (in /usr/lib64/ld-2.17.so)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)