[jira] [Commented] (ARROW-9530) jemalloc autoinit on dl init

["Sebastian Dressler (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248590#comment-17248590 ] 

Sebastian Dressler commented on ARROW-9530:
-------------------------------------------

[~uwe]: IMHO that statement is not correct, since {{arrow/cpp/src/arrow/memory_pool.cc}} defines:

{code}
#undef USE_JEMALLOC_BACKGROUND_THREAD
#ifndef __APPLE__
// ARROW-6977: jemalloc's background_thread isn't always enabled on macOS
#define USE_JEMALLOC_BACKGROUND_THREAD
#endif
{code}

I agree with [~robambalu] and think there should be an option to disable these on {{_dl_init}}. Please also see [this comment|https://github.com/jemalloc/jemalloc/issues/1884#issuecomment-662740143] from David Goldblatt on the very same subject where he clearly states:

{quote}Background threads are off by default in the standard configuration{quote}

I am currently using libarrow for a project with PostgreSQL where multiple processes are used per default and I would like to see these background threads gone. Most notably because I am specifically using {{arrow::system_memory_pool}} and disable threads wherever possible. Still, there are threads spawned.

> jemalloc autoinit on dl init
> ----------------------------
>
>                 Key: ARROW-9530
>                 URL: https://issues.apache.org/jira/browse/ARROW-9530
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>            Reporter: Rob Ambalu
>            Priority: Minor
>
> We noticed that after we upgraded pyarrow to version 0.17.1 from 0.9.0 we now get two unwanted side effects just by linking in arrow libs into our c++ build.  We link arrow libs into one of our low level core libraries so now all of our applications are affected by these side effects:
> 1) a "jemalloc_bg_thd" thread is spawned on dlinit before we ever hit main
> 2) all our apps are now hitting valgrind leak warnings due to a (potential) leak in jemalloc code:
> ==33515== 656 bytes in 1 blocks are possibly lost in loss record 1 of 1
> ==33515== at 0x402E9EA: calloc (vg_replace_malloc.c:752)
> ==33515== by 0x4011F44: _dl_allocate_tls (in /usr/lib64/ld-2.17.so)
> ==33515== by 0x5DFF9C0: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
> ==33515== by 0x589186B: je_arrow_private_je_pthread_create_wrapper (background_thread.c:48)
> ==33515== by 0x589186B: background_thread_create_signals_masked (background_thread.c:365)
> ==33515== by 0x589186B: background_thread_create_locked (background_thread.c:573)
> ==33515== by 0x5891A47: je_arrow_private_je_background_thread_create (background_thread.c:598)
> ==33515== by 0x400F502: _dl_init (in /usr/lib64/ld-2.17.so)
> ==33515== by 0x40011A9: ??? (in /usr/lib64/ld-2.17.so)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)