You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by "Matei Zaharia (Updated) (JIRA)" <ji...@apache.org> on 2012/03/12 22:14:38 UTC
[jira] [Updated] (MESOS-164) Crash on Mac OS X due to dlopen not
being thread-safe
[ https://issues.apache.org/jira/browse/MESOS-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matei Zaharia updated MESOS-164:
--------------------------------
Description:
I get a crash fairly deterministically that seems to be due to the Python library calling dlopen() in a separate thread from the main thread. I've included a stack trace in GDB below. The reason I think it's due to thread unsafety is the comments on http://stackoverflow.com/questions/8744608/dlopen-in-multithreaded-application-exit-with-trace-bpt-trap.
matei@dhcp-44-161:~/workspace/mesos/build$ bin/mesos-master.sh
I0312 14:12:58.237947 1959004512 logging.cpp:70] Logging to /var/tmp/mesos-dev-logs
I0312 14:12:58.245913 1959004512 main.cpp:97] Build: 2012-03-12 14:12:17 by matei
I0312 14:12:58.245970 1959004512 main.cpp:98] Starting Mesos master
I0312 14:12:58.246098 61464576 master.cpp:266] Master started at mesos://master@128.32.44.161:5050
I0312 14:12:58.246271 61464576 master.cpp:281] Master ID: 201203121412-1590943616-5050-74763
I0312 14:12:58.247436 61464576 master.cpp:483] Elected as master!
I0312 14:12:58.275187 63610880 webui_utils.cpp:45] Loading webui script at '/Users/matei/workspace/mesos/build/../src/webui/master/webui.py'
Trace/BPT trap: 5
#0 0x00007fff8a57b1cf in __CFInitialize ()
#1 0x00007fff5fc0fe6f in __dyld__ZN16ImageLoaderMachO11doImageInitERKN11ImageLoader11LinkContextE ()
#2 0x00007fff5fc0fae7 in __dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkContextE ()
#3 0x00007fff5fc0d2e4 in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
#4 0x00007fff5fc0d27d in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
#5 0x00007fff5fc0d27d in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
#6 0x00007fff5fc0e0b7 in __dyld__ZN11ImageLoader15runInitializersERKNS_11LinkContextERNS_21InitializerTimingListE ()
#7 0x00007fff5fc031b9 in __dyld__ZN4dyld15runInitializersEP11ImageLoader ()
#8 0x00007fff5fc09657 in __dyld_dlopen ()
#9 0x00007fff8b16e95b in dlopen ()
#10 0x00000001000cf8c5 in _PyImport_GetDynLoadFunc ()
#11 0x00000001000bba96 in _PyImport_LoadDynamicModule ()
#12 0x00000001000ba246 in PyImport_ReloadModule ()
#13 0x00000001000ba454 in PyImport_ReloadModule ()
#14 0x00000001000bb201 in PyImport_ImportModuleLevel ()
#15 0x000000010009f506 in _PyBuiltin_Init ()
#16 0x0000000100027d32 in PyObject_Call ()
#17 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
#18 0x00000001000a98d9 in PyEval_EvalFrameEx ()
#19 0x00000001000aacd8 in PyEval_EvalCodeEx ()
#20 0x00000001000aad4d in PyEval_EvalCode ()
#21 0x00000001000b8f3a in PyImport_ExecCodeModuleEx ()
#22 0x00000001000b9482 in PyImport_ExecCodeModuleEx ()
#23 0x00000001000ba246 in PyImport_ReloadModule ()
#24 0x00000001000ba454 in PyImport_ReloadModule ()
#25 0x00000001000bb201 in PyImport_ImportModuleLevel ()
#26 0x000000010009f506 in _PyBuiltin_Init ()
#27 0x0000000100027d32 in PyObject_Call ()
#28 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
#29 0x00000001000a98d9 in PyEval_EvalFrameEx ()
#30 0x00000001000aacd8 in PyEval_EvalCodeEx ()
#31 0x00000001000aad4d in PyEval_EvalCode ()
#32 0x00000001000b8f3a in PyImport_ExecCodeModuleEx ()
#33 0x00000001000b9482 in PyImport_ExecCodeModuleEx ()
#34 0x00000001000ba246 in PyImport_ReloadModule ()
#35 0x00000001000ba454 in PyImport_ReloadModule ()
#36 0x00000001000bb201 in PyImport_ImportModuleLevel ()
#37 0x000000010009f506 in _PyBuiltin_Init ()
#38 0x0000000100027d32 in PyObject_Call ()
#39 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
#40 0x00000001000a98d9 in PyEval_EvalFrameEx ()
#41 0x00000001000aacd8 in PyEval_EvalCodeEx ()
#42 0x00000001000aad4d in PyEval_EvalCode ()
#43 0x00000001000b8f3a in PyImport_ExecCodeModuleEx ()
#44 0x00000001000b9482 in PyImport_ExecCodeModuleEx ()
#45 0x00000001000ba246 in PyImport_ReloadModule ()
#46 0x00000001000ba454 in PyImport_ReloadModule ()
#47 0x00000001000bb201 in PyImport_ImportModuleLevel ()
#48 0x000000010009f506 in _PyBuiltin_Init ()
#49 0x0000000100027d32 in PyObject_Call ()
#50 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
#51 0x00000001000a98d9 in PyEval_EvalFrameEx ()
#52 0x00000001000aacd8 in PyEval_EvalCodeEx ()
#53 0x00000001000aad4d in PyEval_EvalCode ()
#54 0x00000001000c208f in Py_CompileString ()
#55 0x00000001000c214f in PyRun_FileExFlags ()
#56 0x00000001000c32a2 in PyRun_SimpleFileExFlags ()
#57 0x0000000100005dde in run (directory=@0x101a044b8, script=@0x100bd5dd0, args=@0x7fff74c3ff78) at webui_utils.cpp:48
#58 0x0000000100005fc9 in thread::__run (arg=0x101a4d750) at functional_iterate.h:865
#59 0x00007fff851678bf in _pthread_start ()
#60 0x00007fff8516ab75 in thread_start ()
was:
I get a crash fairly deterministically that seems to be due to the Python library calling dlopen() in a separate thread from the main thread. I've included a stack trace in GDB below. The reason I think it's due to thread unsafety is the comments on http://stackoverflow.com/questions/8744608/dlopen-in-multithreaded-application-exit-with-trace-bpt-trap.
#0 0x00007fff8a57b1cf in __CFInitialize ()
#1 0x00007fff5fc0fe6f in __dyld__ZN16ImageLoaderMachO11doImageInitERKN11ImageLoader11LinkContextE ()
#2 0x00007fff5fc0fae7 in __dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkContextE ()
#3 0x00007fff5fc0d2e4 in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
#4 0x00007fff5fc0d27d in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
#5 0x00007fff5fc0d27d in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
#6 0x00007fff5fc0e0b7 in __dyld__ZN11ImageLoader15runInitializersERKNS_11LinkContextERNS_21InitializerTimingListE ()
#7 0x00007fff5fc031b9 in __dyld__ZN4dyld15runInitializersEP11ImageLoader ()
#8 0x00007fff5fc09657 in __dyld_dlopen ()
#9 0x00007fff8b16e95b in dlopen ()
#10 0x00000001000cf8c5 in _PyImport_GetDynLoadFunc ()
#11 0x00000001000bba96 in _PyImport_LoadDynamicModule ()
#12 0x00000001000ba246 in PyImport_ReloadModule ()
#13 0x00000001000ba454 in PyImport_ReloadModule ()
#14 0x00000001000bb201 in PyImport_ImportModuleLevel ()
#15 0x000000010009f506 in _PyBuiltin_Init ()
#16 0x0000000100027d32 in PyObject_Call ()
#17 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
#18 0x00000001000a98d9 in PyEval_EvalFrameEx ()
#19 0x00000001000aacd8 in PyEval_EvalCodeEx ()
#20 0x00000001000aad4d in PyEval_EvalCode ()
#21 0x00000001000b8f3a in PyImport_ExecCodeModuleEx ()
#22 0x00000001000b9482 in PyImport_ExecCodeModuleEx ()
#23 0x00000001000ba246 in PyImport_ReloadModule ()
#24 0x00000001000ba454 in PyImport_ReloadModule ()
#25 0x00000001000bb201 in PyImport_ImportModuleLevel ()
#26 0x000000010009f506 in _PyBuiltin_Init ()
#27 0x0000000100027d32 in PyObject_Call ()
#28 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
#29 0x00000001000a98d9 in PyEval_EvalFrameEx ()
#30 0x00000001000aacd8 in PyEval_EvalCodeEx ()
#31 0x00000001000aad4d in PyEval_EvalCode ()
#32 0x00000001000b8f3a in PyImport_ExecCodeModuleEx ()
#33 0x00000001000b9482 in PyImport_ExecCodeModuleEx ()
#34 0x00000001000ba246 in PyImport_ReloadModule ()
#35 0x00000001000ba454 in PyImport_ReloadModule ()
#36 0x00000001000bb201 in PyImport_ImportModuleLevel ()
#37 0x000000010009f506 in _PyBuiltin_Init ()
#38 0x0000000100027d32 in PyObject_Call ()
#39 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
#40 0x00000001000a98d9 in PyEval_EvalFrameEx ()
#41 0x00000001000aacd8 in PyEval_EvalCodeEx ()
#42 0x00000001000aad4d in PyEval_EvalCode ()
#43 0x00000001000b8f3a in PyImport_ExecCodeModuleEx ()
#44 0x00000001000b9482 in PyImport_ExecCodeModuleEx ()
#45 0x00000001000ba246 in PyImport_ReloadModule ()
#46 0x00000001000ba454 in PyImport_ReloadModule ()
#47 0x00000001000bb201 in PyImport_ImportModuleLevel ()
#48 0x000000010009f506 in _PyBuiltin_Init ()
#49 0x0000000100027d32 in PyObject_Call ()
#50 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
#51 0x00000001000a98d9 in PyEval_EvalFrameEx ()
#52 0x00000001000aacd8 in PyEval_EvalCodeEx ()
#53 0x00000001000aad4d in PyEval_EvalCode ()
#54 0x00000001000c208f in Py_CompileString ()
#55 0x00000001000c214f in PyRun_FileExFlags ()
#56 0x00000001000c32a2 in PyRun_SimpleFileExFlags ()
#57 0x0000000100005dde in run (directory=@0x101a044b8, script=@0x100bd5dd0, args=@0x7fff74c3ff78) at webui_utils.cpp:48
#58 0x0000000100005fc9 in thread::__run (arg=0x101a4d750) at functional_iterate.h:865
#59 0x00007fff851678bf in _pthread_start ()
#60 0x00007fff8516ab75 in thread_start ()
> Crash on Mac OS X due to dlopen not being thread-safe
> -----------------------------------------------------
>
> Key: MESOS-164
> URL: https://issues.apache.org/jira/browse/MESOS-164
> Project: Mesos
> Issue Type: Bug
> Environment: Mac OS X 10.7.3
> Reporter: Matei Zaharia
>
> I get a crash fairly deterministically that seems to be due to the Python library calling dlopen() in a separate thread from the main thread. I've included a stack trace in GDB below. The reason I think it's due to thread unsafety is the comments on http://stackoverflow.com/questions/8744608/dlopen-in-multithreaded-application-exit-with-trace-bpt-trap.
> matei@dhcp-44-161:~/workspace/mesos/build$ bin/mesos-master.sh
> I0312 14:12:58.237947 1959004512 logging.cpp:70] Logging to /var/tmp/mesos-dev-logs
> I0312 14:12:58.245913 1959004512 main.cpp:97] Build: 2012-03-12 14:12:17 by matei
> I0312 14:12:58.245970 1959004512 main.cpp:98] Starting Mesos master
> I0312 14:12:58.246098 61464576 master.cpp:266] Master started at mesos://master@128.32.44.161:5050
> I0312 14:12:58.246271 61464576 master.cpp:281] Master ID: 201203121412-1590943616-5050-74763
> I0312 14:12:58.247436 61464576 master.cpp:483] Elected as master!
> I0312 14:12:58.275187 63610880 webui_utils.cpp:45] Loading webui script at '/Users/matei/workspace/mesos/build/../src/webui/master/webui.py'
> Trace/BPT trap: 5
> #0 0x00007fff8a57b1cf in __CFInitialize ()
> #1 0x00007fff5fc0fe6f in __dyld__ZN16ImageLoaderMachO11doImageInitERKN11ImageLoader11LinkContextE ()
> #2 0x00007fff5fc0fae7 in __dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkContextE ()
> #3 0x00007fff5fc0d2e4 in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
> #4 0x00007fff5fc0d27d in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
> #5 0x00007fff5fc0d27d in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
> #6 0x00007fff5fc0e0b7 in __dyld__ZN11ImageLoader15runInitializersERKNS_11LinkContextERNS_21InitializerTimingListE ()
> #7 0x00007fff5fc031b9 in __dyld__ZN4dyld15runInitializersEP11ImageLoader ()
> #8 0x00007fff5fc09657 in __dyld_dlopen ()
> #9 0x00007fff8b16e95b in dlopen ()
> #10 0x00000001000cf8c5 in _PyImport_GetDynLoadFunc ()
> #11 0x00000001000bba96 in _PyImport_LoadDynamicModule ()
> #12 0x00000001000ba246 in PyImport_ReloadModule ()
> #13 0x00000001000ba454 in PyImport_ReloadModule ()
> #14 0x00000001000bb201 in PyImport_ImportModuleLevel ()
> #15 0x000000010009f506 in _PyBuiltin_Init ()
> #16 0x0000000100027d32 in PyObject_Call ()
> #17 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
> #18 0x00000001000a98d9 in PyEval_EvalFrameEx ()
> #19 0x00000001000aacd8 in PyEval_EvalCodeEx ()
> #20 0x00000001000aad4d in PyEval_EvalCode ()
> #21 0x00000001000b8f3a in PyImport_ExecCodeModuleEx ()
> #22 0x00000001000b9482 in PyImport_ExecCodeModuleEx ()
> #23 0x00000001000ba246 in PyImport_ReloadModule ()
> #24 0x00000001000ba454 in PyImport_ReloadModule ()
> #25 0x00000001000bb201 in PyImport_ImportModuleLevel ()
> #26 0x000000010009f506 in _PyBuiltin_Init ()
> #27 0x0000000100027d32 in PyObject_Call ()
> #28 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
> #29 0x00000001000a98d9 in PyEval_EvalFrameEx ()
> #30 0x00000001000aacd8 in PyEval_EvalCodeEx ()
> #31 0x00000001000aad4d in PyEval_EvalCode ()
> #32 0x00000001000b8f3a in PyImport_ExecCodeModuleEx ()
> #33 0x00000001000b9482 in PyImport_ExecCodeModuleEx ()
> #34 0x00000001000ba246 in PyImport_ReloadModule ()
> #35 0x00000001000ba454 in PyImport_ReloadModule ()
> #36 0x00000001000bb201 in PyImport_ImportModuleLevel ()
> #37 0x000000010009f506 in _PyBuiltin_Init ()
> #38 0x0000000100027d32 in PyObject_Call ()
> #39 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
> #40 0x00000001000a98d9 in PyEval_EvalFrameEx ()
> #41 0x00000001000aacd8 in PyEval_EvalCodeEx ()
> #42 0x00000001000aad4d in PyEval_EvalCode ()
> #43 0x00000001000b8f3a in PyImport_ExecCodeModuleEx ()
> #44 0x00000001000b9482 in PyImport_ExecCodeModuleEx ()
> #45 0x00000001000ba246 in PyImport_ReloadModule ()
> #46 0x00000001000ba454 in PyImport_ReloadModule ()
> #47 0x00000001000bb201 in PyImport_ImportModuleLevel ()
> #48 0x000000010009f506 in _PyBuiltin_Init ()
> #49 0x0000000100027d32 in PyObject_Call ()
> #50 0x00000001000a3c40 in PyEval_CallObjectWithKeywords ()
> #51 0x00000001000a98d9 in PyEval_EvalFrameEx ()
> #52 0x00000001000aacd8 in PyEval_EvalCodeEx ()
> #53 0x00000001000aad4d in PyEval_EvalCode ()
> #54 0x00000001000c208f in Py_CompileString ()
> #55 0x00000001000c214f in PyRun_FileExFlags ()
> #56 0x00000001000c32a2 in PyRun_SimpleFileExFlags ()
> #57 0x0000000100005dde in run (directory=@0x101a044b8, script=@0x100bd5dd0, args=@0x7fff74c3ff78) at webui_utils.cpp:48
> #58 0x0000000100005fc9 in thread::__run (arg=0x101a4d750) at functional_iterate.h:865
> #59 0x00007fff851678bf in _pthread_start ()
> #60 0x00007fff8516ab75 in thread_start ()
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira