You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@celix.apache.org by GitBox <gi...@apache.org> on 2020/04/07 09:12:39 UTC
[GitHub] [celix] m-veenvliet opened a new issue #188: Deadlock in pubsub zmq
m-veenvliet opened a new issue #188: Deadlock in pubsub zmq
URL: https://github.com/apache/celix/issues/188
When communicating using the pubsub mechanics and the ZeroMQ protocol a deadlock can occur. look at the back trace bellow.
Thread 6 (Thread 0x7fffedff2700 (LWP 27884)):
#0 0x0000003ded00e334 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003ded00aab4 in pthread_rwlock_rdlock () from /lib64/libpthread.so.0
#2 0x00007ffff7db5dab in serviceReference_getServiceRegistration () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#3 0x00007ffff7dc21aa in serviceDependency_invokeRemove () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#4 0x00007ffff7dbfde4 in component_handleEventTask () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#5 0x00007ffff7dbee30 in executor_execute () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#6 0x00007ffff7dc09ff in celix_private_dmComponent_handleEvent () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#7 0x00007ffff7dc1220 in serviceDependency_addedService () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#8 0x00007ffff7db97d3 in serviceTracker_track.isra.1 () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#9 0x00007ffff7db9ac4 in serviceTracker_serviceChanged () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#10 0x00007ffff7db8145 in celix_serviceRegistry_serviceChanged () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#11 0x00007ffff7db82de in serviceRegistry_registerServiceInternal () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#12 0x00007ffff7db8b31 in celix_serviceRegistry_registerServiceFactory () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#13 0x00007ffff7db1631 in celix_framework_registerServiceFactory () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#14 0x00007ffff7dab283 in celix_bundleContext_registerServiceWithOptions () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#15 0x00007fffec5d5530 in pubsub_zmqTopicSender_create () from .cpp_zeromq_subscriber_cache/bundle10/version0.0/libcelix_pubsub_admin_zmq.so.1
#16 0x00007fffec5d39cc in pubsub_zmqAdmin_setupTopicSender () from .cpp_zeromq_subscriber_cache/bundle10/version0.0/libcelix_pubsub_admin_zmq.so.1
#17 0x00007fffedff6b67 in pstm_setupTopicSenderCallback () from .cpp_zeromq_subscriber_cache/bundle7/version0.0/libcelix_pubsub_topology_manager.so.1
#18 0x00007ffff7db923f in serviceTracker_useHighestRankingServiceInternal () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#19 0x00007ffff7dba94e in celix_serviceTracker_useHighestRankingService () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#20 0x00007ffff7dab99f in celix_bundleContext_useServiceWithOptions () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#21 0x00007ffff7daba39 in celix_bundleContext_useServiceWithId () from /celix/linux64_gcc_6_5_0/lib64/libcelix_framework.so.2
#22 0x00007fffedff7125 in pstm_psaHandlingThread () from .cpp_zeromq_subscriber_cache/bundle7/version0.0/libcelix_pubsub_topology_manager.so.1
#23 0x0000003ded007aa1 in start_thread () from /lib64/libpthread.so.0
#24 0x0000003dec4e8c4d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7fffeebfd700 (LWP 27883)):
#0 0x0000003ded00e334 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003ded0095d8 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x0000003ded0094a7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007fffec5d41de in pubsub_zmqAdmin_executeCommand () from .cpp_zeromq_subscriber_cache/bundle10/version0.0/libcelix_pubsub_admin_zmq.so.1
#4 0x00007ffff4dcc576 in shell_executeCommand () from .cpp_zeromq_subscriber_cache/bundle1/version0.0/libshell.so.2
#5 0x00007ffff4391705 in shellTui_runnable () from .cpp_zeromq_subscriber_cache/bundle5/version0.0/libshell_tui.so.1
#6 0x0000003ded007aa1 in start_thread () from /lib64/libpthread.so.0
#7 0x0000003dec4e8c4d in clone () from /lib64/libc.so.6
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [celix] Oipo commented on issue #188: Deadlock in pubsub zmq
Posted by GitBox <gi...@apache.org>.
Oipo commented on issue #188: Deadlock in pubsub zmq
URL: https://github.com/apache/celix/issues/188#issuecomment-614686491
Probably fixed in #199, cannot reproduce.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [celix] Oipo edited a comment on issue #188: Deadlock in pubsub zmq
Posted by GitBox <gi...@apache.org>.
Oipo edited a comment on issue #188: Deadlock in pubsub zmq
URL: https://github.com/apache/celix/issues/188#issuecomment-614686491
Probably fixed in #192, #195 and #199, cannot reproduce.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [celix] mdelangthales commented on issue #188: Deadlock in pubsub
zmq
Posted by GitBox <gi...@apache.org>.
mdelangthales commented on issue #188: Deadlock in pubsub zmq
URL: https://github.com/apache/celix/issues/188#issuecomment-611467008
```
-> ERROR: Cannot find reference 0x60c000029b00 in serviceReferences map
at serviceRegistry_ungetServiceReference(libs/framework/src/service_registry.c:552)
/work/protools/top/nds/gcc/6.5.0/x86_64.el6-linux2.6/lib64/libasan.so(backtrace+0x41) [0x7f0f40f7f491]
libcelix_framework.so.2(+0xf0eec) [0x7f0f40c1deec]
libcelix_framework.so.2(frameworkLogger_log+0xd6) [0x7f0f40c1e7cc]
libcelix_framework.so.2(framework_log+0x210) [0x7f0f40c1e303]
libcelix_framework.so.2(serviceRegistry_ungetServiceReference+0x482) [0x7f0f40c03bc7]
libcelix_framework.so.2(framework_ungetServiceReference+0x7c) [0x7f0f40be7211]
libcelix_framework.so.2(bundleContext_ungetServiceReference+0xdc) [0x7f0f40bd30df]
libcelix_framework.so.2(+0xe0b12) [0x7f0f40c0db12]
libcelix_framework.so.2(serviceTracker_close+0x6ab) [0x7f0f40c0aa4a]
libcelix_framework.so.2(celix_serviceTracker_destroy+0x1f) [0x7f0f40c0fc68]
libcelix_framework.so.2(celix_bundleContext_useServicesWithOptions+0x498) [0x7f0f40bd804f]
.cpp_zeromq_publisher_cache/bundle10/version0.0/libcelix_pubsub_admin_zmq.so.1(+0x4a16f) [0x7f0f3526f16f]
.cpp_zeromq_publisher_cache/bundle10/version0.0/libcelix_pubsub_admin_zmq.so.1(pubsub_utils_matchEndpoint+0xf0) [0x7f0f3526ffd4]
.cpp_zeromq_publisher_cache/bundle10/version0.0/libcelix_pubsub_admin_zmq.so.1(pubsub_zmqAdmin_matchDiscoveredEndpoint+0xff) [0x7f0f3525bc2b]
.cpp_zeromq_publisher_cache/bundle7/version0.0/libcelix_pubsub_topology_manager.so.1(+0x2cc6a) [0x7f0f37743c6a]
.cpp_zeromq_publisher_cache/bundle7/version0.0/libcelix_pubsub_topology_manager.so.1(+0x2f3e4) [0x7f0f377463e4]
/lib64/libpthread.so.0() [0x348f807aa1]
/lib64/libc.so.6(clone+0x6d) [0x348ece8c4d]
=================================================================
==44091==ERROR: AddressSanitizer: heap-use-after-free on address 0x60c000029e38 at pc 0x7f0f40bfb8e8 bp 0x7f0f3770dd20 sp 0x7f0f3770dd18
READ of size 8 at 0x60c000029e38 thread T5
#0 0x7f0f40bfb8e7 in serviceReference_retain libs/framework/src/service_reference.c:72
#1 0x7f0f40c02c27 in serviceRegistry_getServiceReference_internal libs/framework/src/service_registry.c:384
#2 0x7f0f40c07458 in celix_serviceRegistry_addServiceListener libs/framework/src/service_registry.c:1089
#3 0x7f0f40be74a2 in fw_addServiceListener libs/framework/src/framework.c:1464
#4 0x7f0f40bd389c in bundleContext_addServiceListener libs/framework/src/bundle_context.c:327
#5 0x7f0f40c0a132 in serviceTracker_open libs/framework/src/service_tracker.c:210
#6 0x7f0f40c0fab5 in celix_serviceTracker_createWithOptions libs/framework/src/service_tracker.c:765
#7 0x7f0f40bd7e2c in celix_bundleContext_useServicesWithOptions libs/framework/src/bundle_context.c:814
#8 0x7f0f3526f564 in getPSAProtocol bundles/pubsub/pubsub_spi/src/pubsub_utils_match.c:132
#9 0x7f0f3527004f in pubsub_utils_matchEndpoint bundles/pubsub/pubsub_spi/src/pubsub_utils_match.c:307
#10 0x7f0f3525bc2a in pubsub_zmqAdmin_matchDiscoveredEndpoint bundles/pubsub/pubsub_admin_zmq/src/pubsub_zmq_admin.c:445
#11 0x7f0f37743c69 in pstm_findPsaForEndpoints bundles/pubsub/pubsub_topology_manager/src/pubsub_topology_manager.c:786
#12 0x7f0f377463e3 in pstm_psaHandlingThread bundles/pubsub/pubsub_topology_manager/src/pubsub_topology_manager.c:975
#13 0x348f807aa0 in start_thread (/lib64/libpthread.so.0+0x348f807aa0)
#14 0x348ece8c4c in __clone (/lib64/libc.so.6+0x348ece8c4c)
0x60c000029e38 is located 56 bytes inside of 128-byte region [0x60c000029e00,0x60c000029e80)
freed by thread T5 here:
#0 0x7f0f40ff3810 in __interceptor_free ../../../../libsanitizer/asan/asan_malloc_linux.cc:45
#1 0x7f0f40bfc38f in serviceReference_destroy libs/framework/src/service_reference.c:175
#2 0x7f0f40bfbb3d in serviceReference_release libs/framework/src/service_reference.c:87
#3 0x7f0f40c038a5 in serviceRegistry_ungetServiceReference libs/framework/src/service_registry.c:514
#4 0x7f0f40be7210 in framework_ungetServiceReference libs/framework/src/framework.c:1442
#5 0x7f0f40bd30de in bundleContext_ungetServiceReference libs/framework/src/bundle_context.c:251
#6 0x7f0f40c0db11 in serviceTracker_untrackTracked libs/framework/src/service_tracker.c:620
#7 0x7f0f40c0aa49 in serviceTracker_close libs/framework/src/service_tracker.c:260
#8 0x7f0f40c0fc67 in celix_serviceTracker_destroy libs/framework/src/service_tracker.c:780
#9 0x7f0f40bd804e in celix_bundleContext_useServicesWithOptions libs/framework/src/bundle_context.c:817
#10 0x7f0f3526f564 in getPSAProtocol bundles/pubsub/pubsub_spi/src/pubsub_utils_match.c:132
#11 0x7f0f3527004f in pubsub_utils_matchEndpoint bundles/pubsub/pubsub_spi/src/pubsub_utils_match.c:307
#12 0x7f0f3525bc2a in pubsub_zmqAdmin_matchDiscoveredEndpoint bundles/pubsub/pubsub_admin_zmq/src/pubsub_zmq_admin.c:445
#13 0x7f0f37743c69 in pstm_findPsaForEndpoints bundles/pubsub/pubsub_topology_manager/src/pubsub_topology_manager.c:786
#14 0x7f0f377463e3 in pstm_psaHandlingThread bundles/pubsub/pubsub_topology_manager/src/pubsub_topology_manager.c:975
#15 0x348f807aa0 in start_thread (/lib64/libpthread.so.0+0x348f807aa0)
previously allocated by thread T5 here:
#0 0x7f0f40ff3cd0 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cc:70
#1 0x7f0f40bfb4c5 in serviceReference_create libs/framework/src/service_reference.c:45
#2 0x7f0f40c02b13 in serviceRegistry_getServiceReference_internal libs/framework/src/service_registry.c:377
#3 0x7f0f40c07458 in celix_serviceRegistry_addServiceListener libs/framework/src/service_registry.c:1089
#4 0x7f0f40be74a2 in fw_addServiceListener libs/framework/src/framework.c:1464
#5 0x7f0f40bd389c in bundleContext_addServiceListener libs/framework/src/bundle_context.c:327
#6 0x7f0f40c0a132 in serviceTracker_open libs/framework/src/service_tracker.c:210
#7 0x7f0f40c0fab5 in celix_serviceTracker_createWithOptions libs/framework/src/service_tracker.c:765
#8 0x7f0f40bd7e2c in celix_bundleContext_useServicesWithOptions libs/framework/src/bundle_context.c:814
#9 0x7f0f3526f564 in getPSAProtocol bundles/pubsub/pubsub_spi/src/pubsub_utils_match.c:132
#10 0x7f0f3527004f in pubsub_utils_matchEndpoint bundles/pubsub/pubsub_spi/src/pubsub_utils_match.c:307
#11 0x7f0f3525bc2a in pubsub_zmqAdmin_matchDiscoveredEndpoint bundles/pubsub/pubsub_admin_zmq/src/pubsub_zmq_admin.c:445
#12 0x7f0f37743c69 in pstm_findPsaForEndpoints bundles/pubsub/pubsub_topology_manager/src/pubsub_topology_manager.c:786
#13 0x7f0f377463e3 in pstm_psaHandlingThread bundles/pubsub/pubsub_topology_manager/src/pubsub_topology_manager.c:975
#14 0x348f807aa0 in start_thread (/lib64/libpthread.so.0+0x348f807aa0)
Thread T5 created by T0 here:
#0 0x7f0f40f62d79 in __interceptor_pthread_create ../../../../libsanitizer/asan/asan_interceptors.cc:236
#1 0x7f0f3ec30252 in celixThread_create libs/utils/src/celix_threads.c:37
#2 0x7f0f3773c3b6 in pubsub_topologyManager_create bundles/pubsub/pubsub_topology_manager/src/pubsub_topology_manager.c:90
#3 0x7f0f3773ad47 in pstm_start bundles/pubsub/pubsub_topology_manager/src/pstm_activator.c:68
#4 0x7f0f3773bb90 in celix_bundleActivator_start bundles/pubsub/pubsub_topology_manager/src/pstm_activator.c:170
#5 0x7f0f40be2ffd in fw_startBundle libs/framework/src/framework.c:884
#6 0x7f0f40bc648b in bundle_startWithOptions libs/framework/src/bundle.c:267
#7 0x7f0f40be0b44 in framework_autoStartConfiguredBundlesForList libs/framework/src/framework.c:603
#8 0x7f0f40be071e in framework_autoStartConfiguredBundles libs/framework/src/framework.c:570
#9 0x7f0f40be0313 in framework_start libs/framework/src/framework.c:549
#10 0x7f0f40c1f096 in celixLauncher_launchWithProperties libs/framework/src/celix_launcher.c:172
#11 0x7f0f40c1eff9 in celixLauncher_launchWithConfigAndProps libs/framework/src/celix_launcher.c:157
#12 0x7f0f40c1eddc in celixLauncher_launchAndWaitForShutdown libs/framework/src/celix_launcher.c:113
#13 0x400ed3 in main celix/gen/containers/ZeroMQPublisher_cnt/main.cc:16
#14 0x348ec1ed1f in __libc_start_main (/lib64/libc.so.6+0x348ec1ed1f)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services