You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Mandeep Chadha (JIRA)" <ji...@apache.org> on 2015/10/06 19:35:26 UTC

[jira] [Created] (MESOS-3595) Framework process hangs after master failover when number frameworks > libprocess thread pool size

Mandeep Chadha created MESOS-3595:
-------------------------------------

             Summary: Framework process hangs after master failover when number frameworks > libprocess thread pool size
                 Key: MESOS-3595
                 URL: https://issues.apache.org/jira/browse/MESOS-3595
             Project: Mesos
          Issue Type: Bug
          Components: scheduler driver
    Affects Versions: 0.24.1
            Reporter: Mandeep Chadha


If the number of framework created exceeds the lib process
threads then during master failover the zookeeper updates can
cause deadlock. E.g. On a machine with 24 cpus, if the framework count exceeds 24 then when the master fails over all the libprocess threads block updating the cache ( GroupProcess) leading to deadlock. Below is the stack trace of one the libprocess thread :


Thread 101 (Thread 0x7f42821f1700 (LWP 5974)):
#0  0x000000314100b5bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f42870d1637 in Gate::arrive(long) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#2  0x00007f42870be87c in process::ProcessManager::wait(process::UPID const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.eg
g/mesos/native/_mesos.so
#3  0x00007f42870c25f7 in process::wait(process::UPID const&, Duration const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.e
gg/mesos/native/_mesos.so
#4  0x00007f428708e294 in process::Latch::await(Duration const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/nativ
e/_mesos.so
#5  0x00007f4286b67dea in process::Future<int>::await(Duration const&) const () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg
/mesos/native/_mesos.so
#6  0x00007f4286b5a0df in process::Future<int>::get() const () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_me
sos.so
#7  0x00007f4286ff0508 in ZooKeeper::getChildren(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::vector<std::basic_string<char, std::cha
r_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >*) () from /Users/mchadha/venv/lib/python2.7/site
-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#8  0x00007f4286cb394e in zookeeper::GroupProcess::cache() () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mes
os.so
#9  0x00007f4286cb1e63 in zookeeper::GroupProcess::updated(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /Users/mchadha/venv/lib/py
thon2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#10 0x00007f4286ce027a in std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>::operator()(zo
okeeper::GroupProcess*, long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.n
ative-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#11 0x00007f4286ce0067 in std::tr1::result_of<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > con
st&)> ()(std::tr1::result_of<std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*&>)>::type, std::tr1::res
ult_of<std::tr1::_Mu<long, false, false> ()(long, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*&>))
>::type, std::tr1::result_of<std::tr1::_Mu<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, false, false> ()(std::basic_string<char, std::char_traits<char>
, std::allocator<char> >, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*&>))>::type)>::type std::tr1
::_Bind<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)> ()(std::tr1::_Placeholder<1>, lo
ng, std::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::__call<zookeeper::GroupProcess*&, 0, 1, 2>(std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ( c
onst&)(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*&>), std::tr1::_Index_tuple<0, 1, 2>) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.nati
ve-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#12 0x00007f4286cdfd16 in std::tr1::result_of<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > con
st&)> ()(std::tr1::result_of<std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*>)>::type, std::tr1::resu
lt_of<std::tr1::_Mu<long, false, false> ()(long, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*>))>:
:type, std::tr1::result_of<std::tr1::_Mu<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, false, false> ()(std::basic_string<char, std::char_traits<char>,
std::allocator<char> >, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*>))>::type)>::type std::tr1::_
Bind<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)> ()(std::tr1::_Placeholder<1>, long,
 std::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::operator()<zookeeper::GroupProcess*>(zookeeper::GroupProcess*&) () from /Users/mchadha/venv/lib/python2
.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#13 0x00007f4286cdf8be in std::tr1::_Function_handler<void ()(zookeeper::GroupProcess*), std::tr1::_Bind<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_stri
ng<char, std::char_traits<char>, std::allocator<char> > const&)> ()(std::tr1::_Placeholder<1>, long, std::basic_string<char, std::char_traits<char>, std::allocator<char> >)> >::_
M_invoke(std::tr1::_Any_data const&, zookeeper::GroupProcess*) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/
_mesos.so
#14 0x00007f4286cc2394 in std::tr1::function<void ()(zookeeper::GroupProcess*)>::operator()(zookeeper::GroupProcess*) const () from /Users/mchadha/venv/lib/python2.7/site-package
s/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#15 0x00007f4286cbc3a2 in void process::internal::vdispatcher<zookeeper::GroupProcess>(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProc
ess*)> >) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#16 0x00007f4286ccdca5 in std::tr1::result_of<void (*()(std::tr1::result_of<std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<pr
ocess::ProcessBase*&>)>::type, std::tr1::result_of<std::tr1::_Mu<std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >, false, false> ()(std::tr1::shared_p
tr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<process::ProcessBa
se*&>))>::type))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >)>::type std::tr1::_Bind<void (*()(std::tr1::_Placeholder<1>,
std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >
)>::__call<process::ProcessBase*&, 0, 1>(std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ( const&)(std::tr1::_Placeholder<1>, std::tr1::tuple<process::ProcessBase*&>), std:
:tr1::_Index_tuple<0, 1>) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#17 0x00007f4286cc7a5a in std::tr1::result_of<void (*()(std::tr1::result_of<std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<pr
ocess::ProcessBase*>)>::type, std::tr1::result_of<std::tr1::_Mu<std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >, false, false> ()(std::tr1::shared_pt
r<std::tr1::function<void ()(zookeeper::GroupProcess*)> >, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<process::ProcessBas
e*>))>::type))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >)>::type std::tr1::_Bind<void (*()(std::tr1::_Placeholder<1>, st
d::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >)>
::operator()<process::ProcessBase*>(process::ProcessBase*&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_me
sos.so
#18 0x00007f4286cc2480 in std::tr1::_Function_handler<void ()(process::ProcessBase*), std::tr1::_Bind<void (*()(std::tr1::_Placeholder<1>, std::tr1::shared_ptr<std::tr1::function
<void ()(zookeeper::GroupProcess*)> >))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >)> >::_M_invoke(std::tr1::_Any_data con
st&, process::ProcessBase*) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#19 0x00007f42870db546 in std::tr1::function<void ()(process::ProcessBase*)>::operator()(process::ProcessBase*) const () from /Users/mchadha/venv/lib/python2.7/site-packages/meso
s.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#20 0x00007f42870c1013 in process::ProcessBase::visit(process::DispatchEvent const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x8
6_64.egg/mesos/native/_mesos.so
#21 0x00007f42870c5582 in process::DispatchEvent::visit(process::EventVisitor*) const () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x
86_64.egg/mesos/native/_mesos.so
#22 0x00007f428666680e in process::ProcessBase::serve(process::Event const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg
/mesos/native/_mesos.so
#23 0x00007f42870bd88f in process::ProcessManager::resume(process::ProcessBase*) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64
.egg/mesos/native/_mesos.so
#24 0x00007f42870b1cb9 in process::schedule(void*) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#25 0x00000031410079d1 in start_thread () from /lib64/libpthread.so.0
#26 0x00000031408e88fd in clone () from /lib64/libc.so.6
Thread 100 (Thread 0x7f42819f0700 (LWP 5975)):
#0  0x000000314100b5bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f42870d1637 in Gate::arrive(long) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#2  0x00007f42870be87c in process::ProcessManager::wait(process::UPID const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.eg
g/mesos/native/_mesos.so
#3  0x00007f42870c25f7 in process::wait(process::UPID const&, Duration const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.e
gg/mesos/native/_mesos.so
#4  0x00007f428708e294 in process::Latch::await(Duration const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/nativ
e/_mesos.so
#5  0x00007f4286ff62e4 in process::Future<long>::await(Duration const&) const () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.eg
g/mesos/native/_mesos.so
#6  0x00007f4286ff3533 in process::Future<long>::get() const () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_m
esos.so
#7  0x00007f4286fefea9 in ZooKeeper::getSessionId() () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#8  0x00007f4286cb1d81 in zookeeper::GroupProcess::updated(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /Users/mchadha/venv/lib/py
thon2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#9  0x00007f4286ce027a in std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>::operator()(zo
okeeper::GroupProcess*, long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.n
ative-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#10 0x00007f4286ce0067 in std::tr1::result_of<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > con
st&)> ()(std::tr1::result_of<std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*&>)>::type, std::tr1::res
ult_of<std::tr1::_Mu<long, false, false> ()(long, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*&>))
>::type, std::tr1::result_of<std::tr1::_Mu<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, false, false> ()(std::basic_string<char, std::char_traits<char>
, std::allocator<char> >, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*&>))>::type)>::type std::tr1
::_Bind<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)> ()(std::tr1::_Placeholder<1>, lo
ng, std::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::__call<zookeeper::GroupProcess*&, 0, 1, 2>(std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ( c
onst&)(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*&>), std::tr1::_Index_tuple<0, 1, 2>) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.nati
ve-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#11 0x00007f4286cdfd16 in std::tr1::result_of<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > con
st&)> ()(std::tr1::result_of<std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*>)>::type, std::tr1::resu
lt_of<std::tr1::_Mu<long, false, false> ()(long, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*>))>:
:type, std::tr1::result_of<std::tr1::_Mu<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, false, false> ()(std::basic_string<char, std::char_traits<char>,
std::allocator<char> >, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<zookeeper::GroupProcess*>))>::type)>::type std::tr1::_
Bind<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)> ()(std::tr1::_Placeholder<1>, long,
 std::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::operator()<zookeeper::GroupProcess*>(zookeeper::GroupProcess*&) () from /Users/mchadha/venv/lib/python2
.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#12 0x00007f4286cdf8be in std::tr1::_Function_handler<void ()(zookeeper::GroupProcess*), std::tr1::_Bind<std::tr1::_Mem_fn<void (zookeeper::GroupProcess::*)(long, std::basic_stri
ng<char, std::char_traits<char>, std::allocator<char> > const&)> ()(std::tr1::_Placeholder<1>, long, std::basic_string<char, std::char_traits<char>, std::allocator<char> >)> >::_
M_invoke(std::tr1::_Any_data const&, zookeeper::GroupProcess*) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/
_mesos.so
#13 0x00007f4286cc2394 in std::tr1::function<void ()(zookeeper::GroupProcess*)>::operator()(zookeeper::GroupProcess*) const () from /Users/mchadha/venv/lib/python2.7/site-package
s/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#14 0x00007f4286cbc3a2 in void process::internal::vdispatcher<zookeeper::GroupProcess>(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProc
ess*)> >) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#15 0x00007f4286ccdca5 in std::tr1::result_of<void (*()(std::tr1::result_of<std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<pr
ocess::ProcessBase*&>)>::type, std::tr1::result_of<std::tr1::_Mu<std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >, false, false> ()(std::tr1::shared_p
tr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<process::ProcessBa
se*&>))>::type))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >)>::type std::tr1::_Bind<void (*()(std::tr1::_Placeholder<1>,
std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >
)>::__call<process::ProcessBase*&, 0, 1>(std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ( const&)(std::tr1::_Placeholder<1>, std::tr1::tuple<process::ProcessBase*&>), std:
:tr1::_Index_tuple<0, 1>) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#16 0x00007f4286cc7a5a in std::tr1::result_of<void (*()(std::tr1::result_of<std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<pr
ocess::ProcessBase*>)>::type, std::tr1::result_of<std::tr1::_Mu<std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >, false, false> ()(std::tr1::shared_pt
r<std::tr1::function<void ()(zookeeper::GroupProcess*)> >, std::tr1::_Mu<std::tr1::_Placeholder<1>, false, true> ()(std::tr1::_Placeholder<1>, std::tr1::tuple<process::ProcessBas
e*>))>::type))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >)>::type std::tr1::_Bind<void (*()(std::tr1::_Placeholder<1>, st
d::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >)>
::operator()<process::ProcessBase*>(process::ProcessBase*&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_me
sos.so
#17 0x00007f4286cc2480 in std::tr1::_Function_handler<void ()(process::ProcessBase*), std::tr1::_Bind<void (*()(std::tr1::_Placeholder<1>, std::tr1::shared_ptr<std::tr1::function
<void ()(zookeeper::GroupProcess*)> >))(process::ProcessBase*, std::tr1::shared_ptr<std::tr1::function<void ()(zookeeper::GroupProcess*)> >)> >::_M_invoke(std::tr1::_Any_data con
st&, process::ProcessBase*) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#18 0x00007f42870db546 in std::tr1::function<void ()(process::ProcessBase*)>::operator()(process::ProcessBase*) const () from /Users/mchadha/venv/lib/python2.7/site-packages/meso
s.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#19 0x00007f42870c1013 in process::ProcessBase::visit(process::DispatchEvent const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x8
6_64.egg/mesos/native/_mesos.so
#20 0x00007f42870c5582 in process::DispatchEvent::visit(process::EventVisitor*) const () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x
86_64.egg/mesos/native/_mesos.so
#21 0x00007f428666680e in process::ProcessBase::serve(process::Event const&) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg
/mesos/native/_mesos.so
#22 0x00007f42870bd88f in process::ProcessManager::resume(process::ProcessBase*) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64
.egg/mesos/native/_mesos.so
#23 0x00007f42870b1cb9 in process::schedule(void*) () from /Users/mchadha/venv/lib/python2.7/site-packages/mesos.native-0.22.1003-py2.7-linux-x86_64.egg/mesos/native/_mesos.so
#24 0x00000031410079d1 in start_thread () from /lib64/libpthread.so.0
#25 0x00000031408e88fd in clone () from /lib64/libc.so.6

Solution: 
 Create master detector per url instead of per framework.
Will send the review request. 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)