You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by pi...@Safe-mail.net on 2015/03/04 16:05:54 UTC

mesos c++ zookeeper blocks indefinately -- any plans to enhance?

hi mesos users and devs,
We've observed that that the mesos 0.22.0-rc1 c++ zookeeper code appears to allow indefinite waits on responses.
This leads to application hangs blocked inside mesos zookeeper calls.
This can happen with a properly running zookeeper presumably able to make all responses.

Heres how we hung it for eg.
We issue a mesos zk set via

int ZooKeeper::set	(	const std::string & 	path,
const std::string & 	data,
int 	version 
)	

then inside a Watcher we process on CHANGED event to issue a mesos zk get on the same path via

int ZooKeeper::get	(	const std::string & 	path,
bool 	watch,
std::string * 	result,
Stat * 	stat 
)	

we end up with two threads in the process both in pthread_cond_waits
#0  0x000000334e20b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f6664ee1cf5 in Gate::arrive (this=0x7f6140, old=0)
    at ../../../3rdparty/libprocess/src/gate.hpp:82
#2  0x00007f6664ecef6e in process::ProcessManager::wait (this=0x7f02e0, pid=...)
    at ../../../3rdparty/libprocess/src/process.cpp:2476
#3  0x00007f6664ed2ce9 in process::wait (pid=..., duration=...)
    at ../../../3rdparty/libprocess/src/process.cpp:2958
#4  0x00007f6664e90558 in process::Latch::await (this=0x7f6ba0, duration=...)
    at ../../../3rdparty/libprocess/src/latch.cpp:49
#5  0x00007f66649452cc in process::Future<int>::await (this=0x7fffa0fd9040, duration=...)
    at ../../3rdparty/libprocess/include/process/future.hpp:1156
#6  0x00007f666493a04d in process::Future<int>::get (this=0x7fffa0fd9040)
    at ../../3rdparty/libprocess/include/process/future.hpp:1167
#7  0x00007f6664ab1aac in ZooKeeper::set (this=0x803ce0, path="/craig/mo", data=
...

and
#0  0x000000334e20b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f6664ee1cf5 in Gate::arrive (this=0x7f66380013f0, old=0)
    at ../../../3rdparty/libprocess/src/gate.hpp:82
#2  0x00007f6664ecef6e in process::ProcessManager::wait (this=0x7f02e0, pid=...)
    at ../../../3rdparty/libprocess/src/process.cpp:2476
#3  0x00007f6664ed2ce9 in process::wait (pid=..., duration=...)
    at ../../../3rdparty/libprocess/src/process.cpp:2958
#4  0x00007f6664e90558 in process::Latch::await (this=0x7f6638000d00, duration=...)
    at ../../../3rdparty/libprocess/src/latch.cpp:49
#5  0x00007f66649452cc in process::Future<int>::await (this=0x7f66595fb6f0, duration=...)
    at ../../3rdparty/libprocess/include/process/future.hpp:1156
#6  0x00007f666493a04d in process::Future<int>::get (this=0x7f66595fb6f0)
    at ../../3rdparty/libprocess/include/process/future.hpp:1167
#7  0x00007f6664ab18d3 in ZooKeeper::get (this=0x803ce0, path="/craig/mo", watch=false,
....

So, really we are asking whether the mesos zk c++ api will be enhanced to not block indefinitely when results are beyond a time bound.

cheers
craig