Attempting to fix windows outgoing federation connections

[Jeff Laughlin <je...@gmail.com>]

I'll move this thread here since it's clearly graduated from "using" to
"developing".

This is my first stab at addressing
https://issues.apache.org/jira/browse/QPID-2199

Here's a substantial clue; in Connection.cpp line 64 in release 0.5 (on my
linux box), the Connection constructor tests the connection to see if it's a
link and if so notifies the link registry. This code is not reached on my
windows trunk build; ie the Connection constructor is never called.

Here's the GDB backtrace from line 65, which is the line of code that
(eventually) causes a link to go from the connecting to operational,
somewhere there's something happening here that's NOT happening in the
windows code, but should be happening in some way. The missing link seems to
be in frame number 5, AsynchIO::writeable, which the windows code doesn't
implement.

Breakpoint 2, Connection (this=0x93d3808, out_=0x93d4e14,
broker_=@0x938f538, mgmtId_=@0x9392b2c,
    isLink_=<value optimized out>, objectId=0) at
qpid/broker/Connection.cpp:65
65              links.notifyConnection(mgmtId, this);
(gdb) bt
#0  Connection (this=0x93d3808, out_=0x93d4e14, broker_=@0x938f538,
mgmtId_=@0x9392b2c,
    isLink_=<value optimized out>, objectId=0) at
qpid/broker/Connection.cpp:65
#1  0xb7f65622 in qpid::broker::
SecureConnectionFactory::create (this=0x938ffe0, out=@0x9392b28,
id=@0x9392b2c)
    at qpid/broker/SecureConnectionFactory.cpp:57
#2  0xb7d8fe29 in qpid::sys::AsynchIOHandler::idle (this=0x9392b28) at
qpid/sys/AsynchIOHandler.cpp:186
#3  0xb7fa6532 in
boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void,
boost::_mfi::mf1<void, qpid::sys::AsynchIOHandler, qpid::sys::AsynchIO&>,
boost::_bi::list2<boost::_bi::value<qpid::sys::AsynchIOHandler*>,
boost::arg<1> (*)()> >, void, qpid::sys::AsynchIO&>::invoke
(function_obj_ptr=@0x93d84c4, a0=@0x93d83e8)
    at /usr/include/boost/bind/mem_fn_template.hpp:162
#4  0xb7d38af3 in boost::function1<void, qpid::sys::AsynchIO&,
std::allocator<boost::function_base> >::operator()
    (this=0x93d84c0, a0=@0x93d83e8) at
/usr/include/boost/function/function_template.hpp:825
#5  0xb7d354b5 in qpid::sys::posix::AsynchIO::writeable (this=0x93d83e8,
h=@0x93d83ec)
    at qpid/sys/posix/AsynchIO.cpp:560
#6  0xb7d36ed2 in
boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void,
boost::_mfi::mf1<void, qpid::sys::posix::AsynchIO,
qpid::sys::DispatchHandle&>,
boost::_bi::list2<boost::_bi::value<qpid::sys::posix::AsynchIO*>,
boost::arg<1> (*)()> >, void, qpid::sys::DispatchHandle&>::invoke
(function_obj_ptr=@0x93d8408,
    a0=@0x93d83ec) at /usr/include/boost/bind/mem_fn_template.hpp:162
#7  0xb7d95503 in boost::function1<void, qpid::sys::DispatchHandle&,
std::allocator<boost::function_base> >::operator() (this=0x93d8404,
a0=@0x93d83ec) at /usr/include/boost/function/function_template.hpp:825
#8  0xb7d94a69 in qpid::sys::DispatchHandle::processEvent (this=0x93d83ec,
type=qpid::sys::Poller::WRITABLE)
    at qpid/sys/DispatchHandle.cpp:431
#9  0xb7d448b3 in qpid::sys::Poller::run (this=0x938d5d8) at
./qpid/sys/Poller.h:122
#10 0xb7d91774 in qpid::sys::Dispatcher::run (this=0xbf9506a4) at
qpid/sys/Dispatcher.cpp:37
#11 0xb7ee090e in qpid::broker::Broker::run (this=0x938f538) at
qpid/broker/Broker.cpp:319
#12 0x08050c65 in QpiddBroker::execute (this=0xbf950a93, options=0x938c3f8)
at posix/QpiddBroker.cpp:165
#13 0x0804daf9 in main (argc=2, argv=0xbf950c44) at qpidd.cpp:77

AsynchIO.cpp:560                 idleCallback(*this);

That call to the idleCallback ultimately results in the Connection object
being instantiated. Inside the Connection constructor is a call which
results in telling the link manager that the link is connected.

The windows code does have the AsynchIO::writeComplete method, which does
call idleCallback, and it called when my client interacts with the broker,
but not after the TCP connection to the remote broker is established.

This might just be another symptom; If the socket is never being written to
for some reason, then writeComplete will never get called, and the
Connection object will never be constructed. Perhaps the broker isn't ever
trying to write to the socket for some reason?

I've confirmed that isClient is beeing set to true, so if the code did reach
idleCallback, then we should be good.

I suspect that writeComplete is NOT getting called; I break when the link
object is created, then set a new breakpoint on writeComplete and continue,
and it never hits. In fact NONE of the write-related methods seem to be
called after the link object is created. Huh.

Any ideas on where to go next? I'd like to trace through the bit of code
that is supposed to negotiate the link with the remote broker, since I
suspect that somehow that's not working correctly.

Thanks,
Jeff


On Sat, Nov 14, 2009 at 6:47 PM, Steve Huston <sh...@riverace.com> wrote:

> > > > Unfortunately it seems that
> > > > https://issues.apache.org/jira/browse/QPID-2199is still in
> effect;
> > > my
> > > > link connection is stuck forever in the "connecting" state.
> > >
> > > Right, this is not fixed.
> > >
> > > > This is a show-stopper for my application :(
> > >
> > > That's too bad. Any chance you can lend a hand in fixing
> > the problem?
> > >
> >
> > I can give it a shot. Can you give me any pointers to get
> > started? qpid is an imposing hunk code...
>
> Yes, it is... I have a feeling the problem is that the broker connect
> is done blocking and the plumbing for noticing the connection is up is
> expecting some kind of callback that's not happening.
>
> If that's it, it'll be in the qpid/cpp/src/qpid/sys/windows directory,
> probably AsynchIO.cpp. I also wrote a blog post about that ;-)
>
> -Steve
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>


-- 
73 de n1ywb


-- 
73 de n1ywb