You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Wayne1285 <Wa...@hotmail.com> on 2006/04/28 07:55:31 UTC
Race Condition?
I am using ActiveMq-4.0-RC3. I have been having intermittent problems
getting sessions created and closed. I have downloaded the code and traced
through it and I think I have found the problem.
There is a race conditon in the TcpTransport class, in the doStart() method.
As near as I can tell, as soon as the socket is created, the Broker responds
with its WireFormatInfo. This information is usually stored as the
remoteWireFormatInfo in the InactivityMonitor. The race condition occurs in
the doStart() method as it appears the broker can sometimes send the
wireFormatInfo before the initializeStreams() method can be called. This
results in the information not being capture in the dataIn input buffer.
Once this occurs, this sets up the session hangs that I have been
experiencing. Since the remoteWireFormatInfo was never captured from the
broker, requests to InactivityMonitor.startMonitorThreads() does not start
as the condition of having a remoteWireFormatInfo is not met and the method
just returns. That causes not socket activities at all. The original thread
eventually proceeds to the WireFormatNegotiator.oneway(command) method that
issues a readyCountDownLatch.await(); which is never met and never times
out.
I have written a thread to call the create session function and give it a
maximum of 15 seconds to complete. If the thread exceeds that time, it is
sent an interrupt, but that does not seem to interrupt the
readyCountDownLatch.await(). Eventually I get all these hung threads that I
cannot clean up.
I would liike to see if the race conditon can be fixed, but in addition why
I cannot issue an interrupt against this thread when it gets into this
state.
--
View this message in context: http://www.nabble.com/Race-Condition--t1522683.html#a4135015
Sent from the ActiveMQ - User forum at Nabble.com.
Re: Race Condition?
Posted by Wayne1285 <Wa...@hotmail.com>.
I am still having this problem, but have no responses. Did I do something
wrong or am I way off base?
--
View this message in context: http://www.nabble.com/Race-Condition--t1522683.html#a4257249
Sent from the ActiveMQ - User forum at Nabble.com.
Re: Race Condition?
Posted by Wayne1285 <Wa...@hotmail.com>.
Yes a network race condition.
I am no expert in sockets, but some times the response from the broker just
simply gets missed. I added a log statement to the class InactivityMonitor
public void onCommand(Command command) {
inReceive.set(true);
try {
if (command.isWireFormatInfo()) {
synchronized (this) {
remoteWireFormatInfo = (WireFormatInfo) command;
log.debug("Just received remoteWireFormatInfo");
try {
startMonitorThreads();
} catch (IOException e) {
onException(e);
}
}
}
getTransportListener().onCommand(command);
} finally {
inReceive.set(false);
commandReceived.set(true);
}
}
But it soemtimes does not execute. Without the remoteWireFormatInfo, the
request to StartMonitorThreads() does not start the threads because all the
preconditions are not met and nothing happens from that point on.
Here is a complete activity list of what I am seeing.
Broker: Waiting for a connection
Client: Opens a connection. Connection opens successfullly
Broker: Gets client connection
Client: Sets up socket reader and writer.
Client: sends LocalWireFormat
Client: waits (forever) for broker to send WireFormatInfo
I have been experimenting with Ethereal to see what is actually transpirig
across the wire. I had originallay thought the broker was sending its
wireFormatInfo, but it now appears that what I was seeing was it recieving
the clients wireFormatInfo. It now looks like the broker does not send its
wireFormatInfo when a connection is made. I am trying to debug this now.
This only occurs after a period of time. Usually the first few clients will
connect and create sessions as expected. I am closing the connections when
a client is finsihed. Perhaps there is some intermittent bug on the broker
that does not set its state correctly when a connection is closed that
causes it not to send the wireFormatInfo the next time a connection is made.
--
View this message in context: http://www.nabble.com/Race-Condition--t1522683.html#a4264483
Sent from the ActiveMQ - User forum at Nabble.com.
Re: Race Condition?
Posted by Rob Davies <ra...@gmail.com>.
I think wayne is saying this is a network race condition - where the
broker can send the wireformat info before the client starts reading.
However, as the client doesn't start reading from the socket until
after initializeStreams is called - I'm not sure how the socket will
miss the wireformat info from the broker?
On 6 May 2006, at 07:34, James Strachan wrote:
> On 4/28/06, Wayne1285 <Wa...@hotmail.com> wrote:
>>
>> I am using ActiveMq-4.0-RC3. I have been having intermittent
>> problems
>> getting sessions created and closed. I have downloaded the code
>> and traced
>> through it and I think I have found the problem.
>>
>> There is a race conditon in the TcpTransport class, in the doStart
>> () method.
>> As near as I can tell, as soon as the socket is created, the
>> Broker responds
>> with its WireFormatInfo. This information is usually stored as the
>> remoteWireFormatInfo in the InactivityMonitor. The race condition
>> occurs in
>> the doStart() method as it appears the broker can sometimes send the
>> wireFormatInfo before the initializeStreams() method can be
>> called. This
>> results in the information not being capture in the dataIn input
>> buffer.
>
> I just refreshed my memory of the code for TcpTransport and took a
> peek at InactivityMonitor. I'm not sure I grok yet the race condition.
> TcpTransport only allows itself to be started once and uses a
> semaphore around the doStart(); so that all looks good. Also
> TcpTransport doesn't allow messages to be sent until the transport is
> started (it throws an exception if an attempt is made to send to the
> transport before its started).
>
> So I don't quite follow the sequence of events causing the race
> condition; is some thread trying to use the TcpTransport before its
> been started? If so I don't understand why the checkStarted() method
> is not throwing an exception?
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
Rob Davies
http://rajdavies.blogspot.com/
Re: Race Condition?
Posted by James Strachan <ja...@gmail.com>.
On 4/28/06, Wayne1285 <Wa...@hotmail.com> wrote:
>
> I am using ActiveMq-4.0-RC3. I have been having intermittent problems
> getting sessions created and closed. I have downloaded the code and traced
> through it and I think I have found the problem.
>
> There is a race conditon in the TcpTransport class, in the doStart() method.
> As near as I can tell, as soon as the socket is created, the Broker responds
> with its WireFormatInfo. This information is usually stored as the
> remoteWireFormatInfo in the InactivityMonitor. The race condition occurs in
> the doStart() method as it appears the broker can sometimes send the
> wireFormatInfo before the initializeStreams() method can be called. This
> results in the information not being capture in the dataIn input buffer.
I just refreshed my memory of the code for TcpTransport and took a
peek at InactivityMonitor. I'm not sure I grok yet the race condition.
TcpTransport only allows itself to be started once and uses a
semaphore around the doStart(); so that all looks good. Also
TcpTransport doesn't allow messages to be sent until the transport is
started (it throws an exception if an attempt is made to send to the
transport before its started).
So I don't quite follow the sequence of events causing the race
condition; is some thread trying to use the TcpTransport before its
been started? If so I don't understand why the checkStarted() method
is not throwing an exception?
--
James
-------
http://radio.weblogs.com/0112098/