You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Trustin Lee <tr...@apache.org> on 2007/11/23 13:27:50 UTC
Re: Possible dropping of messages received in MINA 1.1.2
Hi Tirthankar,
Please do not send the message directly to me but to the mailing list.
On Nov 23, 2007 9:25 PM, Tirthankar Ghosh <Ti...@synechron.com> wrote:
> Hi,
>
> I work in maintaining a trading order management system which uses Quickfix
> as a FIX engine. Quickfix uses MINA for socket level interaction. Currently
> one of our FIX components were upgraded to a higher version of Quickfix
> (from 1.2.0 to 1.3.0) and we potentially saw that the FIX engine didn't seem
> to receive a few FIX messages (identifiable by the sequence number of the
> FIX messges and the messages observed to be sent from the source). We don't
> think network connection break is a possibility since we observed no
> exceptions either at the source of the messages nor in our engine.
>
> The MINA version we were using after the upgrade (along with Quickfix 1.3.0)
> is 1.1.2. Do you think there was any bug in 1.1.2 which could potentially
> lead to such dropping of messages? Please note that this engine was running
> for a substantial period with a lower version of Quickfix and MINA with no
> complains, in the same setup (network and load).
MINA doesn't have such a bug as far as I know.
> I was looking at DIRMINA-305 which seems to be talking about prioritization
> of writing of messages compared to reads over socket, but could that
> potentially lead to complete dropping of messages. Or could it lead to a
> case where a later message is received before the earlier one (i.e a higher
> sequence number FIX message is read before a lower one is read).
Never.
> We are also looking at possible Quickfix related bugs which could lead to
> this, but since it seams to be network level failure to accept messages that
> were sent, we wanted to be check up with you too. Once again please note
> that our system runs just fine after the rollback to the lower version of
> Quickfix 1.2.0.
You could also upgrade MINA to the latest stable version: 1.1.5.
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Need help to identify possible race condition in this snippet
Posted by Brad Harvey <ha...@gmail.com>.
Hi Tirthankar,
Generally speaking, you have to go out of your way to make
messageReceived not process messages from the same connection in order
one at a time (something to do with UnorderedExecutorFilter in the
trunk, I believe). So, if all your messages are on a single TCP/IP
connection then I don't think there's a race condition in that code
snippet.
If you have multiple FIX sessions running then each will have its own
sequencing, so in that case you won't have any guarantees except that
they'll be processed in the order they arrived relative to other
messages in the same session. Also, the fix protocol has resend
requests that can cause "old" messages to be sent again, leading to old
messages with lower sequence numbers arriving after previously processed
messages with higher sequence numbers.
You appear to be using quickfix/j to parse the messages, but you're not
using the standard quickfix/j interfaces. Is this because you're not
using the standard fix session layer?
Hope that helps your investigation.
Brad.
Tirthankar Ghosh wrote:
> Hi,
>
>
>
> We are using MINA to capture high-throughput messages over network. It is
> important for us to process the messages in the exact sequence they come in
> to our application, over the network. The messages contain sequence numbers
> and processing needs to ensure that lower sequence numbers are being
> processed before the higher ones.
>
>
>
> Given this context, please look at the code snippet below. It would help us,
> if you could let me know if the code marked within the comments, // IS THIS
> SECTION PRONED TO RACE CONDITION, can possibly lead to a race condition
> leading to the higher sequenced messages getting queued and hence processed
> before the lower ones.
>
>
>
> The code below shows how we have messageReceived callback coded. After the
> message gets queued in a blockingqueue, the subsequent code is to poll and
> queue and process the messages one at a time. But there is some bit of code
> even before the message gets queued and that is where I am under the
> impression that a race condition can occur. Please let me know what you
> think since we are currently experiencing this problem under rare cases, and
> reproducing the problem in QA has not been successful so far.
>
>
>
> public void messageReceived(IoSession ioSession, Object message) throws
> Exception {
>
> // IS THIS SECTION PRONED TO RACE CONDITION
>
> String messageString = (String) message;
>
> SessionID remoteSessionID =
> MessageUtils.getReverseSessionID(messageString);
>
> Session fixSession = findQFSession(ioSession, remoteSessionID);
>
> if (fixSession != null) {
>
> fixSession.getLog().onIncoming(messageString); // log the
> incoming message in database
>
> MessageFactory messageFactory = fixSession.getMessageFactory();
>
> DataDictionary dataDictionary = fixSession.getDataDictionary();
>
> Message fixMessage;
>
> try {
>
> fixMessage = MessageUtils.parse(messageFactory,
> dataDictionary, messageString);
>
> // IS THIS SECTION PRONED TO RACE CONDITION
>
> queue.put(fixMessage); // put the message into a singleton
> linkedblockingqueue
>
> } catch (InvalidMessage e) {
>
> log.error("Invalid message: " + e.getMessage());
>
> }
>
> } else {
>
> log.error("Disconnecting; received message for unknown session:
> " + messageString);
>
> ioSession.close();
>
> }
>
> }
>
>
>
>
>
> Any help would be appreciated.
>
>
>
> Thanks and regards,
>
> Tirthankar
>
>
>
>
>
Re: unsubscribe
Posted by Emmanuel Lecharny <el...@gmail.com>.
Better send this mail to : dev-unsubscribe@mina.apache.org^
<mailto:dev-unsubscribe@mina.apache.org?subject=unsubscribe&body=unsubscribe>
(http://mina.apache.org/mailing-lists.html(
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
unsubscribe
Posted by Burca Ciprian <ci...@deuromedia.ro>.
-----Original Message-----
From: Tirthankar Ghosh [mailto:Tirthankarg@synechron.com]
Sent: Tuesday, November 27, 2007 7:50 AM
To: dev@mina.apache.org
Subject: Need help to identify possible race condition in this snippet
Hi,
We are using MINA to capture high-throughput messages over network. It is
important for us to process the messages in the exact sequence they come in
to our application, over the network. The messages contain sequence numbers
and processing needs to ensure that lower sequence numbers are being
processed before the higher ones.
Given this context, please look at the code snippet below. It would help us,
if you could let me know if the code marked within the comments, // IS THIS
SECTION PRONED TO RACE CONDITION, can possibly lead to a race condition
leading to the higher sequenced messages getting queued and hence processed
before the lower ones.
The code below shows how we have messageReceived callback coded. After the
message gets queued in a blockingqueue, the subsequent code is to poll and
queue and process the messages one at a time. But there is some bit of code
even before the message gets queued and that is where I am under the
impression that a race condition can occur. Please let me know what you
think since we are currently experiencing this problem under rare cases, and
reproducing the problem in QA has not been successful so far.
public void messageReceived(IoSession ioSession, Object message) throws
Exception {
// IS THIS SECTION PRONED TO RACE CONDITION
String messageString = (String) message;
SessionID remoteSessionID =
MessageUtils.getReverseSessionID(messageString);
Session fixSession = findQFSession(ioSession, remoteSessionID);
if (fixSession != null) {
fixSession.getLog().onIncoming(messageString); // log the
incoming message in database
MessageFactory messageFactory = fixSession.getMessageFactory();
DataDictionary dataDictionary = fixSession.getDataDictionary();
Message fixMessage;
try {
fixMessage = MessageUtils.parse(messageFactory,
dataDictionary, messageString);
// IS THIS SECTION PRONED TO RACE CONDITION
queue.put(fixMessage); // put the message into a singleton
linkedblockingqueue
} catch (InvalidMessage e) {
log.error("Invalid message: " + e.getMessage());
}
} else {
log.error("Disconnecting; received message for unknown session:
" + messageString);
ioSession.close();
}
}
Any help would be appreciated.
Thanks and regards,
Tirthankar
Need help to identify possible race condition in this snippet
Posted by Tirthankar Ghosh <Ti...@synechron.com>.
Hi,
We are using MINA to capture high-throughput messages over network. It is
important for us to process the messages in the exact sequence they come in
to our application, over the network. The messages contain sequence numbers
and processing needs to ensure that lower sequence numbers are being
processed before the higher ones.
Given this context, please look at the code snippet below. It would help us,
if you could let me know if the code marked within the comments, // IS THIS
SECTION PRONED TO RACE CONDITION, can possibly lead to a race condition
leading to the higher sequenced messages getting queued and hence processed
before the lower ones.
The code below shows how we have messageReceived callback coded. After the
message gets queued in a blockingqueue, the subsequent code is to poll and
queue and process the messages one at a time. But there is some bit of code
even before the message gets queued and that is where I am under the
impression that a race condition can occur. Please let me know what you
think since we are currently experiencing this problem under rare cases, and
reproducing the problem in QA has not been successful so far.
public void messageReceived(IoSession ioSession, Object message) throws
Exception {
// IS THIS SECTION PRONED TO RACE CONDITION
String messageString = (String) message;
SessionID remoteSessionID =
MessageUtils.getReverseSessionID(messageString);
Session fixSession = findQFSession(ioSession, remoteSessionID);
if (fixSession != null) {
fixSession.getLog().onIncoming(messageString); // log the
incoming message in database
MessageFactory messageFactory = fixSession.getMessageFactory();
DataDictionary dataDictionary = fixSession.getDataDictionary();
Message fixMessage;
try {
fixMessage = MessageUtils.parse(messageFactory,
dataDictionary, messageString);
// IS THIS SECTION PRONED TO RACE CONDITION
queue.put(fixMessage); // put the message into a singleton
linkedblockingqueue
} catch (InvalidMessage e) {
log.error("Invalid message: " + e.getMessage());
}
} else {
log.error("Disconnecting; received message for unknown session:
" + messageString);
ioSession.close();
}
}
Any help would be appreciated.
Thanks and regards,
Tirthankar