You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Victor <dr...@mail333.com> on 2009/02/24 23:47:45 UTC

Connector: sessionCreated is not called (unstable behavior)

Hello,

I have a problem with NIO connector in mina 2.0 M4 when handling many 
simultaneous connections - sometimes after calling connect() I do not 
receive sessionCreated() callback.

I am trying to prepare a small test, but I can not reproduce the problem 
in this test yet, so I tried to investigate mina sources myself using 
out working system.

I can see that a new Channel is always created, channel.connect() is 
called, SYN, SYN-ACK, ACK are sent, then I see that 
AbstractPollingIoConnector.Connector registers the new channel in 
Selector, but after that AbstractPollingIoConnector.Connector is stopped 
because nHandles=0 and connectQueue is already empty (no more connection 
requests).

I mean the following code in AbstractPollingIoConnector.Connector.run():

if (nHandles == 0) {
	synchronized (lock) {
		if (connectQueue.isEmpty()) {
			connector = null;
			break;
		}
	}
}

After that, 'select' is not called on the Selector anymore for this 
session.

One reason of this behavior (which I can assume now) is 'nHandles' 
variable initialized outside of 'while' loop. Maybe, it should be 
initialized on each iteration? So every time we calculate the number of 
new sessions, canceled sessions, etc.
In my situation, nHandles=-1, then, after registering the new session, 
nHandles becomes 0 and the Connector.run() is stopped.


Victor N

Re: Connector: sessionCreated is not called (unstable behavior)

Posted by Emmanuel Lecharny <el...@apache.org>.
Julien Vermillard wrote:
> Le Tue, 10 Mar 2009 16:32:37 +0100,
> Emmanuel Lecharny <el...@apache.org> a écrit :
>
>   
>> On Tue, Mar 10, 2009 at 4:18 PM, Victor <dr...@mail333.com>
>> wrote:
>>     
>>> Hello,
>>>
>>> I have finally found where the problem was. It was in my code:
>>> I called ConnectionRequest.cancel() for a connection request in the
>>> "done" state. Now I added the check:
>>>
>>> if (!connectionRequest.isDone()) {
>>>    connectionRequest.cancel();
>>> }
>>>
>>> I think it would be great to add this check to ConnectionRequest
>>> class itself. Otherwise mina adds the request to cancelQueue, but
>>> in fact there is nothing to cancel - that's why the Connector's
>>> run() loop works incorrectly. What do you think?
>>>       
>> Well, i'm a bit relieved that you found the problem you had, as I
>> didn't had any clue about what could hev been the origin of your
>> problem.
>>
>> However, your suggestion is smart. We may want to add a check in the
>> cancel() method to avoid such a problem.
>>
>> Can you fill a JIRA describing the problem and including the solution
>> proposal ?
>>
>> Many thanks !
>>     
>
> Yep why not firing a good old exception if the task is already
> completed ?
>   
I fixed it in a simpler way : if the connection is already closed, then 
we do nothing.

Patched in http://svn.apache.org/viewvc?rev=752297&view=rev

Please tell me if it's OK.

Thanks !

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org



Re: Connector: sessionCreated is not called (unstable behavior)

Posted by Julien Vermillard <jv...@archean.fr>.
Le Tue, 10 Mar 2009 16:32:37 +0100,
Emmanuel Lecharny <el...@apache.org> a écrit :

> On Tue, Mar 10, 2009 at 4:18 PM, Victor <dr...@mail333.com>
> wrote:
> > Hello,
> >
> > I have finally found where the problem was. It was in my code:
> > I called ConnectionRequest.cancel() for a connection request in the
> > "done" state. Now I added the check:
> >
> > if (!connectionRequest.isDone()) {
> >    connectionRequest.cancel();
> > }
> >
> > I think it would be great to add this check to ConnectionRequest
> > class itself. Otherwise mina adds the request to cancelQueue, but
> > in fact there is nothing to cancel - that's why the Connector's
> > run() loop works incorrectly. What do you think?
> 
> Well, i'm a bit relieved that you found the problem you had, as I
> didn't had any clue about what could hev been the origin of your
> problem.
> 
> However, your suggestion is smart. We may want to add a check in the
> cancel() method to avoid such a problem.
> 
> Can you fill a JIRA describing the problem and including the solution
> proposal ?
> 
> Many thanks !

Yep why not firing a good old exception if the task is already
completed ?

Julien

Re: Connector: sessionCreated is not called (unstable behavior)

Posted by Emmanuel Lecharny <el...@apache.org>.
On Tue, Mar 10, 2009 at 4:18 PM, Victor <dr...@mail333.com> wrote:
> Hello,
>
> I have finally found where the problem was. It was in my code:
> I called ConnectionRequest.cancel() for a connection request in the "done"
> state. Now I added the check:
>
> if (!connectionRequest.isDone()) {
>    connectionRequest.cancel();
> }
>
> I think it would be great to add this check to ConnectionRequest class
> itself. Otherwise mina adds the request to cancelQueue, but in fact there is
> nothing to cancel - that's why the Connector's run() loop works incorrectly.
> What do you think?

Well, i'm a bit relieved that you found the problem you had, as I
didn't had any clue about what could hev been the origin of your
problem.

However, your suggestion is smart. We may want to add a check in the
cancel() method to avoid such a problem.

Can you fill a JIRA describing the problem and including the solution proposal ?

Many thanks !
-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Connector: sessionCreated is not called (unstable behavior)

Posted by Victor <dr...@mail333.com>.
Hello,

I have finally found where the problem was. It was in my code:
I called ConnectionRequest.cancel() for a connection request in the 
"done" state. Now I added the check:

if (!connectionRequest.isDone()) {
     connectionRequest.cancel();
}

I think it would be great to add this check to ConnectionRequest class 
itself. Otherwise mina adds the request to cancelQueue, but in fact 
there is nothing to cancel - that's why the Connector's run() loop works 
incorrectly.
What do you think?

Thanks,

Victor


Victor wrote:
> Just a question:
> 
> why does Connector.run() thread exit when nHandles==0? Why not to 
> continue executing select() in the loop infinitely?
> 
> Thanks,
> Victor
> 
> 
> Victor wrote:
>>
>> Unfortunately, even with the patch in Connector and Processor, the 
>> problem is reproduced randomly, most of time everything works, but 
>> sometimes I see that sessionCreated is not called. But if I start a 
>> new connection, this "old" pending connection (for which I am waiting 
>> for sessionCreated callback) is completed instantly, so I get 
>> sessionCreated.
>>
>> Victor N
>>
>>
>> Victor wrote:
>>>
>>> I have added a similar fix in 
>>> AbstractPollingIoProcessor.Processor.run() - nSessions is now 
>>> initialized inside the loop, on each iteration.
>>>
>>> It seems to work now, I can not reproduce the bug, but I will 
>>> continue testing.
>>> Not sure whether this patch is adequate or nor. Any comments?
>>>
>>> Victor N
>>>
>>>
>>> Victor wrote:
>>>> I tried to play with my fix - from the first glance, it seemed to 
>>>> work - connection was established, sessionCreated was called,
>>>> but at the second attempt the problem reproduced.
>>>>
>>>> Any ideas what is wrong here?
>>>>
>>>> Thanks,
>>>> Victor N
>>>>
>>>>
>>>> Victor wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I have a problem with NIO connector in mina 2.0 M4 when handling 
>>>>> many simultaneous connections - sometimes after calling connect() I 
>>>>> do not receive sessionCreated() callback.
>>>>>
>>>>> I am trying to prepare a small test, but I can not reproduce the 
>>>>> problem in this test yet, so I tried to investigate mina sources 
>>>>> myself using out working system.
>>>>>
>>>>> I can see that a new Channel is always created, channel.connect() 
>>>>> is called, SYN, SYN-ACK, ACK are sent, then I see that 
>>>>> AbstractPollingIoConnector.Connector registers the new channel in 
>>>>> Selector, but after that AbstractPollingIoConnector.Connector is 
>>>>> stopped because nHandles=0 and connectQueue is already empty (no 
>>>>> more connection requests).
>>>>>
>>>>> I mean the following code in 
>>>>> AbstractPollingIoConnector.Connector.run():
>>>>>
>>>>> if (nHandles == 0) {
>>>>>     synchronized (lock) {
>>>>>         if (connectQueue.isEmpty()) {
>>>>>             connector = null;
>>>>>             break;
>>>>>         }
>>>>>     }
>>>>> }
>>>>>
>>>>> After that, 'select' is not called on the Selector anymore for this 
>>>>> session.
>>>>>
>>>>> One reason of this behavior (which I can assume now) is 'nHandles' 
>>>>> variable initialized outside of 'while' loop. Maybe, it should be 
>>>>> initialized on each iteration? So every time we calculate the 
>>>>> number of new sessions, canceled sessions, etc.
>>>>> In my situation, nHandles=-1, then, after registering the new 
>>>>> session, nHandles becomes 0 and the Connector.run() is stopped.
>>>>>
>>>>>
>>>>> Victor N
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
> 

Re: Connector: sessionCreated is not called (unstable behavior)

Posted by Victor <dr...@mail333.com>.
Just a question:

why does Connector.run() thread exit when nHandles==0? Why not to 
continue executing select() in the loop infinitely?

Thanks,
Victor


Victor wrote:
> 
> Unfortunately, even with the patch in Connector and Processor, the 
> problem is reproduced randomly, most of time everything works, but 
> sometimes I see that sessionCreated is not called. But if I start a new 
> connection, this "old" pending connection (for which I am waiting for 
> sessionCreated callback) is completed instantly, so I get sessionCreated.
> 
> Victor N
> 
> 
> Victor wrote:
>>
>> I have added a similar fix in 
>> AbstractPollingIoProcessor.Processor.run() - nSessions is now 
>> initialized inside the loop, on each iteration.
>>
>> It seems to work now, I can not reproduce the bug, but I will continue 
>> testing.
>> Not sure whether this patch is adequate or nor. Any comments?
>>
>> Victor N
>>
>>
>> Victor wrote:
>>> I tried to play with my fix - from the first glance, it seemed to 
>>> work - connection was established, sessionCreated was called,
>>> but at the second attempt the problem reproduced.
>>>
>>> Any ideas what is wrong here?
>>>
>>> Thanks,
>>> Victor N
>>>
>>>
>>> Victor wrote:
>>>>
>>>> Hello,
>>>>
>>>> I have a problem with NIO connector in mina 2.0 M4 when handling 
>>>> many simultaneous connections - sometimes after calling connect() I 
>>>> do not receive sessionCreated() callback.
>>>>
>>>> I am trying to prepare a small test, but I can not reproduce the 
>>>> problem in this test yet, so I tried to investigate mina sources 
>>>> myself using out working system.
>>>>
>>>> I can see that a new Channel is always created, channel.connect() is 
>>>> called, SYN, SYN-ACK, ACK are sent, then I see that 
>>>> AbstractPollingIoConnector.Connector registers the new channel in 
>>>> Selector, but after that AbstractPollingIoConnector.Connector is 
>>>> stopped because nHandles=0 and connectQueue is already empty (no 
>>>> more connection requests).
>>>>
>>>> I mean the following code in 
>>>> AbstractPollingIoConnector.Connector.run():
>>>>
>>>> if (nHandles == 0) {
>>>>     synchronized (lock) {
>>>>         if (connectQueue.isEmpty()) {
>>>>             connector = null;
>>>>             break;
>>>>         }
>>>>     }
>>>> }
>>>>
>>>> After that, 'select' is not called on the Selector anymore for this 
>>>> session.
>>>>
>>>> One reason of this behavior (which I can assume now) is 'nHandles' 
>>>> variable initialized outside of 'while' loop. Maybe, it should be 
>>>> initialized on each iteration? So every time we calculate the number 
>>>> of new sessions, canceled sessions, etc.
>>>> In my situation, nHandles=-1, then, after registering the new 
>>>> session, nHandles becomes 0 and the Connector.run() is stopped.
>>>>
>>>>
>>>> Victor N
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>>
> 
> 
> 
> 

Re: Connector: sessionCreated is not called (unstable behavior)

Posted by Victor <dr...@mail333.com>.
Unfortunately, even with the patch in Connector and Processor, the 
problem is reproduced randomly, most of time everything works, but 
sometimes I see that sessionCreated is not called. But if I start a new 
connection, this "old" pending connection (for which I am waiting for 
sessionCreated callback) is completed instantly, so I get sessionCreated.

Victor N


Victor wrote:
> 
> I have added a similar fix in AbstractPollingIoProcessor.Processor.run() 
> - nSessions is now initialized inside the loop, on each iteration.
> 
> It seems to work now, I can not reproduce the bug, but I will continue 
> testing.
> Not sure whether this patch is adequate or nor. Any comments?
> 
> Victor N
> 
> 
> Victor wrote:
>> I tried to play with my fix - from the first glance, it seemed to work 
>> - connection was established, sessionCreated was called,
>> but at the second attempt the problem reproduced.
>>
>> Any ideas what is wrong here?
>>
>> Thanks,
>> Victor N
>>
>>
>> Victor wrote:
>>>
>>> Hello,
>>>
>>> I have a problem with NIO connector in mina 2.0 M4 when handling many 
>>> simultaneous connections - sometimes after calling connect() I do not 
>>> receive sessionCreated() callback.
>>>
>>> I am trying to prepare a small test, but I can not reproduce the 
>>> problem in this test yet, so I tried to investigate mina sources 
>>> myself using out working system.
>>>
>>> I can see that a new Channel is always created, channel.connect() is 
>>> called, SYN, SYN-ACK, ACK are sent, then I see that 
>>> AbstractPollingIoConnector.Connector registers the new channel in 
>>> Selector, but after that AbstractPollingIoConnector.Connector is 
>>> stopped because nHandles=0 and connectQueue is already empty (no more 
>>> connection requests).
>>>
>>> I mean the following code in AbstractPollingIoConnector.Connector.run():
>>>
>>> if (nHandles == 0) {
>>>     synchronized (lock) {
>>>         if (connectQueue.isEmpty()) {
>>>             connector = null;
>>>             break;
>>>         }
>>>     }
>>> }
>>>
>>> After that, 'select' is not called on the Selector anymore for this 
>>> session.
>>>
>>> One reason of this behavior (which I can assume now) is 'nHandles' 
>>> variable initialized outside of 'while' loop. Maybe, it should be 
>>> initialized on each iteration? So every time we calculate the number 
>>> of new sessions, canceled sessions, etc.
>>> In my situation, nHandles=-1, then, after registering the new 
>>> session, nHandles becomes 0 and the Connector.run() is stopped.
>>>
>>>
>>> Victor N
>>>
>>>
>>>
>>>
>>
> 
> 
> 
> 

Re: Connector: sessionCreated is not called (unstable behavior)

Posted by Victor <dr...@mail333.com>.
I have added a similar fix in AbstractPollingIoProcessor.Processor.run() 
- nSessions is now initialized inside the loop, on each iteration.

It seems to work now, I can not reproduce the bug, but I will continue 
testing.
Not sure whether this patch is adequate or nor. Any comments?

Victor N


Victor wrote:
> I tried to play with my fix - from the first glance, it seemed to work - 
> connection was established, sessionCreated was called,
> but at the second attempt the problem reproduced.
> 
> Any ideas what is wrong here?
> 
> Thanks,
> Victor N
> 
> 
> Victor wrote:
>>
>> Hello,
>>
>> I have a problem with NIO connector in mina 2.0 M4 when handling many 
>> simultaneous connections - sometimes after calling connect() I do not 
>> receive sessionCreated() callback.
>>
>> I am trying to prepare a small test, but I can not reproduce the 
>> problem in this test yet, so I tried to investigate mina sources 
>> myself using out working system.
>>
>> I can see that a new Channel is always created, channel.connect() is 
>> called, SYN, SYN-ACK, ACK are sent, then I see that 
>> AbstractPollingIoConnector.Connector registers the new channel in 
>> Selector, but after that AbstractPollingIoConnector.Connector is 
>> stopped because nHandles=0 and connectQueue is already empty (no more 
>> connection requests).
>>
>> I mean the following code in AbstractPollingIoConnector.Connector.run():
>>
>> if (nHandles == 0) {
>>     synchronized (lock) {
>>         if (connectQueue.isEmpty()) {
>>             connector = null;
>>             break;
>>         }
>>     }
>> }
>>
>> After that, 'select' is not called on the Selector anymore for this 
>> session.
>>
>> One reason of this behavior (which I can assume now) is 'nHandles' 
>> variable initialized outside of 'while' loop. Maybe, it should be 
>> initialized on each iteration? So every time we calculate the number 
>> of new sessions, canceled sessions, etc.
>> In my situation, nHandles=-1, then, after registering the new session, 
>> nHandles becomes 0 and the Connector.run() is stopped.
>>
>>
>> Victor N
>>
>>
>>
>>
> 

Re: Connector: sessionCreated is not called (unstable behavior)

Posted by Victor <dr...@mail333.com>.
I tried to play with my fix - from the first glance, it seemed to work - 
connection was established, sessionCreated was called,
but at the second attempt the problem reproduced.

Any ideas what is wrong here?

Thanks,
Victor N


Victor wrote:
> 
> Hello,
> 
> I have a problem with NIO connector in mina 2.0 M4 when handling many 
> simultaneous connections - sometimes after calling connect() I do not 
> receive sessionCreated() callback.
> 
> I am trying to prepare a small test, but I can not reproduce the problem 
> in this test yet, so I tried to investigate mina sources myself using 
> out working system.
> 
> I can see that a new Channel is always created, channel.connect() is 
> called, SYN, SYN-ACK, ACK are sent, then I see that 
> AbstractPollingIoConnector.Connector registers the new channel in 
> Selector, but after that AbstractPollingIoConnector.Connector is stopped 
> because nHandles=0 and connectQueue is already empty (no more connection 
> requests).
> 
> I mean the following code in AbstractPollingIoConnector.Connector.run():
> 
> if (nHandles == 0) {
>     synchronized (lock) {
>         if (connectQueue.isEmpty()) {
>             connector = null;
>             break;
>         }
>     }
> }
> 
> After that, 'select' is not called on the Selector anymore for this 
> session.
> 
> One reason of this behavior (which I can assume now) is 'nHandles' 
> variable initialized outside of 'while' loop. Maybe, it should be 
> initialized on each iteration? So every time we calculate the number of 
> new sessions, canceled sessions, etc.
> In my situation, nHandles=-1, then, after registering the new session, 
> nHandles becomes 0 and the Connector.run() is stopped.
> 
> 
> Victor N
> 
> 
> 
>