You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Toralf Lund <to...@pgs.com> on 2019/03/28 14:56:56 UTC
"Session detached by peer" when trying to re-establish lost
connection
Hi,
I have another question related to my recent post about trying to
re-establish lost connections. Recall that my system is set up without
automatic reconnect, but may do a "manual reconnect" via
qpid::Messaging connection(url);
...
if(!connection.isOpen()) {
connection.open();
}
and that this is sometimes triggered by an "internal disconnect"
accompanied by an exception with message "Failed to connect (reconnect
disabled)".
Now, I just found that some of the reconnects are accompanied by a
"Session detached by peer" exception. I'm struggling a bit to find out
what exactly triggers this; the problems occur once every few hours at a
remote site, and I can't just start or stop or update the software at
any old time, so it's a bit hard to debug...
My question is: Is there a chance that the above open() might trigger
this exception as it tries to re-establish a session that was there
before the disconnect? If that's the case, is there a way I can avoid
getting into the situation? Would it help to do something like
if(session.isValid() && session.hasError()) {
session.close();
}
first? (Where "session" represents the session associated with
"connection" - there is one at the most.)
Thanks,
- Toralf
Re: Re: "Session detached by peer" when trying to re-establish lost
connection
Posted by Toralf Lund <to...@pgs.com>.
On 01/04/2019 22:33, Gordon Sim wrote:
> On 01/04/2019 12:54 pm, Toralf Lund wrote:
>> On 28/03/2019 15:56, Toralf Lund wrote:
>>>
>>> Hi,
>>>
>>> I have another question related to my recent post about trying to
>>> re-establish lost connections. Recall that my system is set up
>>> without automatic reconnect, but may do a "manual reconnect" via
>>>
>>> qpid::Messaging connection(url);
>>>
>>> ...
>>>
>>> if(!connection.isOpen()) {
>>> connection.open();
>>> }
>>>
>>> and that this is sometimes triggered by an "internal disconnect"
>>> accompanied by an exception with message "Failed to connect
>>> (reconnect disabled)".
>>>
>>> Now, I just found that some of the reconnects are accompanied by a
>>> "Session detached by peer" exception. I'm struggling a bit to find
>>> out what exactly triggers this; the problems occur once every few
>>> hours at a remote site, and I can't just start or stop or update the
>>> software at any old time, so it's a bit hard to debug...
>>>
>> I just noticed that the broker log has something like
>>
>> 2019-04-01 11:11:42 [Broker] error Channel exception: session-busy:
>> Session already attached:
>> anonymous.c9a1f4cf-7427-4e77-8870-e118b099c6da
>> (/work/build/rpmbuild/BUILD/msat-qpid-cpp-1.36/msat-qpid-cpp-1.36.0/src/qpid/broker/SessionManager.cpp:55)
>>
>> 2019-04-01 11:11:46 [Protocol] error Connection
>> qpid.10.31.193.215:5672-10.31.192.61:35954 timed out: closing
>>
>> for most or all the reconnects. The first time-stamp seems to
>> correspond to the connection open and "Session detached by peer"
>> exception.
>>
>> Perhaps this means the broker still thinks the old connection is
>> open, even though the client considers it as lost? And maybe the
>> attempt to re-establish/re-use the old session with the new
>> connection fails as a result of that?
>
> Correct. Does the broker log the loss of the connection, e.g. a timed
> out notice? If so how does the time of that compare to the
> session-busy error and to the client detecting the failed connection?
The broker also has a message saying "Connection ... timed out:
closing". This occurs 4-5 seconds after the session-busy error,
according to the timestamps.
- Toralf
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Re: "Session detached by peer" when trying to re-establish lost
connection
Posted by Toralf Lund <to...@pgs.com>.
On 03/04/2019 09:32, Gordon Sim wrote:
> On 03/04/2019 7:32 am, Toralf Lund wrote:
>> The broker also has a message saying "Connection ... timed out:
>> closing". This occurs 4-5 seconds after the session-busy error,
>> according to the timestamps.
>
> So for some reason, the broker is deciding the connection is dead 4-5
> seconds after the client has reconnected... what caused the client to
> reconnect? Was there a timed out notice there as well?
The direct reason for the reconnect is that "connection::isOpen()"
returns false on a connection that was successfully opened some time in
the past. The connection status is checked periodically. The "reconnect"
option is not set on the connection. Prior to getting a "false"
isOpen(), an exception with message "Failed to connect (reconnect
disabled)" is caught, typically while waiting for incoming messages.
session::nextReceiver() is used for that purpose.
The broker log does not seem to have any messages related to the
original issue.
- Toralf
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [External] Re: "Session detached by peer" when trying to
re-establish lost connection
Posted by Gordon Sim <gs...@redhat.com>.
On 03/04/2019 7:32 am, Toralf Lund wrote:
> The broker also has a message saying "Connection ... timed out:
> closing". This occurs 4-5 seconds after the session-busy error,
> according to the timestamps.
So for some reason, the broker is deciding the connection is dead 4-5
seconds after the client has reconnected... what caused the client to
reconnect? Was there a timed out notice there as well?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [External] Re: "Session detached by peer" when trying to
re-establish lost connection
Posted by Toralf Lund <to...@pgs.com>.
On 01/04/2019 22:33, Gordon Sim wrote:
> On 01/04/2019 12:54 pm, Toralf Lund wrote:
>> On 28/03/2019 15:56, Toralf Lund wrote:
>>>
>>> Hi,
>>>
>>> I have another question related to my recent post about trying to
>>> re-establish lost connections. Recall that my system is set up
>>> without automatic reconnect, but may do a "manual reconnect" via
>>>
>>> qpid::Messaging connection(url);
>>>
>>> ...
>>>
>>> if(!connection.isOpen()) {
>>> connection.open();
>>> }
>>>
>>> and that this is sometimes triggered by an "internal disconnect"
>>> accompanied by an exception with message "Failed to connect
>>> (reconnect disabled)".
>>>
>>> Now, I just found that some of the reconnects are accompanied by a
>>> "Session detached by peer" exception. I'm struggling a bit to find
>>> out what exactly triggers this; the problems occur once every few
>>> hours at a remote site, and I can't just start or stop or update the
>>> software at any old time, so it's a bit hard to debug...
>>>
>> I just noticed that the broker log has something like
>>
>> 2019-04-01 11:11:42 [Broker] error Channel exception: session-busy:
>> Session already attached:
>> anonymous.c9a1f4cf-7427-4e77-8870-e118b099c6da
>> (/work/build/rpmbuild/BUILD/msat-qpid-cpp-1.36/msat-qpid-cpp-1.36.0/src/qpid/broker/SessionManager.cpp:55)
>>
>> 2019-04-01 11:11:46 [Protocol] error Connection
>> qpid.10.31.193.215:5672-10.31.192.61:35954 timed out: closing
>>
>> for most or all the reconnects. The first time-stamp seems to
>> correspond to the connection open and "Session detached by peer"
>> exception.
>>
>> Perhaps this means the broker still thinks the old connection is
>> open, even though the client considers it as lost? And maybe the
>> attempt to re-establish/re-use the old session with the new
>> connection fails as a result of that?
>
> Correct. Does the broker log the loss of the connection, e.g. a timed
> out notice? If so how does the time of that compare to the
> session-busy error and to the client detecting the failed connection?
The broker also has a message saying "Connection ... timed out:
closing". This occurs 4-5 seconds after the session-busy error,
according to the timestamps.
- Toralf
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: "Session detached by peer" when trying to re-establish lost
connection
Posted by Gordon Sim <gs...@redhat.com>.
On 01/04/2019 12:54 pm, Toralf Lund wrote:
> On 28/03/2019 15:56, Toralf Lund wrote:
>>
>> Hi,
>>
>> I have another question related to my recent post about trying to
>> re-establish lost connections. Recall that my system is set up without
>> automatic reconnect, but may do a "manual reconnect" via
>>
>> qpid::Messaging connection(url);
>>
>> ...
>>
>> if(!connection.isOpen()) {
>> connection.open();
>> }
>>
>> and that this is sometimes triggered by an "internal disconnect"
>> accompanied by an exception with message "Failed to connect (reconnect
>> disabled)".
>>
>> Now, I just found that some of the reconnects are accompanied by a
>> "Session detached by peer" exception. I'm struggling a bit to find out
>> what exactly triggers this; the problems occur once every few hours at
>> a remote site, and I can't just start or stop or update the software
>> at any old time, so it's a bit hard to debug...
>>
> I just noticed that the broker log has something like
>
> 2019-04-01 11:11:42 [Broker] error Channel exception: session-busy:
> Session already attached:
> anonymous.c9a1f4cf-7427-4e77-8870-e118b099c6da
>
> (/work/build/rpmbuild/BUILD/msat-qpid-cpp-1.36/msat-qpid-cpp-1.36.0/src/qpid/broker/SessionManager.cpp:55)
>
> 2019-04-01 11:11:46 [Protocol] error Connection
> qpid.10.31.193.215:5672-10.31.192.61:35954 timed out: closing
>
> for most or all the reconnects. The first time-stamp seems to correspond
> to the connection open and "Session detached by peer" exception.
>
> Perhaps this means the broker still thinks the old connection is open,
> even though the client considers it as lost? And maybe the attempt to
> re-establish/re-use the old session with the new connection fails as a
> result of that?
Correct. Does the broker log the loss of the connection, e.g. a timed
out notice? If so how does the time of that compare to the session-busy
error and to the client detecting the failed connection?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: "Session detached by peer" when trying to re-establish lost
connection
Posted by Toralf Lund <to...@pgs.com>.
On 28/03/2019 15:56, Toralf Lund wrote:
>
> Hi,
>
> I have another question related to my recent post about trying to
> re-establish lost connections. Recall that my system is set up without
> automatic reconnect, but may do a "manual reconnect" via
>
> qpid::Messaging connection(url);
>
> ...
>
> if(!connection.isOpen()) {
> connection.open();
> }
>
> and that this is sometimes triggered by an "internal disconnect"
> accompanied by an exception with message "Failed to connect (reconnect
> disabled)".
>
> Now, I just found that some of the reconnects are accompanied by a
> "Session detached by peer" exception. I'm struggling a bit to find out
> what exactly triggers this; the problems occur once every few hours at
> a remote site, and I can't just start or stop or update the software
> at any old time, so it's a bit hard to debug...
>
I just noticed that the broker log has something like
2019-04-01 11:11:42 [Broker] error Channel exception: session-busy:
Session already attached:
anonymous.c9a1f4cf-7427-4e77-8870-e118b099c6da
(/work/build/rpmbuild/BUILD/msat-qpid-cpp-1.36/msat-qpid-cpp-1.36.0/src/qpid/broker/SessionManager.cpp:55)
2019-04-01 11:11:46 [Protocol] error Connection
qpid.10.31.193.215:5672-10.31.192.61:35954 timed out: closing
for most or all the reconnects. The first time-stamp seems to correspond
to the connection open and "Session detached by peer" exception.
Perhaps this means the broker still thinks the old connection is open,
even though the client considers it as lost? And maybe the attempt to
re-establish/re-use the old session with the new connection fails as a
result of that?
> My question is: Is there a chance that the above open() might trigger
> this exception as it tries to re-establish a session that was there
> before the disconnect? If that's the case, is there a way I can avoid
> getting into the situation? Would it help to do something like
>
> if(session.isValid() && session.hasError()) {
> session.close();
> }
>
> first? (Where "session" represents the session associated with
> "connection" - there is one at the most.)
>
> Thanks,
>
> - Toralf
>