You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Barys Ilyushonak (JIRA)" <ji...@apache.org> on 2011/05/19 14:10:47 UTC
[jira] [Created] (QPID-3264) Channel exception: session-busy:
Session already attached
Channel exception: session-busy: Session already attached
---------------------------------------------------------
Key: QPID-3264
URL: https://issues.apache.org/jira/browse/QPID-3264
Project: Qpid
Issue Type: Bug
Components: C++ Broker, C++ Client
Affects Versions: 0.8
Environment: Broker on RedHat 5.6 64bit; Client on Windows 7.
Reporter: Barys Ilyushonak
While client reconnect we got:
2011-05-11 16:15:07 error Channel exception: session-busy: Session already attached: .a7235961-c1f9-4a76-923c-ff06b09ad0c5 (qpid/broker/SessionManager.cpp:55)
To reproduce:
Start linux broker on default port 5672. Create durable queue and exchange.
Start the broker. Start client. Client sent the first message. We sow this message on the broker. After that enable IPTABLES to reject the port. Client still sent messages, but there was no new messages on the broker. After that disable IPTABLES to reject the port.
It is important do not restart the client.
After that we have got the error in the broker log.
On the client:
std::auto_ptr<qpid::messaging::Connection> m_connection
m_connection.reset(new Connection(m_url));
m_connection->setOption("reconnect", true);
We limit network access on the broker machine. To close the port we use:
iptables -A INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
where 10.30.24.181 - windows client IP
to enable access we use:
iptables -D INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3264) Channel exception: session-busy:
Session already attached
Posted by "Gordon Sim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039057#comment-13039057 ]
Gordon Sim commented on QPID-3264:
----------------------------------
"Why It is impossible to setup c++ client to create new session while reconnect failure?" - it is not impossible, however it may not be sufficient either. If the old session is still considered 'live' by the server than any unacknowledged messages remain locked, any exclusive queues remain etc. In other words I think choosing a new session name might simply be masking the problem rather than solving it.
> Channel exception: session-busy: Session already attached
> ---------------------------------------------------------
>
> Key: QPID-3264
> URL: https://issues.apache.org/jira/browse/QPID-3264
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker, C++ Client
> Affects Versions: 0.8
> Environment: Broker on RedHat 5.6 64bit; Client on Windows 7.
> Reporter: Barys Ilyushonak
>
> While client reconnect we got:
> 2011-05-11 16:15:07 error Channel exception: session-busy: Session already attached: .a7235961-c1f9-4a76-923c-ff06b09ad0c5 (qpid/broker/SessionManager.cpp:55)
> To reproduce:
> Start linux broker on default port 5672. Create durable queue and exchange.
> Start the broker. Start client. Client sent the first message. We sow this message on the broker. After that enable IPTABLES to reject the port. Client still sent messages, but there was no new messages on the broker. After that disable IPTABLES to reject the port.
> It is important do not restart the client.
> After that we have got the error in the broker log.
> On the client:
> std::auto_ptr<qpid::messaging::Connection> m_connection
> m_connection.reset(new Connection(m_url));
> m_connection->setOption("reconnect", true);
> We limit network access on the broker machine. To close the port we use:
> iptables -A INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
> where 10.30.24.181 - windows client IP
> to enable access we use:
> iptables -D INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3264) Channel exception: session-busy:
Session already attached
Posted by "Pavel Moravec (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13037825#comment-13037825 ]
Pavel Moravec commented on QPID-3264:
-------------------------------------
The case seems for me hard to be fully fixed. See below my analysis of what happens (using Linux terminology, but analogue behavior is correct for Widnows as well):
1) When TCP retransmission gives up, kernel sends ETIMEDOUT to userspace and moves
the local socket from ESTABLISHED to CLOSED status - i.e. _no_ RST packet is
sent on the connection. This results in situation where:
a) the client does not have the TCP connection established
b) the server _has_ the connection established (in ESTABLISHED state even)
c) the server has not received any indication that the client has some
transmitting problems. The server can detect the TCP connection being broken
only when sending TCP keepalive probe (based on the default setting of kernel
parameter net.ipv4.tcp_keepalive_time, it will be done after 3 hours)
That seems to follow RFCs (at least, I haven't found any indication that
dropping the TCP connection on client side in the way above is not allowed).
2) The client tries to reconnect to the server using new TCP session but
previously used connection ID. That leads to the AMQP session rejection.
I don't see an easy fix of this - both sides are behaving properly per information and state they have and are in..
I see two workarounds here:
1) Use heartbeats to detect the network problems sooner than TCP and OS do so. I.e. set heartbeats to at most 1/2 of TCP retransmission overall-time (that by default around 15 minutes on Linux and 12 seconds(!) on Windows).
2) Align number of TCP retransmissions so that any (expected) network outage is shorter than the full TCP retransmission round. Then, some retransmission will be successfull and no disconnection would happen.
> Channel exception: session-busy: Session already attached
> ---------------------------------------------------------
>
> Key: QPID-3264
> URL: https://issues.apache.org/jira/browse/QPID-3264
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker, C++ Client
> Affects Versions: 0.8
> Environment: Broker on RedHat 5.6 64bit; Client on Windows 7.
> Reporter: Barys Ilyushonak
>
> While client reconnect we got:
> 2011-05-11 16:15:07 error Channel exception: session-busy: Session already attached: .a7235961-c1f9-4a76-923c-ff06b09ad0c5 (qpid/broker/SessionManager.cpp:55)
> To reproduce:
> Start linux broker on default port 5672. Create durable queue and exchange.
> Start the broker. Start client. Client sent the first message. We sow this message on the broker. After that enable IPTABLES to reject the port. Client still sent messages, but there was no new messages on the broker. After that disable IPTABLES to reject the port.
> It is important do not restart the client.
> After that we have got the error in the broker log.
> On the client:
> std::auto_ptr<qpid::messaging::Connection> m_connection
> m_connection.reset(new Connection(m_url));
> m_connection->setOption("reconnect", true);
> We limit network access on the broker machine. To close the port we use:
> iptables -A INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
> where 10.30.24.181 - windows client IP
> to enable access we use:
> iptables -D INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3264) Channel exception: session-busy:
Session already attached
Posted by "Pavel Moravec (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038425#comment-13038425 ]
Pavel Moravec commented on QPID-3264:
-------------------------------------
Question to Barys: "Why It is impossible to setup c++ client to create new session while reconnect failure?" - can you please elaborate? You mean when automatic reconnection fails due to the case above, you run some commands attempting to reconnect explicitly but the client fail to be connected? Can you paste relevant part of the source code, please?
> Channel exception: session-busy: Session already attached
> ---------------------------------------------------------
>
> Key: QPID-3264
> URL: https://issues.apache.org/jira/browse/QPID-3264
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker, C++ Client
> Affects Versions: 0.8
> Environment: Broker on RedHat 5.6 64bit; Client on Windows 7.
> Reporter: Barys Ilyushonak
>
> While client reconnect we got:
> 2011-05-11 16:15:07 error Channel exception: session-busy: Session already attached: .a7235961-c1f9-4a76-923c-ff06b09ad0c5 (qpid/broker/SessionManager.cpp:55)
> To reproduce:
> Start linux broker on default port 5672. Create durable queue and exchange.
> Start the broker. Start client. Client sent the first message. We sow this message on the broker. After that enable IPTABLES to reject the port. Client still sent messages, but there was no new messages on the broker. After that disable IPTABLES to reject the port.
> It is important do not restart the client.
> After that we have got the error in the broker log.
> On the client:
> std::auto_ptr<qpid::messaging::Connection> m_connection
> m_connection.reset(new Connection(m_url));
> m_connection->setOption("reconnect", true);
> We limit network access on the broker machine. To close the port we use:
> iptables -A INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
> where 10.30.24.181 - windows client IP
> to enable access we use:
> iptables -D INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3264) Channel exception: session-busy:
Session already attached
Posted by "Barys Ilyushonak (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13037893#comment-13037893 ]
Barys Ilyushonak commented on QPID-3264:
----------------------------------------
We have checked heartbeats internally and it looks like good solution.
Why It is impossible to setup c++ client to create new session while reconnect failure?
> Channel exception: session-busy: Session already attached
> ---------------------------------------------------------
>
> Key: QPID-3264
> URL: https://issues.apache.org/jira/browse/QPID-3264
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker, C++ Client
> Affects Versions: 0.8
> Environment: Broker on RedHat 5.6 64bit; Client on Windows 7.
> Reporter: Barys Ilyushonak
>
> While client reconnect we got:
> 2011-05-11 16:15:07 error Channel exception: session-busy: Session already attached: .a7235961-c1f9-4a76-923c-ff06b09ad0c5 (qpid/broker/SessionManager.cpp:55)
> To reproduce:
> Start linux broker on default port 5672. Create durable queue and exchange.
> Start the broker. Start client. Client sent the first message. We sow this message on the broker. After that enable IPTABLES to reject the port. Client still sent messages, but there was no new messages on the broker. After that disable IPTABLES to reject the port.
> It is important do not restart the client.
> After that we have got the error in the broker log.
> On the client:
> std::auto_ptr<qpid::messaging::Connection> m_connection
> m_connection.reset(new Connection(m_url));
> m_connection->setOption("reconnect", true);
> We limit network access on the broker machine. To close the port we use:
> iptables -A INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
> where 10.30.24.181 - windows client IP
> to enable access we use:
> iptables -D INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3264) Channel exception: session-busy:
Session already attached
Posted by "Jason Dillaman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070582#comment-13070582 ]
Jason Dillaman commented on QPID-3264:
--------------------------------------
We have run into this issue as well in our testing environment. Setting the QPID heartbeat to a short interval helps, but it doesn't solve it 100% of the time. As an alternative suggestion, what if the broker sends a sync to the original session whenever a new session attempts to attach to the same name? If the original session doesn't respond within a timeout, the original session is declared lost and the broker can respond to the new session attach request successfully. If the original session does respond within the timeout, the broker can respond w/ the session attach error.
> Channel exception: session-busy: Session already attached
> ---------------------------------------------------------
>
> Key: QPID-3264
> URL: https://issues.apache.org/jira/browse/QPID-3264
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker, C++ Client
> Affects Versions: 0.8
> Environment: Broker on RedHat 5.6 64bit; Client on Windows 7.
> Reporter: Barys Ilyushonak
>
> While client reconnect we got:
> 2011-05-11 16:15:07 error Channel exception: session-busy: Session already attached: .a7235961-c1f9-4a76-923c-ff06b09ad0c5 (qpid/broker/SessionManager.cpp:55)
> To reproduce:
> Start linux broker on default port 5672. Create durable queue and exchange.
> Start the broker. Start client. Client sent the first message. We sow this message on the broker. After that enable IPTABLES to reject the port. Client still sent messages, but there was no new messages on the broker. After that disable IPTABLES to reject the port.
> It is important do not restart the client.
> After that we have got the error in the broker log.
> On the client:
> std::auto_ptr<qpid::messaging::Connection> m_connection
> m_connection.reset(new Connection(m_url));
> m_connection->setOption("reconnect", true);
> We limit network access on the broker machine. To close the port we use:
> iptables -A INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
> where 10.30.24.181 - windows client IP
> to enable access we use:
> iptables -D INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3264) Channel exception: session-busy:
Session already attached
Posted by "Gordon Sim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13037888#comment-13037888 ]
Gordon Sim commented on QPID-3264:
----------------------------------
I think workaround 1 is a good one. We should document that somewhere, or perhaps even see if we can set the default appropriately in code.
> Channel exception: session-busy: Session already attached
> ---------------------------------------------------------
>
> Key: QPID-3264
> URL: https://issues.apache.org/jira/browse/QPID-3264
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker, C++ Client
> Affects Versions: 0.8
> Environment: Broker on RedHat 5.6 64bit; Client on Windows 7.
> Reporter: Barys Ilyushonak
>
> While client reconnect we got:
> 2011-05-11 16:15:07 error Channel exception: session-busy: Session already attached: .a7235961-c1f9-4a76-923c-ff06b09ad0c5 (qpid/broker/SessionManager.cpp:55)
> To reproduce:
> Start linux broker on default port 5672. Create durable queue and exchange.
> Start the broker. Start client. Client sent the first message. We sow this message on the broker. After that enable IPTABLES to reject the port. Client still sent messages, but there was no new messages on the broker. After that disable IPTABLES to reject the port.
> It is important do not restart the client.
> After that we have got the error in the broker log.
> On the client:
> std::auto_ptr<qpid::messaging::Connection> m_connection
> m_connection.reset(new Connection(m_url));
> m_connection->setOption("reconnect", true);
> We limit network access on the broker machine. To close the port we use:
> iptables -A INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
> where 10.30.24.181 - windows client IP
> to enable access we use:
> iptables -D INPUT -p tcp -s 10.30.24.181 --dport 5672 -j REJECT
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org