You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by jglu2000 <ji...@hotmail.com> on 2013/05/15 17:53:52 UTC
Qpid throwed WSAENOBUFS while receiving data from a broker
We have a Windows Qpid 0.20 C++ client that receives huge amount of data from
a Qpid broker. The other day, after enabling the debug+ logging level on
Qpid, we saw the following message recorded in the Qpid log file:
2013-05-14 10:44:41 [System] debug Exception constructed: An operation on a
socket could not be performed because the system lacked sufficient buffer
space or because a queue was full.
(...\qpid\sys\windows\SslAsynchIO.cpp:354)
While researching the message "An operation on a socket could not be
performed...", I have found that it is for error WSAENOBUFS on Windows.
Most people talked about the operating systems running out of memory for TCP
buffers or TCP "ephemeral" ports, but when I looked up the specific line of
codes in Qpid, I saw that it was thrown by the following line in Qpid, not
by a Windows socket-related function calls:
void SslAsynchIO::sslDataIn(qpid::sys::AsynchIO& a, BufferBase *buff) {
...
extraBuff = a.getQueuedBuffer();
if (0 == extraBuff)
throw QPID_WINDOWS_ERROR(WSAENOBUFS);
After the exception, our client continued to receive several messages before
it stopped, and when we tried to send requests to the broker, we got no
responses. We are not sure that the later events were related to the
exception. We just like to know what does the exception mean.
Thanks.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by jglu2000 <ji...@hotmail.com>.
Hi Steve,
Thank you very much for your response. As I mentioned before, we have not
seen the issue in our in-house tests. But I will try to come up with some
test cases to reproduce the issue.
Thanks again,
Gene
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7593280.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by jingheelu <ji...@hotmail.com>.
Since my last post, we have done a number of tests involving some
combinations of the following changes to determine which is more helpful in
alleviating this issue:
1. Increasing the buffer count from 4 to 8;
2. Reducing the receiver capacity from 100 to 10;
3. Upgrading from Windows Server 2003 to 2008;
Our conclusion is that #1 and #2 will delay the issue, but eventually, the
exception still happened. Upgrading our operating systems to run our
application under Windows Server 2008 seems to be the winner, as we have not
seen the issue happens on that version of OS. According to our research,
Windows Server 2008 has a redesigned network stack implementation, and we
believe that it is helping to prevent the issue from happening.
Thanks again for all your replies and helpful suggestions.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7597675.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by jingheelu <ji...@hotmail.com>.
Hi Jakub,
Thank you for your efforts in reproducing the issue.
We have been running some tests after increasing the buffer count from 4 to
8. It seems to help a bit in our high load tests, in term of delaying the
exception, but eventually, the exception still occurred after longer runs.
We will also try your other recommendation, which is to reduce the receiver
capacity and see if it helps (currently at 100).
Thanks again.
Sincerely,
Jin-Ghee Lu (Gene)
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7596328.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by Jakub Scholz <ja...@scholz.cz>.
Hi Hamid,
I'm sorry, I do not really know the idea behind the buffers. Steve
mentioned the possibility of increasing the buffer count. From the code
where the exception is created it is apparent that there is additional
buffer missing, but I do not really know how the SSL data are decrypted and
stored into the buffers and why do they actually need additional buffer. I
just gave it a try :-).
I didn't tried to further increase the number of buffers. I changed it to 5
and it seemed to work, so there was no reason for it. Today I moved more
then 10GB of messages of different sizes with different capacity settings
and the problem didn't appeared any-more with 5 buffers.
Thanks & Regards
Jakub
On Thu, Aug 1, 2013 at 7:40 PM, Hamid.Shahid <ha...@sungard.com>wrote:
> Hi Jakub,
>
> It is really good to know that increasing the BufferCount value to 5 helps
> to resolve the problem.
> Should we double the BufferCount from 4 to 8 instead? Or do you thing there
> any be any negative effects of doing this?
>
> Thanks
>
>
>
>
> -----
> Best Regards,
> Hamid.
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7596320.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by "Hamid.Shahid" <ha...@sungard.com>.
Hi Jakub,
It is really good to know that increasing the BufferCount value to 5 helps
to resolve the problem.
Should we double the BufferCount from 4 to 8 instead? Or do you thing there
any be any negative effects of doing this?
Thanks
-----
Best Regards,
Hamid.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7596320.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by Jakub Scholz <ja...@scholz.cz>.
One more update from my side ... I finally managed to build the 0.22
release. The problem is fully reproducible there as well. However, I tried
to increase the BufferCount value in AsynchIO.h from 4 to 5 and that seems
to solve the problem - at least in the terms that the error doesn't
reproduce anymore.
Thanks & Regards
Jakub
On Thu, Aug 1, 2013 at 11:37 AM, Hamid.Shahid <ha...@sungard.com>wrote:
> Hi Jakub,
>
> Thank you for investigation and keeping the thread alive. I hope we will
> get
> to a conclusion soon about how to fix this.
>
>
>
>
> -----
> Best Regards,
> Hamid.
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7596281.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by "Hamid.Shahid" <ha...@sungard.com>.
Hi Jakub,
Thank you for investigation and keeping the thread alive. I hope we will get
to a conclusion soon about how to fix this.
-----
Best Regards,
Hamid.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7596281.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by Jakub Scholz <ja...@scholz.cz>.
As usually, half of the attachments didn't made it to the conference. I
created a JIRA QPID-5033 and attached both files there. I guess sooner or
later the JIRA will be anyway needed.
Thanks & Regards
Jakub
On Wed, Jul 31, 2013 at 7:38 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> Hi all,
>
> This issue doesn't seem to be so complicated to reproduce. I used a
> following scenario:
>
> 1) Create a large queue on a broker (C++ / Linux)
> 2) Start feeding messages into the queue using C++/Linux program (in my
> case I used approximately 1kB messages)
> 3) Connect with a receiver (C++/Windows) using SSL and prefetch 1000 (no
> client authentication, I used username & password)
> 4) Wait few seconds to see the error in the receiver
>
> For the C++/Windows receiver, I used the Red Hat MRG-M 2.3.3 WinSDK
> (building it from Qpid spource codes almost never works for me :-o). But
> given the experience from Hamid and Gene, I do not think this might work
> the same way with builds based on latest Qpid sources.
>
> The key to reproducing it seems to be the capacity & the SSL. Without SSL
> it doesn't seem to be reproducible. With SSL and small receiver capacity
> (in my scenario above, with capacity 50 it seems to need much more time to
> fail, with capacity 10 I didn't managed to reproduce it), it seems to be
> working fine as well. In a similar way - increasing the message size to 1MB
> causes even the capacity 1 to stop working.
>
> Source code of the receiver as well as its full trace+ log are attached.
>
> @Steve: Is there something else I can collect / do to help find and solve
> the problem?
>
> Thanks & Regards
> Jakub
>
>
>
> On Mon, Jul 29, 2013 at 4:24 PM, Hamid.Shahid <ha...@sungard.com>wrote:
>
>> It is the same case for us, we have seen this issue in production
>> environment
>> more then 10 times using Qpid 0.18.
>>
>> After reading the post, I went through the code of QPID 0.18 and I found
>> that there is no BufferCount specified. Now, I have downloaded the latest
>> QPID 0.22 C++ version on windows 7 as I saw the BufferCount logic in 0.22
>> version.
>>
>> Please let me know, is there any recommend number for BufferCount?
>>
>> Moreover, surprisingly, I still have to do the SSL patching, I dont know,
>> why its sill not part of the API?
>>
>> Thanks,
>> Hamid.
>>
>>
>>
>> -----
>> Best Regards,
>> Hamid.
>> --
>> View this message in context:
>> http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7596041.html
>> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
>
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by Jakub Scholz <ja...@scholz.cz>.
Hi all,
This issue doesn't seem to be so complicated to reproduce. I used a
following scenario:
1) Create a large queue on a broker (C++ / Linux)
2) Start feeding messages into the queue using C++/Linux program (in my
case I used approximately 1kB messages)
3) Connect with a receiver (C++/Windows) using SSL and prefetch 1000 (no
client authentication, I used username & password)
4) Wait few seconds to see the error in the receiver
For the C++/Windows receiver, I used the Red Hat MRG-M 2.3.3 WinSDK
(building it from Qpid spource codes almost never works for me :-o). But
given the experience from Hamid and Gene, I do not think this might work
the same way with builds based on latest Qpid sources.
The key to reproducing it seems to be the capacity & the SSL. Without SSL
it doesn't seem to be reproducible. With SSL and small receiver capacity
(in my scenario above, with capacity 50 it seems to need much more time to
fail, with capacity 10 I didn't managed to reproduce it), it seems to be
working fine as well. In a similar way - increasing the message size to 1MB
causes even the capacity 1 to stop working.
Source code of the receiver as well as its full trace+ log are attached.
@Steve: Is there something else I can collect / do to help find and solve
the problem?
Thanks & Regards
Jakub
On Mon, Jul 29, 2013 at 4:24 PM, Hamid.Shahid <ha...@sungard.com>wrote:
> It is the same case for us, we have seen this issue in production
> environment
> more then 10 times using Qpid 0.18.
>
> After reading the post, I went through the code of QPID 0.18 and I found
> that there is no BufferCount specified. Now, I have downloaded the latest
> QPID 0.22 C++ version on windows 7 as I saw the BufferCount logic in 0.22
> version.
>
> Please let me know, is there any recommend number for BufferCount?
>
> Moreover, surprisingly, I still have to do the SSL patching, I dont know,
> why its sill not part of the API?
>
> Thanks,
> Hamid.
>
>
>
> -----
> Best Regards,
> Hamid.
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7596041.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by "Hamid.Shahid" <ha...@sungard.com>.
It is the same case for us, we have seen this issue in production environment
more then 10 times using Qpid 0.18.
After reading the post, I went through the code of QPID 0.18 and I found
that there is no BufferCount specified. Now, I have downloaded the latest
QPID 0.22 C++ version on windows 7 as I saw the BufferCount logic in 0.22
version.
Please let me know, is there any recommend number for BufferCount?
Moreover, surprisingly, I still have to do the SSL patching, I dont know,
why its sill not part of the API?
Thanks,
Hamid.
-----
Best Regards,
Hamid.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7596041.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by jingheelu <ji...@hotmail.com>.
We have been trying to perform in-house tests in a more controlled testing
environment (so that we can run debugger and other tools) to reproduce this
issue with no luck. But we are continuously getting reports from our
production environment. One reason we think the issue is happening more
frequent in production is that we have a lot of incoming messages from the
brokers (external to us at a different institution). Some information about
our configurations and patterns we have seen in production after enabling
the DEBUG+ logging for Qpid:
- Windows Server 2003
- Qpid 0.20 build on Windows / Visual C++ 2010
- SSL enabled with client/server certificates
- Made changes based on https://issues.apache.org/jira/browse/QPID-3914 to
resolve client certificate authentication issue in Windows
- Our Windows client has been set up to have 3 client queues to receive
incoming messages from brokers, with capacities set to 100. One of the
queues is busier than the other two, with continuous downloads coming from
the brokers.
- The busy queue could see as many as 100 messages per second being
"delivered" or "pushed".
- Prior to the exception, we usually saw a lot more "pushed" than
"delivered".
We had speculated that maybe our thread that is responsible for fetching had
stopped fetching or receiving, and we had tried to simulate that by not
fetching / receiving. But Qpid would simply signal the brokers to stop
sending, and no exception was observed.
We were hesitant to put in the BufferCount changes without knowing more.
But given the circumstance, we will give it a try. Will update this post
one way or another.
Thanks,
Jin-Ghee Lu (Gene)
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7595892.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by Steve Huston <sh...@riverace.com>.
The original poster didn't send a test case or open a jira so it hasn't been worked on.
Steve Huston
(sent from my iPhone - please excuse brevity and typos)
On Jul 24, 2013, at 12:49 PM, "Hamid.Shahid" <ha...@sungard.com> wrote:
> Hi Gene / Steve,
>
> We are having this issue once or twice every week in QPID 0.18 windows
> client.
>
> Have you guys got a chance to test this after increasing BufferCount in
> qpid/cpp/src/qpid/sys/AsynchIO.h?
>
> If yes, can you please share patch?
>
> Thank you!
>
> - Hamid
>
>
>
> -----
> Best Regards,
> Hamid.
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7595858.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by "Hamid.Shahid" <ha...@sungard.com>.
Hi Gene / Steve,
We are having this issue once or twice every week in QPID 0.18 windows
client.
Have you guys got a chance to test this after increasing BufferCount in
qpid/cpp/src/qpid/sys/AsynchIO.h?
If yes, can you please share patch?
Thank you!
- Hamid
-----
Best Regards,
Hamid.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7595858.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by Steve Huston <sh...@riverace.com>.
Hi Gene,
Apologies for the delay.
> 1. So, do anyone have any recommendation on how we can allocate more
> buffer space programmatically to avoid this problem?
You can't. You could try increasing BufferCount in qpid/cpp/src/qpid/sys/AsynchIO.h and rebuilding but I don't think that's a real solution at this point.
> 2. Could the exception in theory have caused the connection issues that I
> mentioned in the Windows build?
Definitely.
I think the problem is in the SSL handling code, not in the number of buffers. However, I'd really like to see a reproducing test case to look closer at it.
-Steve
> View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-
> throwed-WSAENOBUFS-while-receiving-data-from-a-broker-
> tp7592938p7593249.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For additional
> commands, e-mail: users-help@qpid.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by jglu2000 <ji...@hotmail.com>.
Hi,
1. So, do anyone have any recommendation on how we can allocate more buffer
space programmatically to avoid this problem?
2. Could the exception in theory have caused the connection issues that I
mentioned in the Windows build?
Thanks,
Jin-Ghee Lu (Gene)
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7593249.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by jglu2000 <ji...@hotmail.com>.
Unfortunately, we have only seen this issue in our production environment,
where there are large amount of incoming messages from the broker. If we
can reliably reproduce the issue, I will raise a JIRA. Some follow-up
questions:
1. Can we allocate more buffer space ourselves programmatically?
2. Could the exception in theory have caused the connection issues that I
mentioned? I.e. after the exception, the Qpid debug log showed that we
continued to receive several messages, then everything stopped, our client
applications sent some requests, but no responses came back.
The following are some excerpts from the Qpid log:
2013-05-14 10:44:41 [Client] debug Pushed {MessageTransferBody:
destination=<Client Broadcast A>; accept-mode=0; acquire-mode=0; } to
received queue
2013-05-14 10:44:41 [Client] debug Pushed {MessageTransferBody:
destination=<Client Broadcast A>; accept-mode=0; acquire-mode=0; } to
received queue
2013-05-14 10:44:41 [Client] debug Pushed {MessageTransferBody:
destination=<Client Broadcast A>; accept-mode=0; acquire-mode=0; } to
received queue
2013-05-14 10:44:41 [System] debug Exception constructed: An operation on a
socket could not be performed because the system lacked sufficient buffer
space or because a queue was full.
(...\src\qpid\sys\windows\SslAsynchIO.cpp:354)
(after the exception, continues to receive...)
2013-05-14 10:44:41 [Client] debug Pushed {MessageTransferBody:
destination=<Client Broadcast A>; accept-mode=0; acquire-mode=0; } to
received queue
2013-05-14 10:44:41 [Client] debug Pushed {MessageTransferBody:
destination=<Client Broadcast A>; accept-mode=0; acquire-mode=0; } to
received queue
...
2013-05-14 10:44:41 [Client] debug Pushed {MessageTransferBody:
destination=<Client Broadcast B>; accept-mode=0; acquire-mode=0; } to
received queue
2013-05-14 10:44:41 [Client] debug Pushed {MessageTransferBody:
destination=<Client Broadcast B>; accept-mode=0; acquire-mode=0; } to
received queue
2013-05-14 10:44:41 [Client] debug Pushed {MessageTransferBody:
destination=<Client Broadcast B>; accept-mode=0; acquire-mode=0; } to
received queue
(the last broadcast message received, no more coming in, but I am pretty
there were more messages)
(some times later, we tried to send some requests)
2013-05-14 11:17:17 [Client] debug Sending to exchange <request.client>
{MessageProperties: content-length=655; correlation-id=; reply-to={ReplyTo:
exchange=response; routing-key=<response.client.response_queue1>; };
content-type=; user-id=; application-headers={}; } {DeliveryProperties:
routing-key=; }
2013-05-14 11:19:31 [Client] debug Sending to exchange <request.client>
{MessageProperties: content-length=655; correlation-id=; reply-to={ReplyTo:
exchange=response; routing-key=<response.client.response_queue1>; };
content-type=; user-id=; application-headers={}; } {DeliveryProperties:
routing-key=; }
(there were supposed to be acknowledgment or responses for the requests, but
it seems everything stopped, we could't send, nor could we receive any
responses or broadcasts from the broker)
(we brought down the client and tried to reconnect)
2013-05-14 11:26:05 [Client] debug Created connection
amqp:tcp:127.0.0.1:5672 with {}
2013-05-14 11:26:05 [Client] debug Created connection amqp:ssl:<broker
host>:<port> with {}
2013-05-14 11:26:05 [Client] debug Starting connection,
urls=[amqp:ssl:<broker host>:<port>]
2013-05-14 11:26:05 [Client] info Trying to connect to amqp:ssl:<broker
host>:<port>...
Thanks.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-throwed-WSAENOBUFS-while-receiving-data-from-a-broker-tp7592938p7592973.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: Qpid throwed WSAENOBUFS while receiving data from a broker
Posted by Steve Huston <sh...@riverace.com>.
> -----Original Message-----
> From: jglu2000 [mailto:jingheelu@hotmail.com]
> Sent: Wednesday, May 15, 2013 11:54 AM
> To: users@qpid.apache.org
> Subject: Qpid throwed WSAENOBUFS while receiving data from a broker
>
> We have a Windows Qpid 0.20 C++ client that receives huge amount of data
> from a Qpid broker. The other day, after enabling the debug+ logging level
> on Qpid, we saw the following message recorded in the Qpid log file:
>
> 2013-05-14 10:44:41 [System] debug Exception constructed: An operation on
> a socket could not be performed because the system lacked sufficient buffer
> space or because a queue was full.
> (...\qpid\sys\windows\SslAsynchIO.cpp:354)
>
> While researching the message "An operation on a socket could not be
> performed...", I have found that it is for error WSAENOBUFS on Windows.
> Most people talked about the operating systems running out of memory for
> TCP buffers or TCP "ephemeral" ports, but when I looked up the specific line
> of codes in Qpid, I saw that it was thrown by the following line in Qpid, not by
> a Windows socket-related function calls:
>
> void SslAsynchIO::sslDataIn(qpid::sys::AsynchIO& a, BufferBase *buff) { ...
> extraBuff = a.getQueuedBuffer();
> if (0 == extraBuff)
> throw QPID_WINDOWS_ERROR(WSAENOBUFS);
>
> After the exception, our client continued to receive several messages before
> it stopped, and when we tried to send requests to the broker, we got no
> responses. We are not sure that the later events were related to the
> exception. We just like to know what does the exception mean.
The exception means the pre-allocated buffers in the AsynchIO object for the connection have been used up. I'm not sure if this is some sort of logic error or there should be more buffer space allocated. If you can develop a reliable way to reproduce this and raise a JIRA for the problem that would be very helpful.
Thanks,
-Steve Huston
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org