You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by "Qin, Dennis" <De...@thalesesec.com> on 2010/12/08 23:39:16 UTC
qpid C++ memory leak
Hi All,
I have downloaded the Linux version of qpid C++ broker 0.8 on my Linux appliance. I ran some stress tests on this broker server and found memory leaks. Here is the simple test I did.
1. Launch the C++ broker using the following CLI,
./src/qpid --daemon
2. Launch the server program under the examples/request-response directory,
3. Create a shell script to send client messages constantly to the server under the examples/request-response directory , here is my script,
#!/bin/bash
while true
do
./client
done
and launch this script
4. Check the C++ broker memory usage using ps auxf and pmap.
It shows memory usage constantly increasing as the script running to send the message.
Looks to me there are somewhere inside the broker code that keeps allocating the memory and not releasing it.
Any idea if this issue has already been reported and will be fixed in the next release?
5. Here is my appliance Linux version,
2.6.18-92.1.6.el5PAE #1 SMP Wed Jun 25 14:21:46 EDT 2008 i686 i686 i386 GNU/Linux
Thanks.
Regards,
Dennis Qin
Re: qpid C++ memory leak
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Hi Dennis,
Hope this [1] will help you.
[1]
http://qpid.apache.org/apis/0.7/cpp/html/a00164.html#a97ff0ef9c47f55ed0f167c574e951b4a
Regards
Lahiru
On Sat, Dec 11, 2010 at 1:00 AM, Qin, Dennis <De...@thalesesec.com>wrote:
> Hi Lahiru,
>
>
>
> Thanks for the info. With this tool, I was able to nail it down to the
> problem in the client.cpp file. What happened was that the queue was never
> been deleted. The qpid tool shows that the number of queues is getting
> bigger and bigger as I’m continuously running the client script. I have made
> the following change in the code,
>
>
>
> before:
>
> session.queueDeclare(arg::queue=response_queue.str());
>
>
>
> now:
>
> session.queueDeclare(arg::queue=response_queue.str(),
> arg::autoDelete=true);
>
>
>
> that will force the queue get automatically deleted once it’s been consumed
> by the client.
>
>
>
> I also would like to know how to call the session.queuePurge() or session. queueDelete()
> APIs to delete the queue specifically once it’s been consumed. Is there any
> sample code for these calls?
>
>
>
> Thanks.
>
>
>
> Regards,
>
>
>
> Dennis Qin
>
>
>
> *From:* Lahiru Gunathilake [mailto:glahiru@gmail.com]
> *Sent:* Thursday, December 09, 2010 6:56 PM
> *To:* users@qpid.apache.org
> *Cc:* Faruqui, Faisal; Ramalingam, Saravanan; Kuo, Jennshi; Harpalani,
> Manish; Qin, Dennis
> *Subject:* Re: qpid C++ memory leak
>
>
>
> Hi Dennis,
>
>
>
> First it was a mistake I mention about Java broker, I am talking about C++
> broker. You can easily monitor whether messages are consuming or not by
> using python tools[1] provided in qpid and see whether incoming messages are
> consumed.
>
>
>
> [1]
> http://www.lahiru.org/2010/12/how-to-monitor-and-manage-apache-qpid-c.html
>
> Regards
>
> Lahiru
>
> On Thu, Dec 9, 2010 at 10:57 PM, Dennis Qin <de...@gmail.com> wrote:
>
> Hi Lahiru,
>
>
>
> Thanks for the reply. But I was just using the examples code coming from
> qpid package. Also I was running against C++ broker, not Java. For your
> convenience, I have attached the source code of the server and client
> programs with this email. They are located in the
> "qpidc-0.8/examples/request-response" qpid package directory. The third one
> is the script I have created to shoot client messages to the server. Would
> you be able to elaborate as where the problem is in the sample code?
>
>
>
> Thanks.
>
>
>
> Regards,
>
>
>
> Dennis Qin
>
>
>
> On Thu, Dec 9, 2010 at 6:36 AM, Lahiru Gunathilake <gl...@gmail.com>
> wrote:
>
> Hi Dennis,
>
> As far as I can see you are simply sending large number of messages to Qpid
> Java broker but not consuming any of the messages. So obviously memory will
> grow unless you consume those messages since you are running without
> persistent storage.
>
> I recently did stress test with Qpid C++ broker with large messages up to
> 5MB and I couldn't find memory leaks with valgrind.
>
> Hope this helps !
>
> Regards
> Lahiru
>
> On Thu, Dec 9, 2010 at 4:09 AM, Qin, Dennis <Dennis.Qin@thalesesec.com
> >wrote:
>
>
> > Hi All,
> >
> > I have downloaded the Linux version of qpid C++ broker 0.8 on my Linux
> > appliance. I ran some stress tests on this broker server and found memory
> > leaks. Here is the simple test I did.
> >
> >
> > 1. Launch the C++ broker using the following CLI,
> >
> > ./src/qpid --daemon
> >
> > 2. Launch the server program under the examples/request-response
> > directory,
> >
> > 3. Create a shell script to send client messages constantly to the
> > server under the examples/request-response directory , here is my script,
> >
> >
> >
> > #!/bin/bash
> >
> >
> >
> > while true
> >
> > do
> >
> > ./client
> >
> > done
> >
> >
> >
> > and launch this script
> >
> >
> >
> > 4. Check the C++ broker memory usage using ps auxf and pmap.
> >
> >
> >
> > It shows memory usage constantly increasing as the script running to send
> > the message.
> >
> >
> >
> > Looks to me there are somewhere inside the broker code that keeps
> > allocating the memory and not releasing it.
> >
> >
> >
> > Any idea if this issue has already been reported and will be fixed in the
> > next release?
> >
> >
> > 5. Here is my appliance Linux version,
> >
> > 2.6.18-92.1.6.el5PAE #1 SMP Wed Jun 25 14:21:46 EDT 2008
> i686
> > i686 i386 GNU/Linux
> >
> >
> >
> > Thanks.
> >
> >
> >
> > Regards,
> >
> >
> >
> > Dennis Qin
> >
> >
> >
> >
>
>
>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
>
RE: qpid C++ memory leak
Posted by "Qin, Dennis" <De...@thalesesec.com>.
Hi Lahiru,
Thanks for the info. With this tool, I was able to nail it down to the problem in the client.cpp file. What happened was that the queue was never been deleted. The qpid tool shows that the number of queues is getting bigger and bigger as I'm continuously running the client script. I have made the following change in the code,
before:
session.queueDeclare(arg::queue=response_queue.str());
now:
session.queueDeclare(arg::queue=response_queue.str(), arg::autoDelete=true);
that will force the queue get automatically deleted once it's been consumed by the client.
I also would like to know how to call the session.queuePurge() or session. queueDelete() APIs to delete the queue specifically once it's been consumed. Is there any sample code for these calls?
Thanks.
Regards,
Dennis Qin
From: Lahiru Gunathilake [mailto:glahiru@gmail.com]
Sent: Thursday, December 09, 2010 6:56 PM
To: users@qpid.apache.org
Cc: Faruqui, Faisal; Ramalingam, Saravanan; Kuo, Jennshi; Harpalani, Manish; Qin, Dennis
Subject: Re: qpid C++ memory leak
Hi Dennis,
First it was a mistake I mention about Java broker, I am talking about C++ broker. You can easily monitor whether messages are consuming or not by using python tools[1] provided in qpid and see whether incoming messages are consumed.
[1]http://www.lahiru.org/2010/12/how-to-monitor-and-manage-apache-qpid-c.html
Regards
Lahiru
On Thu, Dec 9, 2010 at 10:57 PM, Dennis Qin <de...@gmail.com>> wrote:
Hi Lahiru,
Thanks for the reply. But I was just using the examples code coming from qpid package. Also I was running against C++ broker, not Java. For your convenience, I have attached the source code of the server and client programs with this email. They are located in the "qpidc-0.8/examples/request-response" qpid package directory. The third one is the script I have created to shoot client messages to the server. Would you be able to elaborate as where the problem is in the sample code?
Thanks.
Regards,
Dennis Qin
On Thu, Dec 9, 2010 at 6:36 AM, Lahiru Gunathilake <gl...@gmail.com>> wrote:
Hi Dennis,
As far as I can see you are simply sending large number of messages to Qpid
Java broker but not consuming any of the messages. So obviously memory will
grow unless you consume those messages since you are running without
persistent storage.
I recently did stress test with Qpid C++ broker with large messages up to
5MB and I couldn't find memory leaks with valgrind.
Hope this helps !
Regards
Lahiru
On Thu, Dec 9, 2010 at 4:09 AM, Qin, Dennis <De...@thalesesec.com>>wrote:
> Hi All,
>
> I have downloaded the Linux version of qpid C++ broker 0.8 on my Linux
> appliance. I ran some stress tests on this broker server and found memory
> leaks. Here is the simple test I did.
>
>
> 1. Launch the C++ broker using the following CLI,
>
> ./src/qpid --daemon
>
> 2. Launch the server program under the examples/request-response
> directory,
>
> 3. Create a shell script to send client messages constantly to the
> server under the examples/request-response directory , here is my script,
>
>
>
> #!/bin/bash
>
>
>
> while true
>
> do
>
> ./client
>
> done
>
>
>
> and launch this script
>
>
>
> 4. Check the C++ broker memory usage using ps auxf and pmap.
>
>
>
> It shows memory usage constantly increasing as the script running to send
> the message.
>
>
>
> Looks to me there are somewhere inside the broker code that keeps
> allocating the memory and not releasing it.
>
>
>
> Any idea if this issue has already been reported and will be fixed in the
> next release?
>
>
> 5. Here is my appliance Linux version,
>
> 2.6.18-92.1.6.el5PAE #1 SMP Wed Jun 25 14:21:46 EDT 2008 i686
> i686 i386 GNU/Linux
>
>
>
> Thanks.
>
>
>
> Regards,
>
>
>
> Dennis Qin
>
>
>
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org<ma...@qpid.apache.org>
Re: qpid C++ memory leak
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Hi Dennis,
First it was a mistake I mention about Java broker, I am talking about C++
broker. You can easily monitor whether messages are consuming or not by
using python tools[1] provided in qpid and see whether incoming messages are
consumed.
[1]
http://www.lahiru.org/2010/12/how-to-monitor-and-manage-apache-qpid-c.html
Regards
Lahiru
On Thu, Dec 9, 2010 at 10:57 PM, Dennis Qin <de...@gmail.com> wrote:
> Hi Lahiru,
>
> Thanks for the reply. But I was just using the examples code coming from
> qpid package. Also I was running against C++ broker, not Java. For your
> convenience, I have attached the source code of the server and client
> programs with this email. They are located in the
> "qpidc-0.8/examples/request-response" qpid package directory. The third one
> is the script I have created to shoot client messages to the server. Would
> you be able to elaborate as where the problem is in the sample code?
>
> Thanks.
>
> Regards,
>
> Dennis Qin
>
>
> On Thu, Dec 9, 2010 at 6:36 AM, Lahiru Gunathilake <gl...@gmail.com>wrote:
>
>> Hi Dennis,
>>
>> As far as I can see you are simply sending large number of messages to
>> Qpid
>> Java broker but not consuming any of the messages. So obviously memory
>> will
>> grow unless you consume those messages since you are running without
>> persistent storage.
>>
>> I recently did stress test with Qpid C++ broker with large messages up to
>> 5MB and I couldn't find memory leaks with valgrind.
>>
>> Hope this helps !
>>
>> Regards
>> Lahiru
>>
>> On Thu, Dec 9, 2010 at 4:09 AM, Qin, Dennis <Dennis.Qin@thalesesec.com
>> >wrote:
>>
>> > Hi All,
>> >
>> > I have downloaded the Linux version of qpid C++ broker 0.8 on my Linux
>> > appliance. I ran some stress tests on this broker server and found
>> memory
>> > leaks. Here is the simple test I did.
>> >
>> >
>> > 1. Launch the C++ broker using the following CLI,
>> >
>> > ./src/qpid --daemon
>> >
>> > 2. Launch the server program under the examples/request-response
>> > directory,
>> >
>> > 3. Create a shell script to send client messages constantly to the
>> > server under the examples/request-response directory , here is my
>> script,
>> >
>> >
>> >
>> > #!/bin/bash
>> >
>> >
>> >
>> > while true
>> >
>> > do
>> >
>> > ./client
>> >
>> > done
>> >
>> >
>> >
>> > and launch this script
>> >
>> >
>> >
>> > 4. Check the C++ broker memory usage using ps auxf and pmap.
>> >
>> >
>> >
>> > It shows memory usage constantly increasing as the script running to
>> send
>> > the message.
>> >
>> >
>> >
>> > Looks to me there are somewhere inside the broker code that keeps
>> > allocating the memory and not releasing it.
>> >
>> >
>> >
>> > Any idea if this issue has already been reported and will be fixed in
>> the
>> > next release?
>> >
>> >
>> > 5. Here is my appliance Linux version,
>> >
>> > 2.6.18-92.1.6.el5PAE #1 SMP Wed Jun 25 14:21:46 EDT 2008
>> i686
>> > i686 i386 GNU/Linux
>> >
>> >
>> >
>> > Thanks.
>> >
>> >
>> >
>> > Regards,
>> >
>> >
>> >
>> > Dennis Qin
>> >
>> >
>> >
>> >
>>
>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
Re: qpid C++ memory leak
Posted by Dennis Qin <de...@gmail.com>.
Hi Lahiru,
Thanks for the reply. But I was just using the examples code coming from
qpid package. Also I was running against C++ broker, not Java. For your
convenience, I have attached the source code of the server and client
programs with this email. They are located in the
"qpidc-0.8/examples/request-response" qpid package directory. The third one
is the script I have created to shoot client messages to the server. Would
you be able to elaborate as where the problem is in the sample code?
Thanks.
Regards,
Dennis Qin
On Thu, Dec 9, 2010 at 6:36 AM, Lahiru Gunathilake <gl...@gmail.com>wrote:
> Hi Dennis,
>
> As far as I can see you are simply sending large number of messages to Qpid
> Java broker but not consuming any of the messages. So obviously memory will
> grow unless you consume those messages since you are running without
> persistent storage.
>
> I recently did stress test with Qpid C++ broker with large messages up to
> 5MB and I couldn't find memory leaks with valgrind.
>
> Hope this helps !
>
> Regards
> Lahiru
>
> On Thu, Dec 9, 2010 at 4:09 AM, Qin, Dennis <Dennis.Qin@thalesesec.com
> >wrote:
>
> > Hi All,
> >
> > I have downloaded the Linux version of qpid C++ broker 0.8 on my Linux
> > appliance. I ran some stress tests on this broker server and found memory
> > leaks. Here is the simple test I did.
> >
> >
> > 1. Launch the C++ broker using the following CLI,
> >
> > ./src/qpid --daemon
> >
> > 2. Launch the server program under the examples/request-response
> > directory,
> >
> > 3. Create a shell script to send client messages constantly to the
> > server under the examples/request-response directory , here is my script,
> >
> >
> >
> > #!/bin/bash
> >
> >
> >
> > while true
> >
> > do
> >
> > ./client
> >
> > done
> >
> >
> >
> > and launch this script
> >
> >
> >
> > 4. Check the C++ broker memory usage using ps auxf and pmap.
> >
> >
> >
> > It shows memory usage constantly increasing as the script running to send
> > the message.
> >
> >
> >
> > Looks to me there are somewhere inside the broker code that keeps
> > allocating the memory and not releasing it.
> >
> >
> >
> > Any idea if this issue has already been reported and will be fixed in the
> > next release?
> >
> >
> > 5. Here is my appliance Linux version,
> >
> > 2.6.18-92.1.6.el5PAE #1 SMP Wed Jun 25 14:21:46 EDT 2008
> i686
> > i686 i386 GNU/Linux
> >
> >
> >
> > Thanks.
> >
> >
> >
> > Regards,
> >
> >
> >
> > Dennis Qin
> >
> >
> >
> >
>
Re: qpid C++ memory leak
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Hi Dennis,
As far as I can see you are simply sending large number of messages to Qpid
Java broker but not consuming any of the messages. So obviously memory will
grow unless you consume those messages since you are running without
persistent storage.
I recently did stress test with Qpid C++ broker with large messages up to
5MB and I couldn't find memory leaks with valgrind.
Hope this helps !
Regards
Lahiru
On Thu, Dec 9, 2010 at 4:09 AM, Qin, Dennis <De...@thalesesec.com>wrote:
> Hi All,
>
> I have downloaded the Linux version of qpid C++ broker 0.8 on my Linux
> appliance. I ran some stress tests on this broker server and found memory
> leaks. Here is the simple test I did.
>
>
> 1. Launch the C++ broker using the following CLI,
>
> ./src/qpid --daemon
>
> 2. Launch the server program under the examples/request-response
> directory,
>
> 3. Create a shell script to send client messages constantly to the
> server under the examples/request-response directory , here is my script,
>
>
>
> #!/bin/bash
>
>
>
> while true
>
> do
>
> ./client
>
> done
>
>
>
> and launch this script
>
>
>
> 4. Check the C++ broker memory usage using ps auxf and pmap.
>
>
>
> It shows memory usage constantly increasing as the script running to send
> the message.
>
>
>
> Looks to me there are somewhere inside the broker code that keeps
> allocating the memory and not releasing it.
>
>
>
> Any idea if this issue has already been reported and will be fixed in the
> next release?
>
>
> 5. Here is my appliance Linux version,
>
> 2.6.18-92.1.6.el5PAE #1 SMP Wed Jun 25 14:21:46 EDT 2008 i686
> i686 i386 GNU/Linux
>
>
>
> Thanks.
>
>
>
> Regards,
>
>
>
> Dennis Qin
>
>
>
>