You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@etch.apache.org by Thomas Marsh <Th...@gamestop.com> on 2011/09/12 23:19:46 UTC
Etch/C Memory Consumption
Hello all,
I have a question about memory consumption in the Etch/C runtime based on behavior we are seeing within our C client. I have modified the C implementation of the HelloWorld example in the distribution to run an infinite loop of requests:
In the main() routine of helloworld_client_main.c:
...
while (1) {
user = new_helloworld_user();
user->id = 5;
user->name = new_stringw(L"User");
result = remote->say_hello(remote, user);
if (is_etch_exception(result)) {
...
}
printf("%S\n", result->v.valw);
etch_object_destroy(result);
}
...
While running this, I see that the memory consumption of the client continually grows. (In this example, it grows by about 1 mB every 5 seconds.) I cannot see the memory leak when testing with valgrind, so it would suggest that the leak is in Etch managed memory which is cleared at exit.
My understanding is that the call to remote->say_hello() should delegate responsibility of deallocation of the parameters to the Etch runtime, and that the client code is only responsible for deallocating the result object. The memory use should be stable within this tight loop. Can anyone comment on the potential cause of the memory consumption?
Thanks, and best regards,
--thomas
RE: Etch/C Memory Consumption
Posted by Thomas Marsh <Th...@gamestop.com>.
Hello Michael,
I have started to investigate the memory consumption issue in more detail, and can confirm it is leaking around 32 bytes per call of say_hello(). That is about the size of the User helloworld_object and the user->name wchar_t which is allocated in the client loop, but clearly must be a copy of that data if this is the issue. The leak traceback is below (for a run with a loop length of 1000):
32,768 bytes in 4 blocks are possibly lost in loss record 567 of 567
at malloc (vg_replace_malloc.c:195)
by apr_pool_create_ex (apr_pools.c:344)
by new_queue (etch_queue.c:59)
by new_mailbox_a (etch_plain_mailbox.c:119)
by new_mailbox (etch_plain_mailbox.c:82)
by pmboxmgr_transport_call (etch_plain_mailbox_manager.c:543)
by tcpdelsvc_begincall (etch_transport.c:525)
by etchremote_begincall (etch_remote.c:129)
by helloworld_remote_begin_server_say_hello (helloworld_remote_server.c:217)
by helloworld_remote_server_say_hello (helloworld_remote_server.c:276)
by main (helloworld_client_main.c:183)
The leak size at this location varies depending on the number of times I iterate over the say_hello() invocation. Here are the leak sizes from this routine at various loop durations:
500 -> 16,384 bytes lost
1,000 -> 32,768 bytes lost
5,000 -> 155,648 bytes lost
10,000 -> 319,488 bytes lost
However, this memory is released upon a call to etch_runtime_shutdown(), so you cannot see the leak unless you interrupt the program execution (or watch the memory grow...).
As for my platform:
- Linux CentOS 5.4 (32 bit i686)
- libapr-1.4.5
- libapr-util-1.3.12
- libapr-iconv-1.2.1
- etch (latest from SVN)
- gcc 4.1.2
I hope that the traceback may give you some clue as to the source of the leak. Otherwise, I will dig deeper when I find more time to focus on this again.
Viele Gruesse,
--thomas
-----Original Message-----
From: Michael Fitzner [mailto:Michael.Fitzner@bmw-carit.de]
Sent: Wednesday, September 14, 2011 6:34 AM
To: etch-dev@incubator.apache.org
Subject: AW: Etch/C Memory Consumption
Hi Thomas,
Your code should be correct and no memory leaks in this part. Could you provide me with some more information about what systems do you use (Linux, Windows) and a call stack of you memory leak if available. I will also try to reproduce your test.
Thanks
Michael
-----Ursprüngliche Nachricht-----
Von: Thomas Marsh [mailto:ThomasMarsh@gamestop.com]
Gesendet: Dienstag, 13. September 2011 16:47
An: etch-dev@incubator.apache.org
Betreff: RE: Etch/C Memory Consumption
Hello Martijn,
Thanks for your response. However, this is not the source of the leak. There is a clear etch_object_destroy in the helloworld_remote_begin_server_say_hello() which deallocates any parameters to the methods (meaning you will get a segfault if you try to reuse the parameters which now no longer point to valid memory). This usage semantic is also clearly stated in the C Binding notes
>From http://incubator.apache.org/etch/c-binding-tips-tricks.html:
The C Binding for Etch has the following memory management rules:
Implementation side: Parameters of functions have to be destroyed by the function implementation using etch_object_destroy.
Caller side: Result Objects have to be freed by the caller using etch_object_destroy. Parameters of calls will be freed by the runtime automatically.
To reiterate, program memory usage in this very simple usage scenario _must_ be stable. I suspect an ever growing hash table, or some other similar culprit within the runtime. Can any of the Etch/C binding developers comment?
Thanks, and best regards,
--thomas
-----Original Message-----
From: Martijn Dashorst [mailto:martijn.dashorst@gmail.com]
Sent: Tuesday, September 13, 2011 7:23 AM
To: etch-dev@incubator.apache.org
Subject: Re: Etch/C Memory Consumption
And user->name = new_stringw(L"User"); will do so as well
Martijn
On Tue, Sep 13, 2011 at 1:52 PM, Martijn Dashorst <ma...@gmail.com> wrote:
> user = new_helloworld_user()
>
> will allocate memory for each pass through the loop.
>
> Martijn
>
> On Mon, Sep 12, 2011 at 11:19 PM, Thomas Marsh <Th...@gamestop.com> wrote:
>> Hello all,
>>
>> I have a question about memory consumption in the Etch/C runtime based on behavior we are seeing within our C client. I have modified the C implementation of the HelloWorld example in the distribution to run an infinite loop of requests:
>>
>> In the main() routine of helloworld_client_main.c:
>>
>> ...
>> while (1) {
>> user = new_helloworld_user();
>> user->id = 5;
>> user->name = new_stringw(L"User");
>> result = remote->say_hello(remote,
>> user);
>> if (is_etch_exception(result)) {
>> ...
>> }
>> printf("%S\n", result->v.valw);
>> etch_object_destroy(result);
>> }
>> ...
>>
>> While running this, I see that the memory consumption of the client continually grows. (In this example, it grows by about 1 mB every 5 seconds.) I cannot see the memory leak when testing with valgrind, so it would suggest that the leak is in Etch managed memory which is cleared at exit.
>>
>> My understanding is that the call to remote->say_hello() should delegate responsibility of deallocation of the parameters to the Etch runtime, and that the client code is only responsible for deallocating the result object. The memory use should be stable within this tight loop. Can anyone comment on the potential cause of the memory consumption?
>>
>> Thanks, and best regards,
>>
>> --thomas
>>
>
>
>
> --
> Become a Wicket expert, learn from the best: http://wicketinaction.com
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
Re: Etch/C Memory Consumption
Posted by Michael Fitzner <fi...@googlemail.com>.
Hi Thomas,
Thanks for further investigations. I will check your patch and give
feedback. It would be still great if you could attach the patch file
to Jira https://issues.apache.org/jira/browse/ETCH-176
Cheers
Michael
2011/9/20 Thomas Marsh <Th...@gamestop.com>:
> Hello Michael and all,
>
> I believe I have located the source of the memory leak. I implemented a fix in my source tree, but due to my lack of familiarity with APR, I am not sure whether there are any issues with my change. The problem lies in the fact that the etch_queue mutexes are created in the context of the g_etch_main_pool, the global apr_pool, and therefore will not be deallocated until etch_runtime_shutdown() is called.
>
> The patch from svn diff is shown below. If there are no issues with this change, I would like to check it in and will close the JIRA issue I submitted.
>
> Thanks, and all the best,
>
> --thomas
>
> Index: common/etch_mutex.c
> ===================================================================
> --- common/etch_mutex.c (revision 1173358)
> +++ common/etch_mutex.c (working copy)
> @@ -68,7 +68,7 @@
> ETCH_ASSERT(newmutex != NULL);
>
> apr_thread_mutex_lock(g_etch_main_pool_mutex);
> - apr_status = apr_thread_mutex_create(&apr_mutex, flags, g_etch_main_pool);
> + apr_status = apr_thread_mutex_create(&apr_mutex, flags, pool);
> apr_thread_mutex_unlock(g_etch_main_pool_mutex);
> if(apr_status != APR_SUCCESS) {
> char temp[1024];
> Index: transport/etch_plain_mailbox.c
> ===================================================================
> --- transport/etch_plain_mailbox.c (revision 1173351)
> +++ transport/etch_plain_mailbox.c (working copy)
> @@ -110,13 +110,14 @@
>
> do
> {
> + if (NULL == (queue = new_queue(capacity))) break;
> +
> // TODO: pool
> - status = etch_mutex_create(&mutex, ETCH_MUTEX_UNNESTED, NULL);
> + status = etch_mutex_create(&mutex, ETCH_MUTEX_UNNESTED, queue->subpool);
> if(status != ETCH_SUCCESS) {
> // error
> break;
> }
> - if (NULL == (queue = new_queue(capacity))) break;
>
> /* - - - - - - - - - - - - - - -
> * i_mailbox
>
> -----Original Message-----
> From: Thomas Marsh
> Sent: Thursday, September 15, 2011 2:41 PM
> To: 'etch-dev@incubator.apache.org'
> Subject: RE: Etch/C Memory Consumption
>
> Hello Michael,
>
> I have started to investigate the memory consumption issue in more detail, and can confirm it is leaking around 32 bytes per call of say_hello(). That is about the size of the User helloworld_object and the user->name wchar_t which is allocated in the client loop, but clearly must be a copy of that data if this is the issue. The leak traceback is below (for a run with a loop length of 1000):
>
> 32,768 bytes in 4 blocks are possibly lost in loss record 567 of 567
> at malloc (vg_replace_malloc.c:195)
> by apr_pool_create_ex (apr_pools.c:344)
> by new_queue (etch_queue.c:59)
> by new_mailbox_a (etch_plain_mailbox.c:119)
> by new_mailbox (etch_plain_mailbox.c:82)
> by pmboxmgr_transport_call (etch_plain_mailbox_manager.c:543)
> by tcpdelsvc_begincall (etch_transport.c:525)
> by etchremote_begincall (etch_remote.c:129)
> by helloworld_remote_begin_server_say_hello (helloworld_remote_server.c:217)
> by helloworld_remote_server_say_hello (helloworld_remote_server.c:276)
> by main (helloworld_client_main.c:183)
>
> The leak size at this location varies depending on the number of times I iterate over the say_hello() invocation. Here are the leak sizes from this routine at various loop durations:
>
> 500 -> 16,384 bytes lost
> 1,000 -> 32,768 bytes lost
> 5,000 -> 155,648 bytes lost
> 10,000 -> 319,488 bytes lost
>
> However, this memory is released upon a call to etch_runtime_shutdown(), so you cannot see the leak unless you interrupt the program execution (or watch the memory grow...).
>
> As for my platform:
> - Linux CentOS 5.4 (32 bit i686)
> - libapr-1.4.5
> - libapr-util-1.3.12
> - libapr-iconv-1.2.1
> - etch (latest from SVN)
> - gcc 4.1.2
>
> I hope that the traceback may give you some clue as to the source of the leak. Otherwise, I will dig deeper when I find more time to focus on this again.
>
> Viele Gruesse,
>
> --thomas
>
> -----Original Message-----
> From: Michael Fitzner [mailto:Michael.Fitzner@bmw-carit.de]
> Sent: Wednesday, September 14, 2011 6:34 AM
> To: etch-dev@incubator.apache.org
> Subject: AW: Etch/C Memory Consumption
>
> Hi Thomas,
> Your code should be correct and no memory leaks in this part. Could you provide me with some more information about what systems do you use (Linux, Windows) and a call stack of you memory leak if available. I will also try to reproduce your test.
>
> Thanks
> Michael
>
> -----Ursprüngliche Nachricht-----
> Von: Thomas Marsh [mailto:ThomasMarsh@gamestop.com]
> Gesendet: Dienstag, 13. September 2011 16:47
> An: etch-dev@incubator.apache.org
> Betreff: RE: Etch/C Memory Consumption
>
> Hello Martijn,
>
> Thanks for your response. However, this is not the source of the leak. There is a clear etch_object_destroy in the helloworld_remote_begin_server_say_hello() which deallocates any parameters to the methods (meaning you will get a segfault if you try to reuse the parameters which now no longer point to valid memory). This usage semantic is also clearly stated in the C Binding notes
>
> From http://incubator.apache.org/etch/c-binding-tips-tricks.html:
>
> The C Binding for Etch has the following memory management rules:
>
> Implementation side: Parameters of functions have to be destroyed by the function implementation using etch_object_destroy.
> Caller side: Result Objects have to be freed by the caller using etch_object_destroy. Parameters of calls will be freed by the runtime automatically.
>
> To reiterate, program memory usage in this very simple usage scenario _must_ be stable. I suspect an ever growing hash table, or some other similar culprit within the runtime. Can any of the Etch/C binding developers comment?
>
> Thanks, and best regards,
>
> --thomas
>
> -----Original Message-----
> From: Martijn Dashorst [mailto:martijn.dashorst@gmail.com]
> Sent: Tuesday, September 13, 2011 7:23 AM
> To: etch-dev@incubator.apache.org
> Subject: Re: Etch/C Memory Consumption
>
> And user->name = new_stringw(L"User"); will do so as well
>
> Martijn
>
> On Tue, Sep 13, 2011 at 1:52 PM, Martijn Dashorst <ma...@gmail.com> wrote:
>> user = new_helloworld_user()
>>
>> will allocate memory for each pass through the loop.
>>
>> Martijn
>>
>> On Mon, Sep 12, 2011 at 11:19 PM, Thomas Marsh <Th...@gamestop.com> wrote:
>>> Hello all,
>>>
>>> I have a question about memory consumption in the Etch/C runtime based on behavior we are seeing within our C client. I have modified the C implementation of the HelloWorld example in the distribution to run an infinite loop of requests:
>>>
>>> In the main() routine of helloworld_client_main.c:
>>>
>>> ...
>>> while (1) {
>>> user = new_helloworld_user();
>>> user->id = 5;
>>> user->name = new_stringw(L"User");
>>> result = remote->say_hello(remote,
>>> user);
>>> if (is_etch_exception(result)) {
>>> ...
>>> }
>>> printf("%S\n", result->v.valw);
>>> etch_object_destroy(result);
>>> }
>>> ...
>>>
>>> While running this, I see that the memory consumption of the client continually grows. (In this example, it grows by about 1 mB every 5 seconds.) I cannot see the memory leak when testing with valgrind, so it would suggest that the leak is in Etch managed memory which is cleared at exit.
>>>
>>> My understanding is that the call to remote->say_hello() should delegate responsibility of deallocation of the parameters to the Etch runtime, and that the client code is only responsible for deallocating the result object. The memory use should be stable within this tight loop. Can anyone comment on the potential cause of the memory consumption?
>>>
>>> Thanks, and best regards,
>>>
>>> --thomas
>>>
>>
>>
>>
>> --
>> Become a Wicket expert, learn from the best: http://wicketinaction.com
>>
>
>
>
> --
> Become a Wicket expert, learn from the best: http://wicketinaction.com
>
RE: Etch/C Memory Consumption
Posted by Thomas Marsh <Th...@gamestop.com>.
Hello Michael and all,
I believe I have located the source of the memory leak. I implemented a fix in my source tree, but due to my lack of familiarity with APR, I am not sure whether there are any issues with my change. The problem lies in the fact that the etch_queue mutexes are created in the context of the g_etch_main_pool, the global apr_pool, and therefore will not be deallocated until etch_runtime_shutdown() is called.
The patch from svn diff is shown below. If there are no issues with this change, I would like to check it in and will close the JIRA issue I submitted.
Thanks, and all the best,
--thomas
Index: common/etch_mutex.c
===================================================================
--- common/etch_mutex.c (revision 1173358)
+++ common/etch_mutex.c (working copy)
@@ -68,7 +68,7 @@
ETCH_ASSERT(newmutex != NULL);
apr_thread_mutex_lock(g_etch_main_pool_mutex);
- apr_status = apr_thread_mutex_create(&apr_mutex, flags, g_etch_main_pool);
+ apr_status = apr_thread_mutex_create(&apr_mutex, flags, pool);
apr_thread_mutex_unlock(g_etch_main_pool_mutex);
if(apr_status != APR_SUCCESS) {
char temp[1024];
Index: transport/etch_plain_mailbox.c
===================================================================
--- transport/etch_plain_mailbox.c (revision 1173351)
+++ transport/etch_plain_mailbox.c (working copy)
@@ -110,13 +110,14 @@
do
{
+ if (NULL == (queue = new_queue(capacity))) break;
+
// TODO: pool
- status = etch_mutex_create(&mutex, ETCH_MUTEX_UNNESTED, NULL);
+ status = etch_mutex_create(&mutex, ETCH_MUTEX_UNNESTED, queue->subpool);
if(status != ETCH_SUCCESS) {
// error
break;
}
- if (NULL == (queue = new_queue(capacity))) break;
/* - - - - - - - - - - - - - - -
* i_mailbox
-----Original Message-----
From: Thomas Marsh
Sent: Thursday, September 15, 2011 2:41 PM
To: 'etch-dev@incubator.apache.org'
Subject: RE: Etch/C Memory Consumption
Hello Michael,
I have started to investigate the memory consumption issue in more detail, and can confirm it is leaking around 32 bytes per call of say_hello(). That is about the size of the User helloworld_object and the user->name wchar_t which is allocated in the client loop, but clearly must be a copy of that data if this is the issue. The leak traceback is below (for a run with a loop length of 1000):
32,768 bytes in 4 blocks are possibly lost in loss record 567 of 567
at malloc (vg_replace_malloc.c:195)
by apr_pool_create_ex (apr_pools.c:344)
by new_queue (etch_queue.c:59)
by new_mailbox_a (etch_plain_mailbox.c:119)
by new_mailbox (etch_plain_mailbox.c:82)
by pmboxmgr_transport_call (etch_plain_mailbox_manager.c:543)
by tcpdelsvc_begincall (etch_transport.c:525)
by etchremote_begincall (etch_remote.c:129)
by helloworld_remote_begin_server_say_hello (helloworld_remote_server.c:217)
by helloworld_remote_server_say_hello (helloworld_remote_server.c:276)
by main (helloworld_client_main.c:183)
The leak size at this location varies depending on the number of times I iterate over the say_hello() invocation. Here are the leak sizes from this routine at various loop durations:
500 -> 16,384 bytes lost
1,000 -> 32,768 bytes lost
5,000 -> 155,648 bytes lost
10,000 -> 319,488 bytes lost
However, this memory is released upon a call to etch_runtime_shutdown(), so you cannot see the leak unless you interrupt the program execution (or watch the memory grow...).
As for my platform:
- Linux CentOS 5.4 (32 bit i686)
- libapr-1.4.5
- libapr-util-1.3.12
- libapr-iconv-1.2.1
- etch (latest from SVN)
- gcc 4.1.2
I hope that the traceback may give you some clue as to the source of the leak. Otherwise, I will dig deeper when I find more time to focus on this again.
Viele Gruesse,
--thomas
-----Original Message-----
From: Michael Fitzner [mailto:Michael.Fitzner@bmw-carit.de]
Sent: Wednesday, September 14, 2011 6:34 AM
To: etch-dev@incubator.apache.org
Subject: AW: Etch/C Memory Consumption
Hi Thomas,
Your code should be correct and no memory leaks in this part. Could you provide me with some more information about what systems do you use (Linux, Windows) and a call stack of you memory leak if available. I will also try to reproduce your test.
Thanks
Michael
-----Ursprüngliche Nachricht-----
Von: Thomas Marsh [mailto:ThomasMarsh@gamestop.com]
Gesendet: Dienstag, 13. September 2011 16:47
An: etch-dev@incubator.apache.org
Betreff: RE: Etch/C Memory Consumption
Hello Martijn,
Thanks for your response. However, this is not the source of the leak. There is a clear etch_object_destroy in the helloworld_remote_begin_server_say_hello() which deallocates any parameters to the methods (meaning you will get a segfault if you try to reuse the parameters which now no longer point to valid memory). This usage semantic is also clearly stated in the C Binding notes
>From http://incubator.apache.org/etch/c-binding-tips-tricks.html:
The C Binding for Etch has the following memory management rules:
Implementation side: Parameters of functions have to be destroyed by the function implementation using etch_object_destroy.
Caller side: Result Objects have to be freed by the caller using etch_object_destroy. Parameters of calls will be freed by the runtime automatically.
To reiterate, program memory usage in this very simple usage scenario _must_ be stable. I suspect an ever growing hash table, or some other similar culprit within the runtime. Can any of the Etch/C binding developers comment?
Thanks, and best regards,
--thomas
-----Original Message-----
From: Martijn Dashorst [mailto:martijn.dashorst@gmail.com]
Sent: Tuesday, September 13, 2011 7:23 AM
To: etch-dev@incubator.apache.org
Subject: Re: Etch/C Memory Consumption
And user->name = new_stringw(L"User"); will do so as well
Martijn
On Tue, Sep 13, 2011 at 1:52 PM, Martijn Dashorst <ma...@gmail.com> wrote:
> user = new_helloworld_user()
>
> will allocate memory for each pass through the loop.
>
> Martijn
>
> On Mon, Sep 12, 2011 at 11:19 PM, Thomas Marsh <Th...@gamestop.com> wrote:
>> Hello all,
>>
>> I have a question about memory consumption in the Etch/C runtime based on behavior we are seeing within our C client. I have modified the C implementation of the HelloWorld example in the distribution to run an infinite loop of requests:
>>
>> In the main() routine of helloworld_client_main.c:
>>
>> ...
>> while (1) {
>> user = new_helloworld_user();
>> user->id = 5;
>> user->name = new_stringw(L"User");
>> result = remote->say_hello(remote,
>> user);
>> if (is_etch_exception(result)) {
>> ...
>> }
>> printf("%S\n", result->v.valw);
>> etch_object_destroy(result);
>> }
>> ...
>>
>> While running this, I see that the memory consumption of the client continually grows. (In this example, it grows by about 1 mB every 5 seconds.) I cannot see the memory leak when testing with valgrind, so it would suggest that the leak is in Etch managed memory which is cleared at exit.
>>
>> My understanding is that the call to remote->say_hello() should delegate responsibility of deallocation of the parameters to the Etch runtime, and that the client code is only responsible for deallocating the result object. The memory use should be stable within this tight loop. Can anyone comment on the potential cause of the memory consumption?
>>
>> Thanks, and best regards,
>>
>> --thomas
>>
>
>
>
> --
> Become a Wicket expert, learn from the best: http://wicketinaction.com
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
AW: Etch/C Memory Consumption
Posted by Michael Fitzner <Mi...@bmw-carit.de>.
Hi Thomas,
Your code should be correct and no memory leaks in this part. Could you provide me with some more information about what systems do you use (Linux, Windows) and a call stack of you memory leak if available. I will also try to reproduce your test.
Thanks
Michael
-----Ursprüngliche Nachricht-----
Von: Thomas Marsh [mailto:ThomasMarsh@gamestop.com]
Gesendet: Dienstag, 13. September 2011 16:47
An: etch-dev@incubator.apache.org
Betreff: RE: Etch/C Memory Consumption
Hello Martijn,
Thanks for your response. However, this is not the source of the leak. There is a clear etch_object_destroy in the helloworld_remote_begin_server_say_hello() which deallocates any parameters to the methods (meaning you will get a segfault if you try to reuse the parameters which now no longer point to valid memory). This usage semantic is also clearly stated in the C Binding notes
>From http://incubator.apache.org/etch/c-binding-tips-tricks.html:
The C Binding for Etch has the following memory management rules:
Implementation side: Parameters of functions have to be destroyed by the function implementation using etch_object_destroy.
Caller side: Result Objects have to be freed by the caller using etch_object_destroy. Parameters of calls will be freed by the runtime automatically.
To reiterate, program memory usage in this very simple usage scenario _must_ be stable. I suspect an ever growing hash table, or some other similar culprit within the runtime. Can any of the Etch/C binding developers comment?
Thanks, and best regards,
--thomas
-----Original Message-----
From: Martijn Dashorst [mailto:martijn.dashorst@gmail.com]
Sent: Tuesday, September 13, 2011 7:23 AM
To: etch-dev@incubator.apache.org
Subject: Re: Etch/C Memory Consumption
And user->name = new_stringw(L"User"); will do so as well
Martijn
On Tue, Sep 13, 2011 at 1:52 PM, Martijn Dashorst <ma...@gmail.com> wrote:
> user = new_helloworld_user()
>
> will allocate memory for each pass through the loop.
>
> Martijn
>
> On Mon, Sep 12, 2011 at 11:19 PM, Thomas Marsh <Th...@gamestop.com> wrote:
>> Hello all,
>>
>> I have a question about memory consumption in the Etch/C runtime based on behavior we are seeing within our C client. I have modified the C implementation of the HelloWorld example in the distribution to run an infinite loop of requests:
>>
>> In the main() routine of helloworld_client_main.c:
>>
>> ...
>> while (1) {
>> user = new_helloworld_user();
>> user->id = 5;
>> user->name = new_stringw(L"User");
>> result = remote->say_hello(remote,
>> user);
>> if (is_etch_exception(result)) {
>> ...
>> }
>> printf("%S\n", result->v.valw);
>> etch_object_destroy(result);
>> }
>> ...
>>
>> While running this, I see that the memory consumption of the client continually grows. (In this example, it grows by about 1 mB every 5 seconds.) I cannot see the memory leak when testing with valgrind, so it would suggest that the leak is in Etch managed memory which is cleared at exit.
>>
>> My understanding is that the call to remote->say_hello() should delegate responsibility of deallocation of the parameters to the Etch runtime, and that the client code is only responsible for deallocating the result object. The memory use should be stable within this tight loop. Can anyone comment on the potential cause of the memory consumption?
>>
>> Thanks, and best regards,
>>
>> --thomas
>>
>
>
>
> --
> Become a Wicket expert, learn from the best: http://wicketinaction.com
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
RE: Etch/C Memory Consumption
Posted by Thomas Marsh <Th...@gamestop.com>.
Hello Martijn,
Thanks for your response. However, this is not the source of the leak. There is a clear etch_object_destroy in the helloworld_remote_begin_server_say_hello() which deallocates any parameters to the methods (meaning you will get a segfault if you try to reuse the parameters which now no longer point to valid memory). This usage semantic is also clearly stated in the C Binding notes
>From http://incubator.apache.org/etch/c-binding-tips-tricks.html:
The C Binding for Etch has the following memory management rules:
Implementation side: Parameters of functions have to be destroyed by the function implementation using etch_object_destroy.
Caller side: Result Objects have to be freed by the caller using etch_object_destroy. Parameters of calls will be freed by the runtime automatically.
To reiterate, program memory usage in this very simple usage scenario _must_ be stable. I suspect an ever growing hash table, or some other similar culprit within the runtime. Can any of the Etch/C binding developers comment?
Thanks, and best regards,
--thomas
-----Original Message-----
From: Martijn Dashorst [mailto:martijn.dashorst@gmail.com]
Sent: Tuesday, September 13, 2011 7:23 AM
To: etch-dev@incubator.apache.org
Subject: Re: Etch/C Memory Consumption
And user->name = new_stringw(L"User"); will do so as well
Martijn
On Tue, Sep 13, 2011 at 1:52 PM, Martijn Dashorst <ma...@gmail.com> wrote:
> user = new_helloworld_user()
>
> will allocate memory for each pass through the loop.
>
> Martijn
>
> On Mon, Sep 12, 2011 at 11:19 PM, Thomas Marsh <Th...@gamestop.com> wrote:
>> Hello all,
>>
>> I have a question about memory consumption in the Etch/C runtime based on behavior we are seeing within our C client. I have modified the C implementation of the HelloWorld example in the distribution to run an infinite loop of requests:
>>
>> In the main() routine of helloworld_client_main.c:
>>
>> ...
>> while (1) {
>> user = new_helloworld_user();
>> user->id = 5;
>> user->name = new_stringw(L"User");
>> result = remote->say_hello(remote,
>> user);
>> if (is_etch_exception(result)) {
>> ...
>> }
>> printf("%S\n", result->v.valw);
>> etch_object_destroy(result);
>> }
>> ...
>>
>> While running this, I see that the memory consumption of the client continually grows. (In this example, it grows by about 1 mB every 5 seconds.) I cannot see the memory leak when testing with valgrind, so it would suggest that the leak is in Etch managed memory which is cleared at exit.
>>
>> My understanding is that the call to remote->say_hello() should delegate responsibility of deallocation of the parameters to the Etch runtime, and that the client code is only responsible for deallocating the result object. The memory use should be stable within this tight loop. Can anyone comment on the potential cause of the memory consumption?
>>
>> Thanks, and best regards,
>>
>> --thomas
>>
>
>
>
> --
> Become a Wicket expert, learn from the best: http://wicketinaction.com
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
Re: Etch/C Memory Consumption
Posted by Martijn Dashorst <ma...@gmail.com>.
And user->name = new_stringw(L"User"); will do so as well
Martijn
On Tue, Sep 13, 2011 at 1:52 PM, Martijn Dashorst
<ma...@gmail.com> wrote:
> user = new_helloworld_user()
>
> will allocate memory for each pass through the loop.
>
> Martijn
>
> On Mon, Sep 12, 2011 at 11:19 PM, Thomas Marsh <Th...@gamestop.com> wrote:
>> Hello all,
>>
>> I have a question about memory consumption in the Etch/C runtime based on behavior we are seeing within our C client. I have modified the C implementation of the HelloWorld example in the distribution to run an infinite loop of requests:
>>
>> In the main() routine of helloworld_client_main.c:
>>
>> ...
>> while (1) {
>> user = new_helloworld_user();
>> user->id = 5;
>> user->name = new_stringw(L"User");
>> result = remote->say_hello(remote, user);
>> if (is_etch_exception(result)) {
>> ...
>> }
>> printf("%S\n", result->v.valw);
>> etch_object_destroy(result);
>> }
>> ...
>>
>> While running this, I see that the memory consumption of the client continually grows. (In this example, it grows by about 1 mB every 5 seconds.) I cannot see the memory leak when testing with valgrind, so it would suggest that the leak is in Etch managed memory which is cleared at exit.
>>
>> My understanding is that the call to remote->say_hello() should delegate responsibility of deallocation of the parameters to the Etch runtime, and that the client code is only responsible for deallocating the result object. The memory use should be stable within this tight loop. Can anyone comment on the potential cause of the memory consumption?
>>
>> Thanks, and best regards,
>>
>> --thomas
>>
>
>
>
> --
> Become a Wicket expert, learn from the best: http://wicketinaction.com
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
Re: Etch/C Memory Consumption
Posted by Martijn Dashorst <ma...@gmail.com>.
user = new_helloworld_user()
will allocate memory for each pass through the loop.
Martijn
On Mon, Sep 12, 2011 at 11:19 PM, Thomas Marsh <Th...@gamestop.com> wrote:
> Hello all,
>
> I have a question about memory consumption in the Etch/C runtime based on behavior we are seeing within our C client. I have modified the C implementation of the HelloWorld example in the distribution to run an infinite loop of requests:
>
> In the main() routine of helloworld_client_main.c:
>
> ...
> while (1) {
> user = new_helloworld_user();
> user->id = 5;
> user->name = new_stringw(L"User");
> result = remote->say_hello(remote, user);
> if (is_etch_exception(result)) {
> ...
> }
> printf("%S\n", result->v.valw);
> etch_object_destroy(result);
> }
> ...
>
> While running this, I see that the memory consumption of the client continually grows. (In this example, it grows by about 1 mB every 5 seconds.) I cannot see the memory leak when testing with valgrind, so it would suggest that the leak is in Etch managed memory which is cleared at exit.
>
> My understanding is that the call to remote->say_hello() should delegate responsibility of deallocation of the parameters to the Etch runtime, and that the client code is only responsible for deallocating the result object. The memory use should be stable within this tight loop. Can anyone comment on the potential cause of the memory consumption?
>
> Thanks, and best regards,
>
> --thomas
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com