You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@opennlp.apache.org by Samarth Seksaria <sa...@germinait.com> on 2013/10/25 08:26:14 UTC
Memory Leak Fix in 1.5.2
Hi,
I am using opennlp for generating parse trees. After quite an amount of
analysis using MAT ( memory analyzer tool ) on eclipse, I suspect the
accumulation of objects is in an instance of the chunking.Parser class.
I may be completely wrong, and the issue may lie somewhere in my
application code. I am still trying to find other possibilities. Still,
would like to get an heads up from someone who is aware of, if any, memory
leak issues in opennlp.
--
Thanks & Regards,
Samarth Seksaria
Re: Memory Leak Fix in 1.5.2
Posted by Samarth Seksaria <sa...@germinait.com>.
Just found the memory leak in another library we were using, GATE.
No memory leak issues in opennlp.
Thanks.
On 30 October 2013 15:26, Jörn Kottmann <ko...@gmail.com> wrote:
> Looks good to me, its needs quite a bit of memory to hold all these
> statistical models.
>
> Does your memory usage grows after processing millions of sentences?
>
> Jörn
>
>
> On 10/25/2013 08:36 AM, Samarth Seksaria wrote:
>
>> Following is an extract from my analysis using MAT:
>>
>> The object of type chunking.Parser has the following memory consumption:
>>
>> Class Name | Shallow Heap |
>> Retained Heap
>> ------------------------------**------------------------------**
>> -----------------------------
>> cParser opennlp.tools.parser.chunking.**Parser @ 0x79604bb08|
>> 104 |
>> *27,41,83,328*
>>
>> ------------------------------**------------------------------**
>> -----------------------------
>>
>> Within this object, following are the objects that have the majority of
>> the
>> memory accumulation:
>>
>> Class Name | Shallow Heap | Retained
>> Heap
>> ------------------------------**------------------------------**
>> -------------------
>> buildModel opennlp.maxent.GISModel @ 0x79249e968| 32 | *
>> 17,02,70,504*
>>
>> ------------------------------**------------------------------**
>> -------------------
>> Class Name | Shallow Heap |
>> Retained Heap
>> ------------------------------**------------------------------**
>> ------------------------
>> chunker opennlp.tools.chunker.**ChunkerME @ 0x79604c850| 24 |
>> *
>> 4,62,08,192*
>>
>> ------------------------------**------------------------------**
>> ------------------------
>> Class Name | Shallow Heap | Retained
>> Heap
>> ------------------------------**------------------------------**
>> -------------------
>> checkModel opennlp.maxent.GISModel @ 0x788ee2810| 32 | *
>> 3,69,60,416*
>>
>> ------------------------------**------------------------------**
>> -------------------
>> Class Name | Shallow Heap |
>> Retained Heap
>> ------------------------------**------------------------------**
>> ------------------------
>> tagger opennlp.tools.postag.**POSTaggerME @ 0x79604c370| 48 |
>> *
>> 2,05,61,544*
>>
>> ------------------------------**------------------------------**
>> ------------------------
>>
>>
>>
>> On 25 October 2013 11:56, Samarth Seksaria
>> <samarth.seksaria@germinait.**com <sa...@germinait.com>>wrote:
>>
>> Hi,
>>>
>>> I am using opennlp for generating parse trees. After quite an amount of
>>> analysis using MAT ( memory analyzer tool ) on eclipse, I suspect the
>>> accumulation of objects is in an instance of the chunking.Parser class.
>>>
>>> I may be completely wrong, and the issue may lie somewhere in my
>>> application code. I am still trying to find other possibilities. Still,
>>> would like to get an heads up from someone who is aware of, if any,
>>> memory
>>> leak issues in opennlp.
>>>
>>> --
>>> Thanks & Regards,
>>> Samarth Seksaria
>>>
>>>
>>
>>
>
--
Thanks & Regards,
Samarth Seksaria
Re: Memory Leak Fix in 1.5.2
Posted by Jörn Kottmann <ko...@gmail.com>.
Looks good to me, its needs quite a bit of memory to hold all these
statistical models.
Does your memory usage grows after processing millions of sentences?
Jörn
On 10/25/2013 08:36 AM, Samarth Seksaria wrote:
> Following is an extract from my analysis using MAT:
>
> The object of type chunking.Parser has the following memory consumption:
>
> Class Name | Shallow Heap |
> Retained Heap
> -----------------------------------------------------------------------------------------
> cParser opennlp.tools.parser.chunking.Parser @ 0x79604bb08| 104 |
> *27,41,83,328*
> -----------------------------------------------------------------------------------------
>
> Within this object, following are the objects that have the majority of the
> memory accumulation:
>
> Class Name | Shallow Heap | Retained
> Heap
> -------------------------------------------------------------------------------
> buildModel opennlp.maxent.GISModel @ 0x79249e968| 32 | *
> 17,02,70,504*
> -------------------------------------------------------------------------------
> Class Name | Shallow Heap |
> Retained Heap
> ------------------------------------------------------------------------------------
> chunker opennlp.tools.chunker.ChunkerME @ 0x79604c850| 24 | *
> 4,62,08,192*
> ------------------------------------------------------------------------------------
> Class Name | Shallow Heap | Retained
> Heap
> -------------------------------------------------------------------------------
> checkModel opennlp.maxent.GISModel @ 0x788ee2810| 32 | *
> 3,69,60,416*
> -------------------------------------------------------------------------------
> Class Name | Shallow Heap |
> Retained Heap
> ------------------------------------------------------------------------------------
> tagger opennlp.tools.postag.POSTaggerME @ 0x79604c370| 48 | *
> 2,05,61,544*
> ------------------------------------------------------------------------------------
>
>
>
> On 25 October 2013 11:56, Samarth Seksaria
> <sa...@germinait.com>wrote:
>
>> Hi,
>>
>> I am using opennlp for generating parse trees. After quite an amount of
>> analysis using MAT ( memory analyzer tool ) on eclipse, I suspect the
>> accumulation of objects is in an instance of the chunking.Parser class.
>>
>> I may be completely wrong, and the issue may lie somewhere in my
>> application code. I am still trying to find other possibilities. Still,
>> would like to get an heads up from someone who is aware of, if any, memory
>> leak issues in opennlp.
>>
>> --
>> Thanks & Regards,
>> Samarth Seksaria
>>
>
>
Re: Memory Leak Fix in 1.5.2
Posted by Samarth Seksaria <sa...@germinait.com>.
Following is an extract from my analysis using MAT:
The object of type chunking.Parser has the following memory consumption:
Class Name | Shallow Heap |
Retained Heap
-----------------------------------------------------------------------------------------
cParser opennlp.tools.parser.chunking.Parser @ 0x79604bb08| 104 |
*27,41,83,328*
-----------------------------------------------------------------------------------------
Within this object, following are the objects that have the majority of the
memory accumulation:
Class Name | Shallow Heap | Retained
Heap
-------------------------------------------------------------------------------
buildModel opennlp.maxent.GISModel @ 0x79249e968| 32 | *
17,02,70,504*
-------------------------------------------------------------------------------
Class Name | Shallow Heap |
Retained Heap
------------------------------------------------------------------------------------
chunker opennlp.tools.chunker.ChunkerME @ 0x79604c850| 24 | *
4,62,08,192*
------------------------------------------------------------------------------------
Class Name | Shallow Heap | Retained
Heap
-------------------------------------------------------------------------------
checkModel opennlp.maxent.GISModel @ 0x788ee2810| 32 | *
3,69,60,416*
-------------------------------------------------------------------------------
Class Name | Shallow Heap |
Retained Heap
------------------------------------------------------------------------------------
tagger opennlp.tools.postag.POSTaggerME @ 0x79604c370| 48 | *
2,05,61,544*
------------------------------------------------------------------------------------
On 25 October 2013 11:56, Samarth Seksaria
<sa...@germinait.com>wrote:
> Hi,
>
> I am using opennlp for generating parse trees. After quite an amount of
> analysis using MAT ( memory analyzer tool ) on eclipse, I suspect the
> accumulation of objects is in an instance of the chunking.Parser class.
>
> I may be completely wrong, and the issue may lie somewhere in my
> application code. I am still trying to find other possibilities. Still,
> would like to get an heads up from someone who is aware of, if any, memory
> leak issues in opennlp.
>
> --
> Thanks & Regards,
> Samarth Seksaria
>
--
Thanks & Regards,
Samarth Seksaria