You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Wayne W <wa...@gmail.com> on 2012/01/14 08:53:45 UTC
Solr - Tika(?) memory leak
Hi,
we're using Solr running on tomcat with 1GB in production, and of late
we've been having a huge number of OutOfMemory issues. It seems from
what I can tell this is coming from the tika extraction of the
content. I've processed the java dump file using a memory analyzer and
its pretty clean at least the class involved. It seems like a leak to
me, as we don't parse any files larger than 20M, and these objects are
taking up ~700M
I've attached 2 screen shots from the tool (not sure if you receive
attachments).
But to summarize (class, number of objects, Used heap size, Retained Heap Size):
org.apache.xmlbeans.impl.store.Xob$ElementXObj 838,993
80,533,728 604,606,040
org.apache.poi.openxml4j.opc.ZipPackage 2
112 87,009,848
char[]
587 32,216,960 38,216,950
We're really desperate to find a solution to this - any ideas or help
is greatly appreciated.
Wayne
Re: Solr - Tomcat new versions
Posted by Luca Cavanna <ca...@gmail.com>.
Hi Alessio,
in order to help you, we'd need to know something more about what's going
wrong. Could you give us a stacktrace or an error you're reading?
How do you know solr isn't working?
Thanks
Luca
On Tue, Jan 17, 2012 at 10:52 AM, Alessio Crisantemi <
alessio.crisantemi@gioconews.it> wrote:
> Dear Luca,
> I follow the Solr installation procedures signed on Official guide, but
> with Solr 3,5 don't works. While with solr 1.4.1 it's all right.
>
> I don't know why...but now I work with Solr 1.4.1
>
> and more:
> I would install TIKA 1.0 on Solr 1.4.1. Is possible?
> How can i do? can you help me?
> best,
> a.
>
>
> ------------------------------**------------------------------**
> ------------------------------**-----
>
>
>
>
>
>
>
>
> -----Messaggio originale----- From: Luca Cavanna
> Sent: Tuesday, January 17, 2012 10:16 AM
> To: solr-user@lucene.apache.org ; Alessio Crisantemi
> Subject: Re: Solr - Tomcat new versions
>
> Hi Alessio,
> I've seen Solr 3.5 running within Tomcat 7.0.23, it shouldn't be a bug I
> guess.
> Could you please provide some more details about the problem you have? Do
> you have a stacktrace?
> Are you upgrading an existing Solr 1.4.1, right?
> By the way, which jdk are you using?
>
> Thanks
> Luca
>
> On Tue, Jan 17, 2012 at 9:40 AM, Alessio Crisantemi <
> alessio.crisantemi@gioconews.**it <al...@gioconews.it>>
> wrote:
>
> Hi,
>> I installed Apache tomct on Windows (Vista) and Solr.
>> But I have any problem between Tomcat 7.0.23 and Solr 3.5
>>
>> No problem if I install Solr 1.4.1 with the same version of Tomcat.
>> (I check it with binary and source code installation for omcat but the
>> result is the same).
>> It's a bug, I think?
>> thank you
>> Alessio
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
Re: Solr - Tomcat new versions
Posted by Erik Hatcher <er...@gmail.com>.
Perhaps this the known issue with the 3.5 example schema being used in Tomcat and the VelocityResponseWriter issue? I'm on my mobile now so don't have easy access to a pointer with details but check the archives if this seems to be the issue on how to resolve it.
Erik
On Jan 17, 2012, at 4:52, "Alessio Crisantemi" <al...@gioconews.it> wrote:
> Dear Luca,
> I follow the Solr installation procedures signed on Official guide, but with Solr 3,5 don't works. While with solr 1.4.1 it's all right.
>
> I don't know why...but now I work with Solr 1.4.1
>
> and more:
> I would install TIKA 1.0 on Solr 1.4.1. Is possible?
> How can i do? can you help me?
> best,
> a.
>
>
> -----------------------------------------------------------------------------------------------
>
>
>
>
>
>
>
> -----Messaggio originale----- From: Luca Cavanna
> Sent: Tuesday, January 17, 2012 10:16 AM
> To: solr-user@lucene.apache.org ; Alessio Crisantemi
> Subject: Re: Solr - Tomcat new versions
>
> Hi Alessio,
> I've seen Solr 3.5 running within Tomcat 7.0.23, it shouldn't be a bug I
> guess.
> Could you please provide some more details about the problem you have? Do
> you have a stacktrace?
> Are you upgrading an existing Solr 1.4.1, right?
> By the way, which jdk are you using?
>
> Thanks
> Luca
>
> On Tue, Jan 17, 2012 at 9:40 AM, Alessio Crisantemi <
> alessio.crisantemi@gioconews.it> wrote:
>
>> Hi,
>> I installed Apache tomct on Windows (Vista) and Solr.
>> But I have any problem between Tomcat 7.0.23 and Solr 3.5
>>
>> No problem if I install Solr 1.4.1 with the same version of Tomcat.
>> (I check it with binary and source code installation for omcat but the
>> result is the same).
>> It's a bug, I think?
>> thank you
>> Alessio
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> --
> Luca Cavanna
> E-mail: cavannaluca@gmail.com
> Skype: just.cavanna
> Italian Mobile: +39 329 0170084
> Dutch Mobile: +31 6 22255262
> Website: http://www.javanna.net
> Stack Overflow Careers: http://careers.stackoverflow.com/lucacavanna
> Linkedin: http://it.linkedin.com/in/lucacavanna
Re: Solr - Tomcat new versions
Posted by Alessio Crisantemi <al...@gioconews.it>.
Dear Luca,
I follow the Solr installation procedures signed on Official guide, but with
Solr 3,5 don't works. While with solr 1.4.1 it's all right.
I don't know why...but now I work with Solr 1.4.1
and more:
I would install TIKA 1.0 on Solr 1.4.1. Is possible?
How can i do? can you help me?
best,
a.
-----------------------------------------------------------------------------------------------
-----Messaggio originale-----
From: Luca Cavanna
Sent: Tuesday, January 17, 2012 10:16 AM
To: solr-user@lucene.apache.org ; Alessio Crisantemi
Subject: Re: Solr - Tomcat new versions
Hi Alessio,
I've seen Solr 3.5 running within Tomcat 7.0.23, it shouldn't be a bug I
guess.
Could you please provide some more details about the problem you have? Do
you have a stacktrace?
Are you upgrading an existing Solr 1.4.1, right?
By the way, which jdk are you using?
Thanks
Luca
On Tue, Jan 17, 2012 at 9:40 AM, Alessio Crisantemi <
alessio.crisantemi@gioconews.it> wrote:
> Hi,
> I installed Apache tomct on Windows (Vista) and Solr.
> But I have any problem between Tomcat 7.0.23 and Solr 3.5
>
> No problem if I install Solr 1.4.1 with the same version of Tomcat.
> (I check it with binary and source code installation for omcat but the
> result is the same).
> It's a bug, I think?
> thank you
> Alessio
>
>
>
>
>
>
>
>
>
--
Luca Cavanna
E-mail: cavannaluca@gmail.com
Skype: just.cavanna
Italian Mobile: +39 329 0170084
Dutch Mobile: +31 6 22255262
Website: http://www.javanna.net
Stack Overflow Careers: http://careers.stackoverflow.com/lucacavanna
Linkedin: http://it.linkedin.com/in/lucacavanna
Re: Solr - Tomcat new versions
Posted by Luca Cavanna <ca...@gmail.com>.
Hi Alessio,
I've seen Solr 3.5 running within Tomcat 7.0.23, it shouldn't be a bug I
guess.
Could you please provide some more details about the problem you have? Do
you have a stacktrace?
Are you upgrading an existing Solr 1.4.1, right?
By the way, which jdk are you using?
Thanks
Luca
On Tue, Jan 17, 2012 at 9:40 AM, Alessio Crisantemi <
alessio.crisantemi@gioconews.it> wrote:
> Hi,
> I installed Apache tomct on Windows (Vista) and Solr.
> But I have any problem between Tomcat 7.0.23 and Solr 3.5
>
> No problem if I install Solr 1.4.1 with the same version of Tomcat.
> (I check it with binary and source code installation for omcat but the
> result is the same).
> It's a bug, I think?
> thank you
> Alessio
>
>
>
>
>
>
>
>
>
--
Luca Cavanna
E-mail: cavannaluca@gmail.com
Skype: just.cavanna
Italian Mobile: +39 329 0170084
Dutch Mobile: +31 6 22255262
Website: http://www.javanna.net
Stack Overflow Careers: http://careers.stackoverflow.com/lucacavanna
Linkedin: http://it.linkedin.com/in/lucacavanna
Re: Solr - Tomcat new versions
Posted by Alessio Crisantemi <al...@gioconews.it>.
Hi,
I installed Apache tomct on Windows (Vista) and Solr.
But I have any problem between Tomcat 7.0.23 and Solr 3.5
No problem if I install Solr 1.4.1 with the same version of Tomcat.
(I check it with binary and source code installation for omcat but the
result is the same).
It's a bug, I think?
thank you
Alessio
Re: Solr - Tika(?) memory leak
Posted by Otis Gospodnetic <ot...@yahoo.com>.
You'll need to reindex everything indeed.
Otis
----
Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html
>________________________________
> From: Wayne W <wa...@gmail.com>
>To: solr-user@lucene.apache.org
>Sent: Tuesday, January 17, 2012 12:36 AM
>Subject: Re: Solr - Tika(?) memory leak
>
>Thanks for the links - I've put a posting on the Tika ML.
>I've just checked and we using tika-0.2.jar - does anyone know which
>version I can use with solr 1.3?
>
>Is there any info on upgrading from this far back to the latest
>version - is it even possible? or would I need to re-index everything?
>
>On Tue, Jan 17, 2012 at 5:39 AM, P Williams
><wi...@gmail.com> wrote:
>> Hi,
>>
>> I'm not sure which version of Solr/Tika you're using but I had a similar
>> experience which turned out to be the result of a design change to PDFBox.
>>
>> https://issues.apache.org/jira/browse/SOLR-2886
>>
>> Tricia
>>
>> On Sat, Jan 14, 2012 at 12:53 AM, Wayne W <wa...@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> we're using Solr running on tomcat with 1GB in production, and of late
>>> we've been having a huge number of OutOfMemory issues. It seems from
>>> what I can tell this is coming from the tika extraction of the
>>> content. I've processed the java dump file using a memory analyzer and
>>> its pretty clean at least the class involved. It seems like a leak to
>>> me, as we don't parse any files larger than 20M, and these objects are
>>> taking up ~700M
>>>
>>> I've attached 2 screen shots from the tool (not sure if you receive
>>> attachments).
>>>
>>> But to summarize (class, number of objects, Used heap size, Retained Heap
>>> Size):
>>>
>>>
>>> org.apache.xmlbeans.impl.store.Xob$ElementXObj 838,993
>>> 80,533,728 604,606,040
>>> org.apache.poi.openxml4j.opc.ZipPackage 2
>>> 112 87,009,848
>>> char[]
>>> 587 32,216,960 38,216,950
>>>
>>>
>>> We're really desperate to find a solution to this - any ideas or help
>>> is greatly appreciated.
>>> Wayne
>>>
>
>
>
Re: Solr - Tika(?) memory leak
Posted by Wayne W <wa...@gmail.com>.
Thanks for the links - I've put a posting on the Tika ML.
I've just checked and we using tika-0.2.jar - does anyone know which
version I can use with solr 1.3?
Is there any info on upgrading from this far back to the latest
version - is it even possible? or would I need to re-index everything?
On Tue, Jan 17, 2012 at 5:39 AM, P Williams
<wi...@gmail.com> wrote:
> Hi,
>
> I'm not sure which version of Solr/Tika you're using but I had a similar
> experience which turned out to be the result of a design change to PDFBox.
>
> https://issues.apache.org/jira/browse/SOLR-2886
>
> Tricia
>
> On Sat, Jan 14, 2012 at 12:53 AM, Wayne W <wa...@gmail.com>wrote:
>
>> Hi,
>>
>> we're using Solr running on tomcat with 1GB in production, and of late
>> we've been having a huge number of OutOfMemory issues. It seems from
>> what I can tell this is coming from the tika extraction of the
>> content. I've processed the java dump file using a memory analyzer and
>> its pretty clean at least the class involved. It seems like a leak to
>> me, as we don't parse any files larger than 20M, and these objects are
>> taking up ~700M
>>
>> I've attached 2 screen shots from the tool (not sure if you receive
>> attachments).
>>
>> But to summarize (class, number of objects, Used heap size, Retained Heap
>> Size):
>>
>>
>> org.apache.xmlbeans.impl.store.Xob$ElementXObj 838,993
>> 80,533,728 604,606,040
>> org.apache.poi.openxml4j.opc.ZipPackage 2
>> 112 87,009,848
>> char[]
>> 587 32,216,960 38,216,950
>>
>>
>> We're really desperate to find a solution to this - any ideas or help
>> is greatly appreciated.
>> Wayne
>>
Re: Solr - Tika(?) memory leak
Posted by P Williams <wi...@gmail.com>.
Hi,
I'm not sure which version of Solr/Tika you're using but I had a similar
experience which turned out to be the result of a design change to PDFBox.
https://issues.apache.org/jira/browse/SOLR-2886
Tricia
On Sat, Jan 14, 2012 at 12:53 AM, Wayne W <wa...@gmail.com>wrote:
> Hi,
>
> we're using Solr running on tomcat with 1GB in production, and of late
> we've been having a huge number of OutOfMemory issues. It seems from
> what I can tell this is coming from the tika extraction of the
> content. I've processed the java dump file using a memory analyzer and
> its pretty clean at least the class involved. It seems like a leak to
> me, as we don't parse any files larger than 20M, and these objects are
> taking up ~700M
>
> I've attached 2 screen shots from the tool (not sure if you receive
> attachments).
>
> But to summarize (class, number of objects, Used heap size, Retained Heap
> Size):
>
>
> org.apache.xmlbeans.impl.store.Xob$ElementXObj 838,993
> 80,533,728 604,606,040
> org.apache.poi.openxml4j.opc.ZipPackage 2
> 112 87,009,848
> char[]
> 587 32,216,960 38,216,950
>
>
> We're really desperate to find a solution to this - any ideas or help
> is greatly appreciated.
> Wayne
>
Re: Solr - Tika(?) memory leak
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Wayne,
Have you asked on Tika's ML?
You may also want to watch https://issues.apache.org/jira/browse/SOLR-2901
Otis
----
Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html
----- Original Message -----
> From: Wayne W <wa...@gmail.com>
> To: solr-user@lucene.apache.org
> Cc:
> Sent: Saturday, January 14, 2012 2:53 AM
> Subject: Solr - Tika(?) memory leak
>
> Hi,
>
> we're using Solr running on tomcat with 1GB in production, and of late
> we've been having a huge number of OutOfMemory issues. It seems from
> what I can tell this is coming from the tika extraction of the
> content. I've processed the java dump file using a memory analyzer and
> its pretty clean at least the class involved. It seems like a leak to
> me, as we don't parse any files larger than 20M, and these objects are
> taking up ~700M
>
> I've attached 2 screen shots from the tool (not sure if you receive
> attachments).
>
> But to summarize (class, number of objects, Used heap size, Retained Heap Size):
>
>
> org.apache.xmlbeans.impl.store.Xob$ElementXObj 838,993
> 80,533,728 604,606,040
> org.apache.poi.openxml4j.opc.ZipPackage 2
> 112 87,009,848
> char[]
> 587 32,216,960 38,216,950
>
>
> We're really desperate to find a solution to this - any ideas or help
> is greatly appreciated.
> Wayne
>