Out of Memory

[Milan Dobrota <mi...@milandobrota.com>]

Hi,

I have a Solr instance with 6M item index. It normally uses around 3G of
memory. I have suddenly started getting out of memory errors and increasing
the Xmx parameter to over 4G didn't fix the problem. It was just buying us
time. Inspecting the heap, I figured that 90% of memory is occupied by
FieldCache. Is this normal? We do very little sorting and no faceting.

Is FieldCache ever supposed to get cleared? Can this be done through HTTP?

Do we need more memory? If so, I don't understand why the minimal set of
changes we introduced (one multivalued field) would cause the memory to
drastically increase.

The communication with the Solr instance is done via HTTP.

Java version:
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode)

Milan

Re: Out of Memory

[Erick Erickson <er...@gmail.com>]

Right. Mutlivalued fields use fieldCache for
faceting (as I remember) whereas single valued
fields don't under some circumstances. See:
http://wiki.apache.org/solr/SolrCaching#The_Lucene_FieldCache

Before your change, you were probably using the
filterCache for what faceting you were doing.

So yes, you're probably memory-constrained at this
point. How much physical memory do you have anyway?

Best
Erick

On Mon, Jan 30, 2012 at 12:10 PM, Milan Dobrota <mi...@milandobrota.com> wrote:
> Hi,
>
> I have a Solr instance with 6M item index. It normally uses around 3G of
> memory. I have suddenly started getting out of memory errors and increasing
> the Xmx parameter to over 4G didn't fix the problem. It was just buying us
> time. Inspecting the heap, I figured that 90% of memory is occupied by
> FieldCache. Is this normal? We do very little sorting and no faceting.
>
> Is FieldCache ever supposed to get cleared? Can this be done through HTTP?
>
> Do we need more memory? If so, I don't understand why the minimal set of
> changes we introduced (one multivalued field) would cause the memory to
> drastically increase.
>
> The communication with the Solr instance is done via HTTP.
>
> Java version:
> java version "1.6.0_17"
> Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
> Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode)
>
> Milan