You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Angel Tchorbadjiiski <an...@antibodies-online.com> on 2014/03/04 10:23:09 UTC
SOLR OutOfMemoryError Java heap space
Hello list,
in the last couple of weeks one of my machines is experiencing
OutOfMemoryError: Java heap space errors. In a couple of hours after
starting the SOLR instance queries with execution times of unter 100ms
need more than 10s to execute and many Java heap space erros appear in
the logs. I would be grateful for any hints, how to localize/fix the
problem.
The system is a single shard SOLR 4.6.1 instance with two cores in the
default jetty container. The CORE1 has ~45M documents (disk size of
32GB) with 40 fields (all stored and indexed). CORE2 has ~20M documents
(disk size of 18GB) with 60 fields (both stored and indexed also). All
the fields are relatively short with a maximal length of 100 characters.
In both cores 20 of the fields are used for faceting, have a cache
populating query on "newSearcher" event and the following cache
configurations:
CORE1
<filterCache class="solr.FastLRUCache" size="128" initialSize="0"
autowarmCount="0"/>
<queryResultCache class="solr.LRUCache" size="128" initialSize="0"
autowarmCount="0"/>
<fieldValueCache class="solr.FastLRUCache" size="128" autowarmCount="0"
showItems="0" />
CORE2
<filterCache class="solr.FastLRUCache" size="32" initialSize="0"
autowarmCount="0"/>
<fieldValueCache class="solr.FastLRUCache" size="32" autowarmCount="0"
showItems="0" />
The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM.
The JVM gets the following parameters to start the jetty container:
-XX:+UseCompressedOops
-XX:+UseCompressedStrings
-XX:+OptimizeStringConcat
-XX:+UseStringCache
-XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSInitiatingOccupancyFraction=75
-XX:MaxTenuringThreshold=1
-XX:SurvivorRatio=8
-XX:+CMSParallelRemarkEnabled
-XX:+UseConcMarkSweepGC
-XX:+UseParNewGC
Cheers
Angel
P.S.: Here the complete error OutOfMemoryError message:
org.apache.solr.common.SolrException; null:java.lang.RuntimeException:
java.lang.OutOfMemoryError: Java heap space
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:735)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:438)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:201)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.OutOfMemoryError: Java heap space
Re: SOLR OutOfMemoryError Java heap space
Posted by Angel Tchorbadjiiski <an...@antibodies-online.com>.
On 05.03.2014 11:51, Toke Eskildsen wrote:
> On Wed, 2014-03-05 at 09:59 +0100, Angel Tchorbadjiiski wrote:
>> On 04.03.2014 11:20, Toke Eskildsen wrote:
>>> Angel Tchorbadjiiski [angel.tchorbadjiiski@antibodies-online.com] wrote:
>>>
>>> [Single shard / 2 cores Solr 4.6.1, 65M docs / 50GB, 20 facet fields]
>>>
>>>> The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM.
>>>
>>> I did not see your memory allocation anywhere. What is your Xmx?
>> At the moment I dont use it. The instance allocates 12G without the
>> parameter set.
>
> I have little experience with OpenJDK, but a quick search suggests that
> the default Xmx is physical memory/4 which is indeed 12G for your
> machine. I strongly recommend that you set Xmx explicitly instead as the
> value should be tuned to your concrete Solr deploy and not whatever
> amount of RAM your machine happens to have.
Yes, that is right. I've now both -Xms and -Xmx set.
> Shawn suggests facets to be the culprit and I find it a fair suggestion.
As both cores are used primarily for faceting, this sounds very
reasonable:).
> The stack trace you pasted did not state exactly what caused the OOM -
> was is a complete trace? You might find better information in the Solr
> log. If it is facet related, it is probably during uninversion.
The traces were mostly exactly the same as the included one, but I'll
have a closer look at the logs again.
> A gotcha in Solr faceting is that it is quite often the number of
> documents, rather than the number of facets or facet values, that
> requires a lot of memory.
>
> Extremely loose numbers: Field-faceting on 65M documents (wildly
> guessing 5000 unique values and 2 references/doc) is somewhere around
> 65M*log2(65M*2) + 2*65M*log(5000) bits ~= 65M*28 + 130M*13 ~= 400MB.
>
> As Solr does not come with facet structure collapsing, each facet is
> independent of the others, so with the above estimate, 8GB will be used
> for faceting.
Thank you for the example:-).
I had a look at the inverted index in Solr and the problems resulting
form it in respect to memory needed. It seems, that the docValue fields
will be the solution for the problem.
> Before DocValues became the answer, I wrote a bit about it here:
> http://sbdevel.wordpress.com/2013/04/16/you-are-faceting-itwrong/
I'll have a look at the post, thank you very much.
Cheers
Angel
Re: SOLR OutOfMemoryError Java heap space
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Wed, 2014-03-05 at 09:59 +0100, Angel Tchorbadjiiski wrote:
> On 04.03.2014 11:20, Toke Eskildsen wrote:
> > Angel Tchorbadjiiski [angel.tchorbadjiiski@antibodies-online.com] wrote:
> >
> > [Single shard / 2 cores Solr 4.6.1, 65M docs / 50GB, 20 facet fields]
> >
> >> The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM.
> >
> > I did not see your memory allocation anywhere. What is your Xmx?
> At the moment I dont use it. The instance allocates 12G without the
> parameter set.
I have little experience with OpenJDK, but a quick search suggests that
the default Xmx is physical memory/4 which is indeed 12G for your
machine. I strongly recommend that you set Xmx explicitly instead as the
value should be tuned to your concrete Solr deploy and not whatever
amount of RAM your machine happens to have.
Shawn suggests facets to be the culprit and I find it a fair suggestion.
The stack trace you pasted did not state exactly what caused the OOM -
was is a complete trace? You might find better information in the Solr
log. If it is facet related, it is probably during uninversion.
A gotcha in Solr faceting is that it is quite often the number of
documents, rather than the number of facets or facet values, that
requires a lot of memory.
Extremely loose numbers: Field-faceting on 65M documents (wildly
guessing 5000 unique values and 2 references/doc) is somewhere around
65M*log2(65M*2) + 2*65M*log(5000) bits ~= 65M*28 + 130M*13 ~= 400MB.
As Solr does not come with facet structure collapsing, each facet is
independent of the others, so with the above estimate, 8GB will be used
for faceting.
Before DocValues became the answer, I wrote a bit about it here:
http://sbdevel.wordpress.com/2013/04/16/you-are-faceting-itwrong/
- Toke Eskildsen, State and University Library, Denmark
Re: SOLR OutOfMemoryError Java heap space
Posted by Angel Tchorbadjiiski <an...@antibodies-online.com>.
Hi Toke,
thank you for the mail.
On 04.03.2014 11:20, Toke Eskildsen wrote:
> Angel Tchorbadjiiski [angel.tchorbadjiiski@antibodies-online.com] wrote:
>
> [Single shard / 2 cores Solr 4.6.1, 65M docs / 50GB, 20 facet fields]
>
>> The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM.
>
> I did not see your memory allocation anywhere. What is your Xmx?
At the moment I dont use it. The instance allocates 12G without the
parameter set.
>> P.S.: Here the complete error OutOfMemoryError message:
>
>> org.apache.solr.common.SolrException; null:java.lang.RuntimeException:
>> java.lang.OutOfMemoryError: Java heap space
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:735)
> ...
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>> at java.lang.Thread.run(Thread.java:724)
>> Caused by: java.lang.OutOfMemoryError: Java heap space
>
> A quick sanity check: Is your max user processes higher than 1024?
> (run 'ulimit -a' to inspect it)
max user processes (-u) 387083
It is set high enough.
Cheers
Angel
RE: SOLR OutOfMemoryError Java heap space
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
Angel Tchorbadjiiski [angel.tchorbadjiiski@antibodies-online.com] wrote:
[Single shard / 2 cores Solr 4.6.1, 65M docs / 50GB, 20 facet fields]
> The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM.
I did not see your memory allocation anywhere. What is your Xmx?
> P.S.: Here the complete error OutOfMemoryError message:
> org.apache.solr.common.SolrException; null:java.lang.RuntimeException:
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:735)
...
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: java.lang.OutOfMemoryError: Java heap space
A quick sanity check: Is your max user processes higher than 1024?
(run 'ulimit -a' to inspect it)
- Toke Eskildsen, State and University Library, Denmark
Re: SOLR OutOfMemoryError Java heap space
Posted by Divyang Shah <di...@yahoo.com>.
hi,
heap problem is due to memory full.
you should remove unnecessary data and restart server once.
On Thursday, 6 March 2014 10:39 AM, Angel Tchorbadjiiski <an...@antibodies-online.com> wrote:
Hi Shawn,
a big thanks for the long and detailed answer. I am aware of how linux
uses free RAM for caching and the the problems related to jvm and GC. It
is nice to hear how this correlates to Solr. I'll take some time and
think over it. The facet.method=enum and probably a combination of
DocValue-Fields could be the solution needed in this case.
Thanks again to both of you and Toke for the feedback!
Cheers
Angel
On 05.03.2014 17:06, Shawn Heisey wrote:
> On 3/5/2014 4:40 AM, Angel Tchorbadjiiski wrote:
>> Hi Shawn,
>>
>> On 05.03.2014 10:05, Angel Tchorbadjiiski wrote:
>>> Hi Shawn,
>>>
>>>> It may be your facets that are killing you here. As Toke mentioned, you
>>>> have not indicated what your max heap is.20 separate facet fields with
>>>> millions of documents will use a lot of fieldcache memory if you use the
>>>> standard facet.method, fc.
>>>>
>>>> Try adding facet.method=enum to all your facet queries, or you can put
>>>> it in the defaults section of each request handler definition.
>>> Ok, that is easy to try out.
>>>
>> Changing the facet.method does not help really as the performance of the
>> queries is really bad. This lies mostly on the small cache values, but
>> even trying to tune them for the "enum" case didn't help much.
>>
>> The number of documents and unique facet values seems to be too high.
>> Trying to cache them even with a size of 512 results in many misses and
>> Solr tries to repopulate the cache all the time. This makes the
>> performances even worse.
>
> Good performance with Solr requires a fair amount of memory. You have
> two choices when it comes to where that memory gets used - inside Solr
> in the form of caches, or free memory, available to the operating system
> for caching purposes.
>
> Solr caches are really amazing things. Data gathered for one query can
> significantly speed up another query, because part (or all) of that
> query can be simply skipped, the results read right out of the cache.
>
> There are two potential problems with relying exclusively on Solr
> caches, though. One is that they require Java heap memory, which
> requires garbage collection. A large heap causes GC issues, some of
> which can be alleviated by GC tuning. The other problem is that you
> must actually do a query in order to get the data into the cache. WHen
> you do a commit and open a new searcher, that cache data does away, so
> you have to do the query over again.
>
> The primary reason for slow uncached queries is disk access. Reading
> index data off the disk is a glacial process, comparatively speaking.
> This is where OS disk caching becomes a benefit. Most queries, even
> complex ones, become lightning fast if all of the relevant index data is
> already in RAM and no disk access is required. When queries are fast to
> begin with, you can reduce the cache sizes in Solr, reducing the heap
> requirements. With a smaller heap, more memory is available for the OS
> disk cache.
>
> The facet.method=enum parameter shifts the RAM requirement from Solr to
> the OS. It does not really reduce the amount of required system memory.
> Because disk caching is a kernel level feature and does not utilize
> garbage collection, it is far more efficient than Solr ever could be at
> caching *raw* data. Solr's caches are designed for *processed* data.
>
> What this all boils down to is that I suspect you'll simply need more
> memory on the machine. With facets on so many fields, your queries are
> probably touching nearly the entire index, so you'll want to put the
> entire index into RAM.
>
> Therefore, after Solr allocates its heap and any other programs on the
> system allocate their required memory, you must have enough left memory
> over to fit all (or most) of your 50GB index data. Combine this with
> facet.method=enum and everything should be good.
Re: SOLR OutOfMemoryError Java heap space
Posted by Angel Tchorbadjiiski <an...@antibodies-online.com>.
Hi Shawn,
a big thanks for the long and detailed answer. I am aware of how linux
uses free RAM for caching and the the problems related to jvm and GC. It
is nice to hear how this correlates to Solr. I'll take some time and
think over it. The facet.method=enum and probably a combination of
DocValue-Fields could be the solution needed in this case.
Thanks again to both of you and Toke for the feedback!
Cheers
Angel
On 05.03.2014 17:06, Shawn Heisey wrote:
> On 3/5/2014 4:40 AM, Angel Tchorbadjiiski wrote:
>> Hi Shawn,
>>
>> On 05.03.2014 10:05, Angel Tchorbadjiiski wrote:
>>> Hi Shawn,
>>>
>>>> It may be your facets that are killing you here. As Toke mentioned, you
>>>> have not indicated what your max heap is.20 separate facet fields with
>>>> millions of documents will use a lot of fieldcache memory if you use the
>>>> standard facet.method, fc.
>>>>
>>>> Try adding facet.method=enum to all your facet queries, or you can put
>>>> it in the defaults section of each request handler definition.
>>> Ok, that is easy to try out.
>>>
>> Changing the facet.method does not help really as the performance of the
>> queries is really bad. This lies mostly on the small cache values, but
>> even trying to tune them for the "enum" case didn't help much.
>>
>> The number of documents and unique facet values seems to be too high.
>> Trying to cache them even with a size of 512 results in many misses and
>> Solr tries to repopulate the cache all the time. This makes the
>> performances even worse.
>
> Good performance with Solr requires a fair amount of memory. You have
> two choices when it comes to where that memory gets used - inside Solr
> in the form of caches, or free memory, available to the operating system
> for caching purposes.
>
> Solr caches are really amazing things. Data gathered for one query can
> significantly speed up another query, because part (or all) of that
> query can be simply skipped, the results read right out of the cache.
>
> There are two potential problems with relying exclusively on Solr
> caches, though. One is that they require Java heap memory, which
> requires garbage collection. A large heap causes GC issues, some of
> which can be alleviated by GC tuning. The other problem is that you
> must actually do a query in order to get the data into the cache. WHen
> you do a commit and open a new searcher, that cache data does away, so
> you have to do the query over again.
>
> The primary reason for slow uncached queries is disk access. Reading
> index data off the disk is a glacial process, comparatively speaking.
> This is where OS disk caching becomes a benefit. Most queries, even
> complex ones, become lightning fast if all of the relevant index data is
> already in RAM and no disk access is required. When queries are fast to
> begin with, you can reduce the cache sizes in Solr, reducing the heap
> requirements. With a smaller heap, more memory is available for the OS
> disk cache.
>
> The facet.method=enum parameter shifts the RAM requirement from Solr to
> the OS. It does not really reduce the amount of required system memory.
> Because disk caching is a kernel level feature and does not utilize
> garbage collection, it is far more efficient than Solr ever could be at
> caching *raw* data. Solr's caches are designed for *processed* data.
>
> What this all boils down to is that I suspect you'll simply need more
> memory on the machine. With facets on so many fields, your queries are
> probably touching nearly the entire index, so you'll want to put the
> entire index into RAM.
>
> Therefore, after Solr allocates its heap and any other programs on the
> system allocate their required memory, you must have enough left memory
> over to fit all (or most) of your 50GB index data. Combine this with
> facet.method=enum and everything should be good.
Re: SOLR OutOfMemoryError Java heap space
Posted by Shawn Heisey <so...@elyograg.org>.
On 3/5/2014 4:40 AM, Angel Tchorbadjiiski wrote:
> Hi Shawn,
>
> On 05.03.2014 10:05, Angel Tchorbadjiiski wrote:
>> Hi Shawn,
>>
>>> It may be your facets that are killing you here. As Toke mentioned, you
>>> have not indicated what your max heap is.20 separate facet fields with
>>> millions of documents will use a lot of fieldcache memory if you use the
>>> standard facet.method, fc.
>>>
>>> Try adding facet.method=enum to all your facet queries, or you can put
>>> it in the defaults section of each request handler definition.
>> Ok, that is easy to try out.
>>
> Changing the facet.method does not help really as the performance of the
> queries is really bad. This lies mostly on the small cache values, but
> even trying to tune them for the "enum" case didn't help much.
>
> The number of documents and unique facet values seems to be too high.
> Trying to cache them even with a size of 512 results in many misses and
> Solr tries to repopulate the cache all the time. This makes the
> performances even worse.
Good performance with Solr requires a fair amount of memory. You have
two choices when it comes to where that memory gets used - inside Solr
in the form of caches, or free memory, available to the operating system
for caching purposes.
Solr caches are really amazing things. Data gathered for one query can
significantly speed up another query, because part (or all) of that
query can be simply skipped, the results read right out of the cache.
There are two potential problems with relying exclusively on Solr
caches, though. One is that they require Java heap memory, which
requires garbage collection. A large heap causes GC issues, some of
which can be alleviated by GC tuning. The other problem is that you
must actually do a query in order to get the data into the cache. WHen
you do a commit and open a new searcher, that cache data does away, so
you have to do the query over again.
The primary reason for slow uncached queries is disk access. Reading
index data off the disk is a glacial process, comparatively speaking.
This is where OS disk caching becomes a benefit. Most queries, even
complex ones, become lightning fast if all of the relevant index data is
already in RAM and no disk access is required. When queries are fast to
begin with, you can reduce the cache sizes in Solr, reducing the heap
requirements. With a smaller heap, more memory is available for the OS
disk cache.
The facet.method=enum parameter shifts the RAM requirement from Solr to
the OS. It does not really reduce the amount of required system memory.
Because disk caching is a kernel level feature and does not utilize
garbage collection, it is far more efficient than Solr ever could be at
caching *raw* data. Solr's caches are designed for *processed* data.
What this all boils down to is that I suspect you'll simply need more
memory on the machine. With facets on so many fields, your queries are
probably touching nearly the entire index, so you'll want to put the
entire index into RAM.
Therefore, after Solr allocates its heap and any other programs on the
system allocate their required memory, you must have enough left memory
over to fit all (or most) of your 50GB index data. Combine this with
facet.method=enum and everything should be good.
Thanks,
Shawn
Re: SOLR OutOfMemoryError Java heap space
Posted by Angel Tchorbadjiiski <an...@antibodies-online.com>.
Hi Shawn,
On 05.03.2014 10:05, Angel Tchorbadjiiski wrote:
> Hi Shawn,
>
>> It may be your facets that are killing you here. As Toke mentioned, you
>> have not indicated what your max heap is.20 separate facet fields with
>> millions of documents will use a lot of fieldcache memory if you use the
>> standard facet.method, fc.
>>
>> Try adding facet.method=enum to all your facet queries, or you can put
>> it in the defaults section of each request handler definition.
> Ok, that is easy to try out.
>
Changing the facet.method does not help really as the performance of the
queries is really bad. This lies mostly on the small cache values, but
even trying to tune them for the "enum" case didn't help much.
The number of documents and unique facet values seems to be too high.
Trying to cache them even with a size of 512 results in many misses and
Solr tries to repopulate the cache all the time. This makes the
performances even worse.
Cheers
Angel
Re: SOLR OutOfMemoryError Java heap space
Posted by Angel Tchorbadjiiski <an...@antibodies-online.com>.
Hi Shawn,
> It may be your facets that are killing you here. As Toke mentioned, you
> have not indicated what your max heap is.20 separate facet fields with
> millions of documents will use a lot of fieldcache memory if you use the
> standard facet.method, fc.
>
> Try adding facet.method=enum to all your facet queries, or you can put
> it in the defaults section of each request handler definition.
Ok, that is easy to try out.
> Alternatively, you can add docValues to all your facet fields, but that
> will require a full reindex.
This will not be so easy to handle, so I'll try it only if nothing else
helps.
Thanks a lot,
Angel
Re: SOLR OutOfMemoryError Java heap space
Posted by Shawn Heisey <so...@elyograg.org>.
On 3/4/2014 2:23 AM, Angel Tchorbadjiiski wrote:
> in the last couple of weeks one of my machines is experiencing
> OutOfMemoryError: Java heap space errors. In a couple of hours after
> starting the SOLR instance queries with execution times of unter 100ms
> need more than 10s to execute and many Java heap space erros appear in
> the logs. I would be grateful for any hints, how to localize/fix the
> problem.
>
>
> The system is a single shard SOLR 4.6.1 instance with two cores in the
> default jetty container. The CORE1 has ~45M documents (disk size of
> 32GB) with 40 fields (all stored and indexed). CORE2 has ~20M
> documents (disk size of 18GB) with 60 fields (both stored and indexed
> also). All the fields are relatively short with a maximal length of
> 100 characters. In both cores 20 of the fields are used for faceting,
> have a cache populating query on "newSearcher" event and the following
> cache configurations:
It may be your facets that are killing you here. As Toke mentioned, you
have not indicated what your max heap is.20 separate facet fields with
millions of documents will use a lot of fieldcache memory if you use the
standard facet.method, fc.
Try adding facet.method=enum to all your facet queries, or you can put
it in the defaults section of each request handler definition.
Alternatively, you can add docValues to all your facet fields, but that
will require a full reindex.
http://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap
Thanks,
Shawn