Solr Heap usage

[Greenhorn Techie <gr...@gmail.com>]

Hi,

Wondering what are the considerations to be aware to arrive at an optimal
heap size for Solr JVM? Though I did discuss this on the IRC, I am still
unclear on how Solr uses the JVM heap space. Are there any pointers to
understand this aspect better?

Given that Solr requires an optimally configured heap, so that the
remaining unused memory can be used for OS disk cache, I wonder how to best
configure Solr heap. Also, on the IRC it was discussed that having 31GB of
heap is better than having 32GB due to Java’s internal usage of heap. Can
anyone guide further on heap configuration please?

Thanks

Re: Solr Heap usage

[Susheel Kumar <su...@gmail.com>]

Take a look at https://wiki.apache.org/solr/SolrPerformanceProblems. The
section "how much heap do i need"  talks about that.
Cache also goes to JVM so take a look how much you need/allocating for
different cache's.

Thnx


On Tue, May 1, 2018 at 7:33 PM, Greenhorn Techie <gr...@gmail.com>
wrote:

> Hi,
>
> Wondering what are the considerations to be aware to arrive at an optimal
> heap size for Solr JVM? Though I did discuss this on the IRC, I am still
> unclear on how Solr uses the JVM heap space. Are there any pointers to
> understand this aspect better?
>
> Given that Solr requires an optimally configured heap, so that the
> remaining unused memory can be used for OS disk cache, I wonder how to best
> configure Solr heap. Also, on the IRC it was discussed that having 31GB of
> heap is better than having 32GB due to Java’s internal usage of heap. Can
> anyone guide further on heap configuration please?
>
> Thanks
>

Re: Solr Heap usage

[Greenhorn Techie <gr...@gmail.com>]

Thanks Shawn for the inputs, which will definitely help us to scale our
cluster better.

Regards


On 2 May 2018 at 18:15:12, Shawn Heisey (apache@elyograg.org) wrote:

On 5/1/2018 5:33 PM, Greenhorn Techie wrote:
> Wondering what are the considerations to be aware to arrive at an optimal
> heap size for Solr JVM? Though I did discuss this on the IRC, I am still
> unclear on how Solr uses the JVM heap space. Are there any pointers to
> understand this aspect better?

I'm one of the people you've been chatting with on IRC.

I also wrote the wiki page that Susheel has recommended to you.

> Given that Solr requires an optimally configured heap, so that the
> remaining unused memory can be used for OS disk cache, I wonder how to
best
> configure Solr heap. Also, on the IRC it was discussed that having 31GB
of
> heap is better than having 32GB due to Java’s internal usage of heap. Can
> anyone guide further on heap configuration please?

With the index size you mentioned on IRC, it's very difficult to project
how much heap you're going to need. Actually setting up a system,
putting data on it, and firing real queries at it may be the only way to
be sure.

The only concrete advice I can give you with the information available
is this: Install as much memory as you can. It is extremely unlikely
that you would ever have too much memory when you're dealing with
terabyte-scale indexes.

Heavy indexing (which you have mentioned as a requirement in another
thread) will tend to require a larger heap.

Thanks,
Shawn

Re: Solr Heap usage

[Shawn Heisey <ap...@elyograg.org>]

On 5/1/2018 5:33 PM, Greenhorn Techie wrote:
> Wondering what are the considerations to be aware to arrive at an optimal
> heap size for Solr JVM? Though I did discuss this on the IRC, I am still
> unclear on how Solr uses the JVM heap space. Are there any pointers to
> understand this aspect better?

I'm one of the people you've been chatting with on IRC.

I also wrote the wiki page that Susheel has recommended to you.

> Given that Solr requires an optimally configured heap, so that the
> remaining unused memory can be used for OS disk cache, I wonder how to best
> configure Solr heap. Also, on the IRC it was discussed that having 31GB of
> heap is better than having 32GB due to Java’s internal usage of heap. Can
> anyone guide further on heap configuration please?

With the index size you mentioned on IRC, it's very difficult to project
how much heap you're going to need.  Actually setting up a system,
putting data on it, and firing real queries at it may be the only way to
be sure.

The only concrete advice I can give you with the information available
is this:  Install as much memory as you can.  It is extremely unlikely
that you would ever have too much memory when you're dealing with
terabyte-scale indexes.

Heavy indexing (which you have mentioned as a requirement in another
thread) will tend to require a larger heap.

Thanks,
Shawn