clarification on index-to-ram ratio

[Vinay Pothnis <po...@gmail.com>]

Hello All,

The documentation and general feedback on the mailing list suggest the
following:

*"... Let's say that you have a Solr index size of 8GB. If your OS, Solr's
Java heap, and all other running programs require 4GB of memory, then
an ideal memory size for that server is at least 12GB ..."*

http://wiki.apache.org/solr/SolrPerformanceProblems#General_information

So, when we say "index size" does it include ALL the replicas or just one
of the replica? Say for example, if the solr instance had 2 replicas each
of size 8GB, should we consider 16GB as our index size or just 8GB - for
the above index-ram-ratio consideration?

Thanks
Vinay

Re: clarification on index-to-ram ratio

[Vinay Pothnis <po...@gmail.com>]

Thanks!
And yes, the replica belongs to a different shard - not the same data.

-Vinay


On 19 June 2014 11:21, Toke Eskildsen <te...@statsbiblioteket.dk> wrote:

> Vinay Pothnis [pothnis@gmail.com] wrote:
> > *"... Let's say that you have a Solr index size of 8GB. If your OS,
> Solr's
> > Java heap, and all other running programs require 4GB of memory, then
> > an ideal memory size for that server is at least 12GB ..."*
>
> > So, when we say "index size" does it include ALL the replicas or just one
> > of the replica? Say for example, if the solr instance had 2 replicas each
> > of size 8GB, should we consider 16GB as our index size or just 8GB - for
> > the above index-ram-ratio consideration?
>
> 16GB, according to the above principle. Enough RAM to hold all index data
> on storage.
>
> Two things though,
>
> 1) If you have replicas of the same data on the same machine, I hope that
> you have them on separate physical drives. If not, it is just wasted disk
> cache with no benefits.
>
> 2) The general advice is only really usable when we're either talking
> fairly small indexes on spinning drives or there is a strong need for the
> absolute lowest latency possible. As soon as we scale up and do not have
> copious amounts of money, solid state drives provides much better bang for
> the buck than a spinning drives + RAM combination.
>
> - Toke Eskildsen
>

RE: clarification on index-to-ram ratio

[Toke Eskildsen <te...@statsbiblioteket.dk>]

Vinay Pothnis [pothnis@gmail.com] wrote:
> *"... Let's say that you have a Solr index size of 8GB. If your OS, Solr's
> Java heap, and all other running programs require 4GB of memory, then
> an ideal memory size for that server is at least 12GB ..."*

> So, when we say "index size" does it include ALL the replicas or just one
> of the replica? Say for example, if the solr instance had 2 replicas each
> of size 8GB, should we consider 16GB as our index size or just 8GB - for
> the above index-ram-ratio consideration?

16GB, according to the above principle. Enough RAM to hold all index data on storage.

Two things though,

1) If you have replicas of the same data on the same machine, I hope that you have them on separate physical drives. If not, it is just wasted disk cache with no benefits.

2) The general advice is only really usable when we're either talking fairly small indexes on spinning drives or there is a strong need for the absolute lowest latency possible. As soon as we scale up and do not have copious amounts of money, solid state drives provides much better bang for the buck than a spinning drives + RAM combination.

- Toke Eskildsen