You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Mugeesh Husain <mu...@gmail.com> on 2016/11/16 19:12:29 UTC
Hardware size in solrcloud
I have lots of document i dont know now how much it would be in future. for
the inilial stage, I am looking for hardware details(assumption).
I are looking forward to setting up a billion document(1 billion approx)
solr index and the size is 7GB.
Can you please suggest the hardware details as per experience.
1. OS(32/64bit):
2. Processor:
3. RAM:
4. No of physical servers/systems :
Thanks
--
View this message in context: http://lucene.472066.n3.nabble.com/Hardware-size-in-solrcloud-tp4306169.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Hardware size in solrcloud
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Thu, 2016-11-17 at 06:28 -0700, Mugeesh Husain wrote:
> > If I have 15 document ,according to above formula
>
> 1 document=7 byte per document
>
> 15 document=15MB
That math does not make sense.
I doubt it is even possible to create a Solr index, where each document
only takes up 7 bytes. If each document taking up 1MB (your 15 docs =
15MB statement) is more realistic. But in that case your full index
would be 10^6MB * 10^9 = 10^15 bytes = 1 petabyte.
- Toke Eskildsen, State and University Library, Denmark
Re: Hardware size in solrcloud
Posted by Shawn Heisey <ap...@elyograg.org>.
On 11/17/2016 6:28 AM, Mugeesh Husain wrote:
>> 7GB/1 billion = 7 bytes per document? That would be basically 7
>> characters?
> If I have 15 document ,according to above formula
>
> 1 document=7 byte per document
>
> 15 document=15MB
>
> if i have 1 billions document then what would be the configuration for this
> kind of data.
>
> 1. OS(32/64bit):
> 2. Processor:
> 3. RAM:
> 4. No of physical servers/systems :
>
> Please suggest.
Did you not read Kevin's response fully, and visit the URL you were
provided? There are so many variables involved that unless you actually
TRY it, you won't know how your install will behave. The nature of your
documents and how you have configured Solr will affect what hardware you
need. The nature of your queries, and how many queries per second you
send, will affect what hardware you need. Here is that URL again:
https://lucidworks.com/blog/2012/07/23/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/
We do not know what hardware you will need, and cannot make calculations
based on information you provide. Even if you provide more information,
we can only make guesses. Those guesses will be very much on the high
end, and may be more than the minimum required.
Memory tends to be the most critical resource for the performance of
individual queries. CPU becomes more important as query load
increases. Here's some information I've put together, which mostly
concerns itself with memory:
https://wiki.apache.org/solr/SolrPerformanceProblems
Thanks,
Shawn
Re: Hardware size in solrcloud
Posted by Mugeesh Husain <mu...@gmail.com>.
>
> First question: is your initial sizing correct?
>
Yes
>
> 7GB/1 billion = 7 bytes per document? That would be basically 7
> characters?
>
>
If I have 15 document ,according to above formula
1 document=7 byte per document
15 document=15MB
if i have 1 billions document then what would be the configuration for this
kind of data.
1. OS(32/64bit):
2. Processor:
3. RAM:
4. No of physical servers/systems :
Please suggest.
*Thanks,*
*Mugeesh Husain *
--
View this message in context: http://lucene.472066.n3.nabble.com/Hardware-size-in-solrcloud-tp4306169p4306282.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Hardware size in solrcloud
Posted by Kevin Risden <co...@gmail.com>.
First question: is your initial sizing correct?
7GB/1 billion = 7 bytes per document? That would be basically 7 characters?
Anyway there are lots of variables regarding sizing. The typical response
is:
https://lucidworks.com/blog/2012/07/23/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/
Kevin Risden
On Wed, Nov 16, 2016 at 1:12 PM, Mugeesh Husain <mu...@gmail.com> wrote:
> I have lots of document i dont know now how much it would be in future. for
> the inilial stage, I am looking for hardware details(assumption).
>
> I are looking forward to setting up a billion document(1 billion approx)
> solr index and the size is 7GB.
>
> Can you please suggest the hardware details as per experience.
> 1. OS(32/64bit):
> 2. Processor:
> 3. RAM:
> 4. No of physical servers/systems :
>
>
> Thanks
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Hardware-size-in-solrcloud-tp4306169.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>