You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Vaillancourt, Tim" <TV...@ea.com> on 2013/03/13 02:08:12 UTC
RE: Poll: Largest SolrCloud out there?
Considering the silence, I'll take the unofficial largest SolrCloud award until beaten :D:
2 VMWare VMs
4GB RAM/VM
4 Virtual CPUs
< 1000mb index
Beat that :)!!
Tim
-----Original Message-----
From: Otis Gospodnetic [mailto:otis.gospodnetic@gmail.com]
Sent: Thursday, February 28, 2013 12:00 AM
To: solr-user@lucene.apache.org
Subject: Re: Poll: Largest SolrCloud out there?
I'd love to know, too.
What we observed at Sematext was that 4.0 SolrCloud very very buggy and difficult, so I suspect there aren't many big Solr 4.0 based clusters out there. 4.1 is much better (thanks Mark & Co.) and I'm looking forward to
4.2 in March.
Also, based on the stats we have access to via SPM ( see http://sematext.com/spm/index.html ) I can tell you that ElasticSearch clusters are, on average, quite a bit bigger than Solr clusters in terms of nodes, which I find interesting, but not surprising -- if you look at http://blog.sematext.com/2013/02/25/poll-solr-cloud-or-not/ you'll see less than 40% of Solr users are SolrCloud users, which kind of explains it.
Otis
--
Solr & ElasticSearch Support
http://sematext.com/
On Tue, Feb 26, 2013 at 9:41 PM, Vaillancourt, Tim <TV...@ea.com>wrote:
> Hey guys,
>
> I wanted to see who's running SolrCloud out there, and at what scales?
>
> I'd start the thread off but I am merely at the R&D phases.
>
> Cheers!
>
> Tim
>
Re: Poll: Largest SolrCloud out there?
Posted by Otis Gospodnetic <ot...@gmail.com>.
Christian,
SSDs will warm up muuuch faster.
Your other questionable require more info / discussion.
Otis
Solr & ElasticSearch Support
http://sematext.com/
On Mar 14, 2013 8:47 AM, "Christian von Wendt-Jensen" <
Christian.vonWendt-Jensen@infopaq.com> wrote:
> Does it only count if you are using SolrCloud? We are using a traditional
> Master/Slave setup with Solr 4.1:
>
> 1 Master per 14 days:
> Documents: ~15mio
> Index size: ~150GB (stored fields)
>
>
> #of masters: +30
> Performance: SUCKS big time until caches catches up. Unfortunately that
> takes quite some time.
>
> Issues:
> #1: Storage: To use SAN or not.
> #2: Cores per instance: what is ideal?
> #3: Size of cores: is 14 days optimal?
> #4: Performance when searching across shards.
> #5: Would SolrCloud be the solution for us?
>
>
>
>
>
> Med venlig hilsen / Best Regards
>
> Christian von Wendt-Jensen
> IT Team Lead, Customer Solutions
>
> Infopaq International A/S
> Kgs. Nytorv 22
> DK-1050 København K
>
> Phone +45 36 99 00 00
> Mobile +45 31 17 10 07
> Email christian.sonne.jensen@infopaq.com<mailto:
> christian.sonne.jensen@infopaq.com>
> Web www.infopaq.com<http://www.infopaq.com/>
>
>
>
>
>
>
>
>
> DISCLAIMER:
> This e-mail and accompanying documents contain privileged confidential
> information. The information is intended only for the recipient(s) named.
> Any unauthorised disclosure, copying, distribution, exploitation or the
> taking of any action in reliance of the content of this e-mail is strictly
> prohibited. If you have received this e-mail in error we would be obliged
> if you would delete the e-mail and attachments and notify the dispatcher by
> return e-mail or at +45 36 99 00 00
> P Please consider the environment before printing this mail note.
>
> From: Annette Newton <annette.newton@servicetick.com<mailto:
> annette.newton@servicetick.com>>
> Reply-To: "solr-user@lucene.apache.org<ma...@lucene.apache.org>"
> <so...@lucene.apache.org>>
> Date: Wed, 13 Mar 2013 15:49:34 +0100
> To: "solr-user@lucene.apache.org<ma...@lucene.apache.org>" <
> solr-user@lucene.apache.org<ma...@lucene.apache.org>>
> Subject: Re: Poll: Largest SolrCloud out there?
>
> 8 AWS hosts.
> 35GB memory per host
> 10Gb allocated to JVM
> 13 aws compute units per instance
> 4 Shards, 2 replicas
> 25M docs in total
> 22.4GB index per shard
> High writes, low reads
>
>
>
>
> On 13 March 2013 09:12, adm1n <evgeni.evgeni@gmail.com<mailto:
> evgeni.evgeni@gmail.com>> wrote:
>
> 4 AWS hosts:
> Memory: 30822868k total
> CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz x8
> 17M docs
> 5 Gb index.
> 8 master-slave shards (2 shards /host).
> 57 msec/query avg. time. (~110K queries/24 hours).
>
>
>
>
>
> --
> View this message in context:
>
> http://lucene.472066.n3.nabble.com/Poll-Largest-SolrCloud-out-there-tp4043293p4046915.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>
>
> --
>
> Annette Newton
>
> Database Administrator
>
> ServiceTick Ltd
>
>
>
> T:+44(0)1603 618326
>
>
>
> Seebohm House, 2-4 Queen Street, Norwich, England NR2 4SQ
>
> www.servicetick.com
>
> *www.sessioncam.com*
>
> --
> *This message is confidential and is intended to be read solely by the
> addressee. The contents should not be disclosed to any other person or
> copies taken unless authorised to do so. If you are not the intended
> recipient, please notify the sender and permanently delete this message. As
> Internet communications are not secure ServiceTick accepts neither legal
> responsibility for the contents of this message nor responsibility for any
> change made to this message after it was forwarded by the original author.*
>
>
Re: Poll: Largest SolrCloud out there?
Posted by Christian von Wendt-Jensen <Ch...@infopaq.com>.
Does it only count if you are using SolrCloud? We are using a traditional Master/Slave setup with Solr 4.1:
1 Master per 14 days:
Documents: ~15mio
Index size: ~150GB (stored fields)
#of masters: +30
Performance: SUCKS big time until caches catches up. Unfortunately that takes quite some time.
Issues:
#1: Storage: To use SAN or not.
#2: Cores per instance: what is ideal?
#3: Size of cores: is 14 days optimal?
#4: Performance when searching across shards.
#5: Would SolrCloud be the solution for us?
Med venlig hilsen / Best Regards
Christian von Wendt-Jensen
IT Team Lead, Customer Solutions
Infopaq International A/S
Kgs. Nytorv 22
DK-1050 København K
Phone +45 36 99 00 00
Mobile +45 31 17 10 07
Email christian.sonne.jensen@infopaq.com<ma...@infopaq.com>
Web www.infopaq.com<http://www.infopaq.com/>
DISCLAIMER:
This e-mail and accompanying documents contain privileged confidential information. The information is intended only for the recipient(s) named. Any unauthorised disclosure, copying, distribution, exploitation or the taking of any action in reliance of the content of this e-mail is strictly prohibited. If you have received this e-mail in error we would be obliged if you would delete the e-mail and attachments and notify the dispatcher by return e-mail or at +45 36 99 00 00
P Please consider the environment before printing this mail note.
From: Annette Newton <an...@servicetick.com>>
Reply-To: "solr-user@lucene.apache.org<ma...@lucene.apache.org>" <so...@lucene.apache.org>>
Date: Wed, 13 Mar 2013 15:49:34 +0100
To: "solr-user@lucene.apache.org<ma...@lucene.apache.org>" <so...@lucene.apache.org>>
Subject: Re: Poll: Largest SolrCloud out there?
8 AWS hosts.
35GB memory per host
10Gb allocated to JVM
13 aws compute units per instance
4 Shards, 2 replicas
25M docs in total
22.4GB index per shard
High writes, low reads
On 13 March 2013 09:12, adm1n <ev...@gmail.com>> wrote:
4 AWS hosts:
Memory: 30822868k total
CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz x8
17M docs
5 Gb index.
8 master-slave shards (2 shards /host).
57 msec/query avg. time. (~110K queries/24 hours).
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poll-Largest-SolrCloud-out-there-tp4043293p4046915.html
Sent from the Solr - User mailing list archive at Nabble.com.
--
Annette Newton
Database Administrator
ServiceTick Ltd
T:+44(0)1603 618326
Seebohm House, 2-4 Queen Street, Norwich, England NR2 4SQ
www.servicetick.com
*www.sessioncam.com*
--
*This message is confidential and is intended to be read solely by the
addressee. The contents should not be disclosed to any other person or
copies taken unless authorised to do so. If you are not the intended
recipient, please notify the sender and permanently delete this message. As
Internet communications are not secure ServiceTick accepts neither legal
responsibility for the contents of this message nor responsibility for any
change made to this message after it was forwarded by the original author.*
Re: Poll: Largest SolrCloud out there?
Posted by Annette Newton <an...@servicetick.com>.
8 AWS hosts.
35GB memory per host
10Gb allocated to JVM
13 aws compute units per instance
4 Shards, 2 replicas
25M docs in total
22.4GB index per shard
High writes, low reads
On 13 March 2013 09:12, adm1n <ev...@gmail.com> wrote:
> 4 AWS hosts:
> Memory: 30822868k total
> CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz x8
> 17M docs
> 5 Gb index.
> 8 master-slave shards (2 shards /host).
> 57 msec/query avg. time. (~110K queries/24 hours).
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Poll-Largest-SolrCloud-out-there-tp4043293p4046915.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Annette Newton
Database Administrator
ServiceTick Ltd
T:+44(0)1603 618326
Seebohm House, 2-4 Queen Street, Norwich, England NR2 4SQ
www.servicetick.com
*www.sessioncam.com*
--
*This message is confidential and is intended to be read solely by the
addressee. The contents should not be disclosed to any other person or
copies taken unless authorised to do so. If you are not the intended
recipient, please notify the sender and permanently delete this message. As
Internet communications are not secure ServiceTick accepts neither legal
responsibility for the contents of this message nor responsibility for any
change made to this message after it was forwarded by the original author.*
RE: Poll: Largest SolrCloud out there?
Posted by adm1n <ev...@gmail.com>.
4 AWS hosts:
Memory: 30822868k total
CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz x8
17M docs
5 Gb index.
8 master-slave shards (2 shards /host).
57 msec/query avg. time. (~110K queries/24 hours).
--
View this message in context: http://lucene.472066.n3.nabble.com/Poll-Largest-SolrCloud-out-there-tp4043293p4046915.html
Sent from the Solr - User mailing list archive at Nabble.com.