You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by dw5ight <dw...@gmail.com> on 2012/03/23 21:57:42 UTC
Practical Optimization
Hey All-
we run a http://carsabi.com car search engine with Solr and did some
benchmarking recently after we switched from a hosted service to
self-hosting. In brief, we went from 800ms complex range queries on a 1.5M
document corpus to 43ms. The major shifts were switching from EC2 Large to
EC2 CC8XL which got us down to 282ms (2.82x speed gain due to 2.75x CPU
speed increase we think), and then down to 43ms when we sharded to 8 cores.
We tried sharding to 12 and 16 but saw negligible gains after this point.
Anyway, hope this might be useful to someone - we write up exact stats and a
step by step sharding procedure on our
http://carsabi.com/car-news/2012/03/23/optimizing-solr-7x-your-search-speed/
tech blog if anyone's interested.
best
Dwight
--
View this message in context: http://lucene.472066.n3.nabble.com/Practical-Optimization-tp3852776p3852776.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Practical Optimization
Posted by Erick Erickson <er...@gmail.com>.
Thanks, this is very helpful!
On Sat, Mar 24, 2012 at 4:50 AM, Martin Koch <ma...@issuu.com> wrote:
> Thanks for writing this up. These are good tips.
>
> /Martin
>
> On Fri, Mar 23, 2012 at 9:57 PM, dw5ight <dw...@gmail.com> wrote:
>
>> Hey All-
>>
>> we run a http://carsabi.com car search engine with Solr and did some
>> benchmarking recently after we switched from a hosted service to
>> self-hosting. In brief, we went from 800ms complex range queries on a 1.5M
>> document corpus to 43ms. The major shifts were switching from EC2 Large to
>> EC2 CC8XL which got us down to 282ms (2.82x speed gain due to 2.75x CPU
>> speed increase we think), and then down to 43ms when we sharded to 8 cores.
>> We tried sharding to 12 and 16 but saw negligible gains after this point.
>>
>> Anyway, hope this might be useful to someone - we write up exact stats and
>> a
>> step by step sharding procedure on our
>>
>> http://carsabi.com/car-news/2012/03/23/optimizing-solr-7x-your-search-speed/
>> tech blog if anyone's interested.
>>
>> best
>> Dwight
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Practical-Optimization-tp3852776p3852776.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
Re: Practical Optimization
Posted by Martin Koch <ma...@issuu.com>.
Thanks for writing this up. These are good tips.
/Martin
On Fri, Mar 23, 2012 at 9:57 PM, dw5ight <dw...@gmail.com> wrote:
> Hey All-
>
> we run a http://carsabi.com car search engine with Solr and did some
> benchmarking recently after we switched from a hosted service to
> self-hosting. In brief, we went from 800ms complex range queries on a 1.5M
> document corpus to 43ms. The major shifts were switching from EC2 Large to
> EC2 CC8XL which got us down to 282ms (2.82x speed gain due to 2.75x CPU
> speed increase we think), and then down to 43ms when we sharded to 8 cores.
> We tried sharding to 12 and 16 but saw negligible gains after this point.
>
> Anyway, hope this might be useful to someone - we write up exact stats and
> a
> step by step sharding procedure on our
>
> http://carsabi.com/car-news/2012/03/23/optimizing-solr-7x-your-search-speed/
> tech blog if anyone's interested.
>
> best
> Dwight
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Practical-Optimization-tp3852776p3852776.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Practical Optimization
Posted by William Bell <bi...@gmail.com>.
What type of logging were you using?
Did you try log back? We get a pretty large increase when using that.
On Fri, Mar 23, 2012 at 2:57 PM, dw5ight <dw...@gmail.com> wrote:
> Hey All-
>
> we run a http://carsabi.com car search engine with Solr and did some
> benchmarking recently after we switched from a hosted service to
> self-hosting. In brief, we went from 800ms complex range queries on a 1.5M
> document corpus to 43ms. The major shifts were switching from EC2 Large to
> EC2 CC8XL which got us down to 282ms (2.82x speed gain due to 2.75x CPU
> speed increase we think), and then down to 43ms when we sharded to 8 cores.
> We tried sharding to 12 and 16 but saw negligible gains after this point.
>
> Anyway, hope this might be useful to someone - we write up exact stats and a
> step by step sharding procedure on our
> http://carsabi.com/car-news/2012/03/23/optimizing-solr-7x-your-search-speed/
> tech blog if anyone's interested.
>
> best
> Dwight
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Practical-Optimization-tp3852776p3852776.html
> Sent from the Solr - User mailing list archive at Nabble.com.
--
Bill Bell
billnbell@gmail.com
cell 720-256-8076