Solr performance issues

[Sébastien Rainville <se...@gmail.com>]

Hi,

I've been using solr for a little without worrying too much about how it
works but now it's becoming a bottleneck in my application. I have a couple
issues with it:

1. My index always gets slower and slower when commiting/optimizing for some
obscure reason. It goes from 1 second with a new index to 45 seconds with an
index with the same amount of data but used for a few days. Restarting solr
doesn't fix it. The only way I found to fix that is to delete the whole
index completely by deleting the index folder. Then when I rebuild the index
everything goes back to normal and fast... and then performance slowly
deteriorates again. So, the amount of data is not a factor because
rebuilding the index from scratch fixes the problem and I am sending
"optimize" once in a while... even maybe too often.

2. I use acts_as_solr and by default they only make "post" requests, even
for /select. With that setup the response time for most queries, simple or
complex ones, were ranging from 150ms to 600ms, with an average of 250ms. I
changed the select request to use "get" requests instead and now the
response time is down to 10ms to 60ms. Did someone seen that before? Why is
it doing it?

Thanks in advance,
Sebastien

Re: Solr performance issues

[Sébastien Rainville <se...@gmail.com>]

On Fri, Jun 20, 2008 at 8:32 AM, Erik Hatcher <er...@ehatchersolutions.com>
wrote:

>
> On Jun 19, 2008, at 6:28 PM, Yonik Seeley wrote:
>
>> 2. I use acts_as_solr and by default they only make "post" requests, even
>>> for /select. With that setup the response time for most queries, simple
>>> or
>>> complex ones, were ranging from 150ms to 600ms, with an average of 250ms.
>>> I
>>> changed the select request to use "get" requests instead and now the
>>> response time is down to 10ms to 60ms. Did someone seen that before? Why
>>> is
>>> it doing it?
>>>
>>
>> Are the get requests being cached by the ruby stuff?
>>
>
> No, I'm sure that the results aren't being cached by Ruby's library,
> solr-ruby, or acts_as_solr.
>

I confirm that the results are not cached by Ruby's library.


But even with no caching, I've seen differences with get/post on Linux
>> with the python client when persistent HTTP connections were in use.
>> I tracked it down to the POST being written in two parts, triggering
>> nagle's algorithm in the networking stack.
>>
>
> There was another post I found that mentioned this a couple of years ago:
>
> <http://markmail.org/message/45qflvwnakhripqp>
>
> I would welcome patches with tests that allow solr-ruby to send most
> requests with GET, and the ones that are actually sending a body beyond just
> parameters (delete, update, commit) as POST.
>
>        Erik
>
>
I made a few modifications but it still need more testing...

Sebastien

Re: Solr performance issues

[Erik Hatcher <er...@ehatchersolutions.com>]

On Jun 19, 2008, at 6:28 PM, Yonik Seeley wrote:
>> 2. I use acts_as_solr and by default they only make "post"  
>> requests, even
>> for /select. With that setup the response time for most queries,  
>> simple or
>> complex ones, were ranging from 150ms to 600ms, with an average of  
>> 250ms. I
>> changed the select request to use "get" requests instead and now the
>> response time is down to 10ms to 60ms. Did someone seen that  
>> before? Why is
>> it doing it?
>
> Are the get requests being cached by the ruby stuff?

No, I'm sure that the results aren't being cached by Ruby's library,  
solr-ruby, or acts_as_solr.

> But even with no caching, I've seen differences with get/post on Linux
> with the python client when persistent HTTP connections were in use.
> I tracked it down to the POST being written in two parts, triggering
> nagle's algorithm in the networking stack.

There was another post I found that mentioned this a couple of years  
ago:

<http://markmail.org/message/45qflvwnakhripqp>

I would welcome patches with tests that allow solr-ruby to send most  
requests with GET, and the ones that are actually sending a body  
beyond just parameters (delete, update, commit) as POST.

	Erik

Re: Solr performance issues

[Yonik Seeley <yo...@apache.org>]

On Thu, Jun 19, 2008 at 6:11 PM, Sébastien Rainville
<se...@gmail.com> wrote:
> I've been using solr for a little without worrying too much about how it
> works but now it's becoming a bottleneck in my application. I have a couple
> issues with it:
>
> 1. My index always gets slower and slower when commiting/optimizing for some
> obscure reason. It goes from 1 second with a new index to 45 seconds with an
> index with the same amount of data but used for a few days. Restarting solr
> doesn't fix it. The only way I found to fix that is to delete the whole
> index completely by deleting the index folder. Then when I rebuild the index
> everything goes back to normal and fast... and then performance slowly
> deteriorates again. So, the amount of data is not a factor because
> rebuilding the index from scratch fixes the problem and I am sending
> "optimize" once in a while... even maybe too often.

This sounds like OS caching to me.  A large amount of a "new" index
that was just written will be in cache and thus much faster to
optimize.

If your index is smaller than the amount of RAM, go to the index
directory of an "old" index, then try "cat * > /dev/null" and then try
optimize to see of that's the case.

> 2. I use acts_as_solr and by default they only make "post" requests, even
> for /select. With that setup the response time for most queries, simple or
> complex ones, were ranging from 150ms to 600ms, with an average of 250ms. I
> changed the select request to use "get" requests instead and now the
> response time is down to 10ms to 60ms. Did someone seen that before? Why is
> it doing it?

Are the get requests being cached by the ruby stuff?

But even with no caching, I've seen differences with get/post on Linux
with the python client when persistent HTTP connections were in use.
I tracked it down to the POST being written in two parts, triggering
nagle's algorithm in the networking stack.

-Yonik