You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jonathan Ariel <io...@gmail.com> on 2008/04/14 23:50:08 UTC
too many queries?
Hi,
I have some questions about performance for you guys.
So basically I have 2 slave solr servers and 1 master solr server load
balanced and around 100 request/second, aprox. 50 request per second per
solr server.
My index is about 4 million documents and the average query response time is
0.6 seconds, retrieving just 4 documents per query.
What happens is that there are too many request to Solr and every second is
getting bigger, so eventually my site stops working.
I don't know if this stats are enough to tell if the servers are supposed to
handle this amount of request. Maybe it's a configuration problem. I don't
think that caching in solr would help in this case because all the queries
are different (I'm not sure how caching works but if it's per query it won't
help much in this case).
Any thoughts about this?
Thanks!
Jonathan
Re: too many queries?
Posted by Jonathan Ariel <io...@gmail.com>.
Ok. So I tried with less cache as I told you, but the server goes down after
a while. I will use jconsole to monitor the servers and see what happens.
Maybe it is a memory issue? With the cache changes I noticed that there are
no evictions at all and the facet query time went from 500ms to 80ms in most
of the cases.
I tried commiting every 5 minutes instead of 2 minutes, but eventually the
server stop responding. I could see that there are lots of open and active
connections to solr in that moment, so I assume that maybe the warmups or
the commits causes a bottleneck.
Anyways, I'll try jconsole and let you know.
On Wed, Apr 16, 2008 at 5:32 PM, oleg_gnatovskiy <
oleg_gnatovskiy@citysearch.com> wrote:
>
> Oh ok. That makes sense. Thanks.
>
> Otis Gospodnetic wrote:
> >
> > Oleg, you can't explicitly say "N GB for index". Wunder was just saying
> > how much you can imagine how much RAM each piece might need and be happy
> > with.
> >
> > Otis
> > --
> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> >
> > ----- Original Message ----
> > From: oleg_gnatovskiy <ol...@citysearch.com>
> > To: solr-user@lucene.apache.org
> > Sent: Wednesday, April 16, 2008 2:05:23 PM
> > Subject: Re: too many queries?
> >
> >
> > Hello. I am having a similar problem as the OP. I see that you
> recommended
> > setting 4GB for the index, and 2 for Solr. How do I allocate memory for
> > the
> > index? I was under the impression that Solr did not support a RAMIndex.
> >
> >
> > Walter Underwood wrote:
> >>
> >> Do it. 32-bit OS's went out of style five years ago in server-land.
> >>
> >> I would start with 8GB of RAM. 4GB for your index, 2 for Solr, 1 for
> >> the OS and 1 for other processes. That might be tight. 12GB would
> >> be a lot better.
> >>
> >> wunder
> >>
> >> On 4/16/08 7:50 AM, "Jonathan Ariel" <io...@gmail.com> wrote:
> >>
> >>> In order to do that I have to change to a 64 bits OS so I can have
> more
> >>> than
> >>> 4 GB of RAM.Is there any way to see how long does it takes to Solr to
> >>> warmup
> >>> the searcher?
> >>>
> >>> On Wed, Apr 16, 2008 at 11:40 AM, Walter Underwood
> >>> <wu...@netflix.com>
> >>> wrote:
> >>>
> >>>> A commit every two minutes means that the Solr caches are flushed
> >>>> before they even start to stabilize. Two things to try:
> >>>>
> >>>> * commit less often, 5 minutes or 10 minutes
> >>>> * have enough RAM that your entire index can fit in OS file buffers
> >>>>
> >>>> wunder
> >>>>
> >>>> On 4/16/08 6:27 AM, "Jonathan Ariel" <io...@gmail.com> wrote:
> >>>>
> >>>>> So I counted the number if distinct values that I have for each
> field
> >>>> that I
> >>>>> want a facet on. In total it's around 100,000. I tried with a
> >>>> filterCache
> >>>>> of 120,000 but it seems like too much because the server went down.
> I
> >>>> will
> >>>>> try with less, around 75,000 and let you know.
> >>>>>
> >>>>> How do you to partition the data to a static set and a dynamic set,
> >>>>> and
> >>>> then
> >>>>> combining them at query time? Do you have a link to read about that?
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Apr 15, 2008 at 7:21 PM, Mike Klaas <mi...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> On 15-Apr-08, at 5:38 AM, Jonathan Ariel wrote:
> >>>>>>
> >>>>>>> My index is 4GB on disk. My servers has 8 GB of RAM each (the OS
> is
> >>>>>>> 32
> >>>>>>> bits).
> >>>>>>> It is optimized twice a day, it takes around 15 minutes to
> optimize.
> >>>>>>> The index is updated (commits) every two minutes. There are
> between
> >>>>>>> 10
> >>>>>>> and
> >>>>>>> 100 inserts/updates every 2 minutes.
> >>>>>>>
> >>>>>>
> >>>>>> Caching could help--you should definitely start there.
> >>>>>>
> >>>>>> The commit every 2 minutes could end up being an unsurmountable
> >>>> problem.
> >>>>>> You may have to partition your data into a large, mostly static
> set
> >>>> and a
> >>>>>> small dynamic set, combining the results at query time.
> >>>>>>
> >>>>>> -Mike
> >>>>>>
> >>>>
> >>>>
> >>
> >>
> >>
> >
> > --
> > View this message in context:
> > http://www.nabble.com/too-many-queries--tp16690870p16727264.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
> >
> >
> >
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/too-many-queries--tp16690870p16732932.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Re: too many queries?
Posted by oleg_gnatovskiy <ol...@citysearch.com>.
Oh ok. That makes sense. Thanks.
Otis Gospodnetic wrote:
>
> Oleg, you can't explicitly say "N GB for index". Wunder was just saying
> how much you can imagine how much RAM each piece might need and be happy
> with.
>
> Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
> ----- Original Message ----
> From: oleg_gnatovskiy <ol...@citysearch.com>
> To: solr-user@lucene.apache.org
> Sent: Wednesday, April 16, 2008 2:05:23 PM
> Subject: Re: too many queries?
>
>
> Hello. I am having a similar problem as the OP. I see that you recommended
> setting 4GB for the index, and 2 for Solr. How do I allocate memory for
> the
> index? I was under the impression that Solr did not support a RAMIndex.
>
>
> Walter Underwood wrote:
>>
>> Do it. 32-bit OS's went out of style five years ago in server-land.
>>
>> I would start with 8GB of RAM. 4GB for your index, 2 for Solr, 1 for
>> the OS and 1 for other processes. That might be tight. 12GB would
>> be a lot better.
>>
>> wunder
>>
>> On 4/16/08 7:50 AM, "Jonathan Ariel" <io...@gmail.com> wrote:
>>
>>> In order to do that I have to change to a 64 bits OS so I can have more
>>> than
>>> 4 GB of RAM.Is there any way to see how long does it takes to Solr to
>>> warmup
>>> the searcher?
>>>
>>> On Wed, Apr 16, 2008 at 11:40 AM, Walter Underwood
>>> <wu...@netflix.com>
>>> wrote:
>>>
>>>> A commit every two minutes means that the Solr caches are flushed
>>>> before they even start to stabilize. Two things to try:
>>>>
>>>> * commit less often, 5 minutes or 10 minutes
>>>> * have enough RAM that your entire index can fit in OS file buffers
>>>>
>>>> wunder
>>>>
>>>> On 4/16/08 6:27 AM, "Jonathan Ariel" <io...@gmail.com> wrote:
>>>>
>>>>> So I counted the number if distinct values that I have for each field
>>>> that I
>>>>> want a facet on. In total it's around 100,000. I tried with a
>>>> filterCache
>>>>> of 120,000 but it seems like too much because the server went down. I
>>>> will
>>>>> try with less, around 75,000 and let you know.
>>>>>
>>>>> How do you to partition the data to a static set and a dynamic set,
>>>>> and
>>>> then
>>>>> combining them at query time? Do you have a link to read about that?
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Apr 15, 2008 at 7:21 PM, Mike Klaas <mi...@gmail.com>
>>>> wrote:
>>>>>
>>>>>> On 15-Apr-08, at 5:38 AM, Jonathan Ariel wrote:
>>>>>>
>>>>>>> My index is 4GB on disk. My servers has 8 GB of RAM each (the OS is
>>>>>>> 32
>>>>>>> bits).
>>>>>>> It is optimized twice a day, it takes around 15 minutes to optimize.
>>>>>>> The index is updated (commits) every two minutes. There are between
>>>>>>> 10
>>>>>>> and
>>>>>>> 100 inserts/updates every 2 minutes.
>>>>>>>
>>>>>>
>>>>>> Caching could help--you should definitely start there.
>>>>>>
>>>>>> The commit every 2 minutes could end up being an unsurmountable
>>>> problem.
>>>>>> You may have to partition your data into a large, mostly static set
>>>> and a
>>>>>> small dynamic set, combining the results at query time.
>>>>>>
>>>>>> -Mike
>>>>>>
>>>>
>>>>
>>
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/too-many-queries--tp16690870p16727264.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>
>
>
>
--
View this message in context: http://www.nabble.com/too-many-queries--tp16690870p16732932.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: too many queries?
Posted by Walter Underwood <wu...@netflix.com>.
4GB for the operating system to use to buffer disk files.
That is not a Solr setting.
wunder
On 4/16/08 11:05 AM, "oleg_gnatovskiy" <ol...@citysearch.com>
wrote:
>
> Hello. I am having a similar problem as the OP. I see that you recommended
> setting 4GB for the index, and 2 for Solr. How do I allocate memory for the
> index? I was under the impression that Solr did not support a RAMIndex.
>
>
> Walter Underwood wrote:
>>
>> Do it. 32-bit OS's went out of style five years ago in server-land.
>>
>> I would start with 8GB of RAM. 4GB for your index, 2 for Solr, 1 for
>> the OS and 1 for other processes. That might be tight. 12GB would
>> be a lot better.
>>
>> wunder
>>
>> On 4/16/08 7:50 AM, "Jonathan Ariel" <io...@gmail.com> wrote:
>>
>>> In order to do that I have to change to a 64 bits OS so I can have more
>>> than
>>> 4 GB of RAM.Is there any way to see how long does it takes to Solr to
>>> warmup
>>> the searcher?
>>>
>>> On Wed, Apr 16, 2008 at 11:40 AM, Walter Underwood
>>> <wu...@netflix.com>
>>> wrote:
>>>
>>>> A commit every two minutes means that the Solr caches are flushed
>>>> before they even start to stabilize. Two things to try:
>>>>
>>>> * commit less often, 5 minutes or 10 minutes
>>>> * have enough RAM that your entire index can fit in OS file buffers
>>>>
>>>> wunder
>>>>
>>>> On 4/16/08 6:27 AM, "Jonathan Ariel" <io...@gmail.com> wrote:
>>>>
>>>>> So I counted the number if distinct values that I have for each field
>>>> that I
>>>>> want a facet on. In total it's around 100,000. I tried with a
>>>> filterCache
>>>>> of 120,000 but it seems like too much because the server went down. I
>>>> will
>>>>> try with less, around 75,000 and let you know.
>>>>>
>>>>> How do you to partition the data to a static set and a dynamic set, and
>>>> then
>>>>> combining them at query time? Do you have a link to read about that?
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Apr 15, 2008 at 7:21 PM, Mike Klaas <mi...@gmail.com>
>>>> wrote:
>>>>>
>>>>>> On 15-Apr-08, at 5:38 AM, Jonathan Ariel wrote:
>>>>>>
>>>>>>> My index is 4GB on disk. My servers has 8 GB of RAM each (the OS is
>>>>>>> 32
>>>>>>> bits).
>>>>>>> It is optimized twice a day, it takes around 15 minutes to optimize.
>>>>>>> The index is updated (commits) every two minutes. There are between
>>>>>>> 10
>>>>>>> and
>>>>>>> 100 inserts/updates every 2 minutes.
>>>>>>>
>>>>>>
>>>>>> Caching could help--you should definitely start there.
>>>>>>
>>>>>> The commit every 2 minutes could end up being an unsurmountable
>>>> problem.
>>>>>> You may have to partition your data into a large, mostly static set
>>>> and a
>>>>>> small dynamic set, combining the results at query time.
>>>>>>
>>>>>> -Mike
>>>>>>
>>>>
>>>>
>>
>>
>>
Re: too many queries?
Posted by oleg_gnatovskiy <ol...@citysearch.com>.
Hello. I am having a similar problem as the OP. I see that you recommended
setting 4GB for the index, and 2 for Solr. How do I allocate memory for the
index? I was under the impression that Solr did not support a RAMIndex.
Walter Underwood wrote:
>
> Do it. 32-bit OS's went out of style five years ago in server-land.
>
> I would start with 8GB of RAM. 4GB for your index, 2 for Solr, 1 for
> the OS and 1 for other processes. That might be tight. 12GB would
> be a lot better.
>
> wunder
>
> On 4/16/08 7:50 AM, "Jonathan Ariel" <io...@gmail.com> wrote:
>
>> In order to do that I have to change to a 64 bits OS so I can have more
>> than
>> 4 GB of RAM.Is there any way to see how long does it takes to Solr to
>> warmup
>> the searcher?
>>
>> On Wed, Apr 16, 2008 at 11:40 AM, Walter Underwood
>> <wu...@netflix.com>
>> wrote:
>>
>>> A commit every two minutes means that the Solr caches are flushed
>>> before they even start to stabilize. Two things to try:
>>>
>>> * commit less often, 5 minutes or 10 minutes
>>> * have enough RAM that your entire index can fit in OS file buffers
>>>
>>> wunder
>>>
>>> On 4/16/08 6:27 AM, "Jonathan Ariel" <io...@gmail.com> wrote:
>>>
>>>> So I counted the number if distinct values that I have for each field
>>> that I
>>>> want a facet on. In total it's around 100,000. I tried with a
>>> filterCache
>>>> of 120,000 but it seems like too much because the server went down. I
>>> will
>>>> try with less, around 75,000 and let you know.
>>>>
>>>> How do you to partition the data to a static set and a dynamic set, and
>>> then
>>>> combining them at query time? Do you have a link to read about that?
>>>>
>>>>
>>>>
>>>> On Tue, Apr 15, 2008 at 7:21 PM, Mike Klaas <mi...@gmail.com>
>>> wrote:
>>>>
>>>>> On 15-Apr-08, at 5:38 AM, Jonathan Ariel wrote:
>>>>>
>>>>>> My index is 4GB on disk. My servers has 8 GB of RAM each (the OS is
>>>>>> 32
>>>>>> bits).
>>>>>> It is optimized twice a day, it takes around 15 minutes to optimize.
>>>>>> The index is updated (commits) every two minutes. There are between
>>>>>> 10
>>>>>> and
>>>>>> 100 inserts/updates every 2 minutes.
>>>>>>
>>>>>
>>>>> Caching could help--you should definitely start there.
>>>>>
>>>>> The commit every 2 minutes could end up being an unsurmountable
>>> problem.
>>>>> You may have to partition your data into a large, mostly static set
>>> and a
>>>>> small dynamic set, combining the results at query time.
>>>>>
>>>>> -Mike
>>>>>
>>>
>>>
>
>
>
--
View this message in context: http://www.nabble.com/too-many-queries--tp16690870p16727264.html
Sent from the Solr - User mailing list archive at Nabble.com.