Re: Limiting facets for huge data - setting indexed=false in schema.xml

[Erik Hatcher <er...@ehatchersolutions.com>]

On Jul 31, 2009, at 7:17 AM, Rahul R wrote:

> Erik,
> I understand that caching is going to improve performance. Infact we  
> did a
> PSR run with caches enabled and we got awesome results. But these  
> wouldn't
> be really representative because the PSR scripts will be doing the  
> same
> searches again and again. These would be cached and there would be  
> virtually
> no evictions. This is not a practical case.

I don't understand how this is not practical.  Why wouldn't having the  
caches warmed and filled with the facets be practical for your needs?

	Erik

Re: Limiting facets for huge data - setting indexed=false in schema.xml

[Rahul R <ra...@gmail.com>]

We are using 1.3.0. Thanks for the suggestion. Will see if I can try one of
the ngihtly builds.

On Fri, Jul 31, 2009 at 7:49 PM, Erik Hatcher <er...@ehatchersolutions.com>wrote:

> What version of Solr?   Try a nightly build if you're at Solr 1.3 or
> earlier and you'll be amazed at the difference.
>
>        Erik
>
>
> On Jul 31, 2009, at 10:00 AM, Rahul R wrote:
>
> In a production environment, having the caches enabled makes a lot of
>> sense.
>> And most definitely we will be enabling them. However, the primary idea of
>> this exercise is to verify if limiting the number of facets will actually
>> improve the performance.
>>
>> An update on this. I did verify and looks like although I set
>> indexed=false
>> for most of the properties, I have not blocked them from participating in
>> the query. I now enabled only 7 properties for faceting. Now at any given
>> time only a maximum of 7 facets will participate in the query. Performance
>> has now improved from an erstwhile 60 seconds to around 10 seconds.
>>
>> This really helped. Thanks a lot !
>>
>> Regards
>> Rahul
>>
>> On Fri, Jul 31, 2009 at 6:34 PM, Erik Hatcher <erik@ehatchersolutions.com
>> >wrote:
>>
>>
>>> On Jul 31, 2009, at 7:17 AM, Rahul R wrote:
>>>
>>> Erik,
>>>
>>>> I understand that caching is going to improve performance. Infact we did
>>>> a
>>>> PSR run with caches enabled and we got awesome results. But these
>>>> wouldn't
>>>> be really representative because the PSR scripts will be doing the same
>>>> searches again and again. These would be cached and there would be
>>>> virtually
>>>> no evictions. This is not a practical case.
>>>>
>>>>
>>> I don't understand how this is not practical.  Why wouldn't having the
>>> caches warmed and filled with the facets be practical for your needs?
>>>
>>>      Erik
>>>
>>>
>>>
>

Re: Limiting facets for huge data - setting indexed=false in schema.xml

[Erik Hatcher <er...@ehatchersolutions.com>]

What version of Solr?   Try a nightly build if you're at Solr 1.3 or  
earlier and you'll be amazed at the difference.

	Erik

On Jul 31, 2009, at 10:00 AM, Rahul R wrote:

> In a production environment, having the caches enabled makes a lot  
> of sense.
> And most definitely we will be enabling them. However, the primary  
> idea of
> this exercise is to verify if limiting the number of facets will  
> actually
> improve the performance.
>
> An update on this. I did verify and looks like although I set  
> indexed=false
> for most of the properties, I have not blocked them from  
> participating in
> the query. I now enabled only 7 properties for faceting. Now at any  
> given
> time only a maximum of 7 facets will participate in the query.  
> Performance
> has now improved from an erstwhile 60 seconds to around 10 seconds.
>
> This really helped. Thanks a lot !
>
> Regards
> Rahul
>
> On Fri, Jul 31, 2009 at 6:34 PM, Erik Hatcher <erik@ehatchersolutions.com 
> >wrote:
>
>>
>> On Jul 31, 2009, at 7:17 AM, Rahul R wrote:
>>
>> Erik,
>>> I understand that caching is going to improve performance. Infact  
>>> we did a
>>> PSR run with caches enabled and we got awesome results. But these  
>>> wouldn't
>>> be really representative because the PSR scripts will be doing the  
>>> same
>>> searches again and again. These would be cached and there would be
>>> virtually
>>> no evictions. This is not a practical case.
>>>
>>
>> I don't understand how this is not practical.  Why wouldn't having  
>> the
>> caches warmed and filled with the facets be practical for your needs?
>>
>>       Erik
>>
>>

Re: Limiting facets for huge data - setting indexed=false in schema.xml

[Rahul R <ra...@gmail.com>]

In a production environment, having the caches enabled makes a lot of sense.
And most definitely we will be enabling them. However, the primary idea of
this exercise is to verify if limiting the number of facets will actually
improve the performance.

An update on this. I did verify and looks like although I set indexed=false
for most of the properties, I have not blocked them from participating in
the query. I now enabled only 7 properties for faceting. Now at any given
time only a maximum of 7 facets will participate in the query. Performance
has now improved from an erstwhile 60 seconds to around 10 seconds.

This really helped. Thanks a lot !

Regards
Rahul

On Fri, Jul 31, 2009 at 6:34 PM, Erik Hatcher <er...@ehatchersolutions.com>wrote:

>
> On Jul 31, 2009, at 7:17 AM, Rahul R wrote:
>
> Erik,
>> I understand that caching is going to improve performance. Infact we did a
>> PSR run with caches enabled and we got awesome results. But these wouldn't
>> be really representative because the PSR scripts will be doing the same
>> searches again and again. These would be cached and there would be
>> virtually
>> no evictions. This is not a practical case.
>>
>
> I don't understand how this is not practical.  Why wouldn't having the
> caches warmed and filled with the facets be practical for your needs?
>
>        Erik
>
>