You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by srinalluri <na...@yahoo.com> on 2012/08/22 16:35:31 UTC
search is slow for URL fields of type String.
This is string fieldType:
<fieldType name="string" class="solr.StrField" sortMissingLast="true" />
These are the filelds using 'string' fieldType:
<field name="image_url" type="string" indexed="true" stored="true"
multiValued="true" />
<field name="url" type="string" indexed="true" stored="true"
multiValued="true" />
And this the sample query:
/select/?q=url:http\://www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/
AND image_url:*
Each query like this taking around 400 milli seconds. What are the change I
can do to the fieldType to improve query performance?
thanks
Srini
--
View this message in context: http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: search is slow for URL fields of type String.
Posted by Karthick Duraisamy Soundararaj <ka...@gmail.com>.
Srini,
Whats the size of your index? You are saying that searching on
'string' fieldType takes 400 milli seconds but did you try searching on any
other fieldType other than string? If so, how much time did it take?
On Wed, Aug 22, 2012 at 10:35 AM, srinalluri <na...@yahoo.com> wrote:
> This is string fieldType:
>
> <fieldType name="string" class="solr.StrField" sortMissingLast="true"
> />
>
> These are the filelds using 'string' fieldType:
>
> <field name="image_url" type="string" indexed="true" stored="true"
> multiValued="true" />
> <field name="url" type="string" indexed="true" stored="true"
> multiValued="true" />
>
> And this the sample query:
> /select/?q=url:http\://
> www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/
> AND image_url:*
>
> Each query like this taking around 400 milli seconds. What are the change I
> can do to the fieldType to improve query performance?
>
> thanks
> Srini
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: search is slow for URL fields of type String.
Posted by Erik Hatcher <er...@gmail.com>.
Also note, emphasizing what Erick said below, that with this type of "has a value in field" type clause, it works fine as an fq as that gets cached so you only take the performance hit once on it. Generally this is a clause that is reused so be sure to peel it off as an fq rather than AND'ing it to a q(uery) parameter.
The advice to make a separate has_<fieldname> field (or field_size) is the best advice, but when dealing with low cardinality fields it's not really an issue to use something like category:* where there are only a handful of category values in use.
Erik
On Aug 23, 2012, at 08:51 , Jack Krupansky wrote:
> And we should probably add a doc note with this same advice since it is an easy "mistake" to make.
>
> -- Jack Krupansky
>
> -----Original Message----- From: Erick Erickson
> Sent: Thursday, August 23, 2012 8:44 AM
> To: solr-user@lucene.apache.org
> Subject: Re: search is slow for URL fields of type String.
>
> There was just a thread on this, it's may be your
> &image_url:*
>
> try removing this clause just to test response time. If
> that shows a vast improvement, try adding a boolean
> field has_image_url, and then add a fq clause like
> &fq=has_image_url:true
>
> Best
> Erick
>
> On Wed, Aug 22, 2012 at 10:35 AM, srinalluri <na...@yahoo.com> wrote:
>> This is string fieldType:
>>
>> <fieldType name="string" class="solr.StrField" sortMissingLast="true" />
>>
>> These are the filelds using 'string' fieldType:
>>
>> <field name="image_url" type="string" indexed="true" stored="true"
>> multiValued="true" />
>> <field name="url" type="string" indexed="true" stored="true"
>> multiValued="true" />
>>
>> And this the sample query:
>> /select/?q=url:http\://www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/
>> AND image_url:*
>>
>> Each query like this taking around 400 milli seconds. What are the change I
>> can do to the fieldType to improve query performance?
>>
>> thanks
>> Srini
>>
>>
>>
>> --
>> View this message in context: http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: search is slow for URL fields of type String.
Posted by Jack Krupansky <ja...@basetechnology.com>.
And we should probably add a doc note with this same advice since it is an
easy "mistake" to make.
-- Jack Krupansky
-----Original Message-----
From: Erick Erickson
Sent: Thursday, August 23, 2012 8:44 AM
To: solr-user@lucene.apache.org
Subject: Re: search is slow for URL fields of type String.
There was just a thread on this, it's may be your
&image_url:*
try removing this clause just to test response time. If
that shows a vast improvement, try adding a boolean
field has_image_url, and then add a fq clause like
&fq=has_image_url:true
Best
Erick
On Wed, Aug 22, 2012 at 10:35 AM, srinalluri <na...@yahoo.com> wrote:
> This is string fieldType:
>
> <fieldType name="string" class="solr.StrField" sortMissingLast="true"
> />
>
> These are the filelds using 'string' fieldType:
>
> <field name="image_url" type="string" indexed="true" stored="true"
> multiValued="true" />
> <field name="url" type="string" indexed="true" stored="true"
> multiValued="true" />
>
> And this the sample query:
> /select/?q=url:http\://www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/
> AND image_url:*
>
> Each query like this taking around 400 milli seconds. What are the change
> I
> can do to the fieldType to improve query performance?
>
> thanks
> Srini
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: search is slow for URL fields of type String.
Posted by Erick Erickson <er...@gmail.com>.
There was just a thread on this, it's may be your
&image_url:*
try removing this clause just to test response time. If
that shows a vast improvement, try adding a boolean
field has_image_url, and then add a fq clause like
&fq=has_image_url:true
Best
Erick
On Wed, Aug 22, 2012 at 10:35 AM, srinalluri <na...@yahoo.com> wrote:
> This is string fieldType:
>
> <fieldType name="string" class="solr.StrField" sortMissingLast="true" />
>
> These are the filelds using 'string' fieldType:
>
> <field name="image_url" type="string" indexed="true" stored="true"
> multiValued="true" />
> <field name="url" type="string" indexed="true" stored="true"
> multiValued="true" />
>
> And this the sample query:
> /select/?q=url:http\://www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/
> AND image_url:*
>
> Each query like this taking around 400 milli seconds. What are the change I
> can do to the fieldType to improve query performance?
>
> thanks
> Srini
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html
> Sent from the Solr - User mailing list archive at Nabble.com.