You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by roySolr <ro...@gmail.com> on 2011/06/07 12:22:50 UTC
How many fields can SOLR handle?
Hello,
I have a SOLR implementation with 1m products. Every products has some
information, lets say a television has some information about pixels and
inches, a computer has information about harddisk, cpu, gpu. When a user
search for computer i want to show the correct facets. An example:
User search for Computer
Facets:
CPU
AMD(10)
Intel(300)
GPU
Nvidia(20)
Ati(290)
Every product has different facets. I have something like this in my schema:
<dynamicField name="*_FACET" type="facetType" indexed="true" stored="true"
multiValued="true"/>
In SOLR i have now a lot of fields: CPU_FACET, GPU_FACET etc. How many
fields can SOLR handle?
Another question: Is it possible to add the FACET fields automatically to my
query? facet.field=*_FACET? Now i do first a request to a DB to get the
FACET titles and add this to the request: facet.field=cpu_FACET,gpu_FACET.
I'm affraid that *_FACET is a overkill solution.
--
View this message in context: http://lucene.472066.n3.nabble.com/How-many-fields-can-SOLR-handle-tp3033910p3033910.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How many fields can SOLR handle?
Posted by roySolr <ro...@gmail.com>.
Thanks Bill,
That's exactly what i mean. But first i do a request to get the right
facetFields from a category.
So a user search for TV, i do request to a db to get tv_size and resolution.
The next step is to
add this to my query like this: facet.field=tv_size&facet.field=resolution.
I thought maybe it's possible to add the FACET fields automatically to my
query(based on category). I understand this isn't possible and i need to do
first a request to get the facet.fields.
--
View this message in context: http://lucene.472066.n3.nabble.com/How-many-fields-can-SOLR-handle-tp3033910p3139921.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How many fields can SOLR handle?
Posted by Bill Bell <bi...@gmail.com>.
This is taxonomy/index design...
One way is to have a series of fields by category:
TV - tv_size, resolution
Computer - cpu, gpu
Solr can have as many fields as you need, and if you do not store them
into the index they are ignored.
So if a user picks "TV", you pass these to Solr:
q=*:*&facet=true&facet.field=tv_size&facet.field=resolution
If a user picks "Computer", you pass these to Solr:
q=*:*&facet=true&facet.field=cpu&facet.field=gpu
The other option is to return ALL of the fields facet'd, but this is not
recommended since
you would certainly have performance issues depending on the number of
fields.
On 7/5/11 1:00 AM, "roySolr" <ro...@gmail.com> wrote:
>Hi,
>
>I know i can add components to my requesthandler. In this situation facets
>are dependent of there category. So if a user choose for the category TV:
>
>Inch:
>32 inch(5)
>34 inch(3)
>40 inch(1)
>
>Resolution:
>Full HD(5)
>HD ready(2)
>
>When a user search for category Computer:
>
>CPU:
>Intel(12)
>AMD(10)
>
>GPU:
>Ati(5)
>Nvidia(2)
>
>So i can't put it in my requesthandler as default. Every search there can
>be
>other facets. Do you understand what i mean?
>
>--
>View this message in context:
>http://lucene.472066.n3.nabble.com/How-many-fields-can-SOLR-handle-tp30339
>10p3139833.html
>Sent from the Solr - User mailing list archive at Nabble.com.
Re: How many fields can SOLR handle?
Posted by roySolr <ro...@gmail.com>.
Hi,
I know i can add components to my requesthandler. In this situation facets
are dependent of there category. So if a user choose for the category TV:
Inch:
32 inch(5)
34 inch(3)
40 inch(1)
Resolution:
Full HD(5)
HD ready(2)
When a user search for category Computer:
CPU:
Intel(12)
AMD(10)
GPU:
Ati(5)
Nvidia(2)
So i can't put it in my requesthandler as default. Every search there can be
other facets. Do you understand what i mean?
--
View this message in context: http://lucene.472066.n3.nabble.com/How-many-fields-can-SOLR-handle-tp3033910p3139833.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How many fields can SOLR handle?
Posted by Marian Steinbach <ma...@gmail.com>.
Hi!
I can't help you with the question about the limit to the number of
fields. But until now I haven't read anywhere that there is a limit.
So I'd assume that there is none.
For your second question:
"Another question: Is it possible to add the FACET fields automatically to my
query? facet.field=*_FACET? Now i do first a request to a DB to get the
FACET titles and add this to the request: facet.field=cpu_FACET,gpu_FACET.
I'm affraid that *_FACET is a overkill solution."
You can add parameters automatically (as defaults) to your requests.
Look into the solrconfig.xml file for requestHandler that handles your
requests. (In the example it's the one starting <requestHandler
name="search" class="solr.SearchHandler" default="true">). There is a
<lst name="defaults"> where you can add as many request parameters as
you like.
Is that what you're talking about?
On Mon, Jul 4, 2011 at 13:44, roySolr <ro...@gmail.com> wrote:
> Nobody? I'm still confused about this
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-many-fields-can-SOLR-handle-tp3033910p3137301.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: How many fields can SOLR handle?
Posted by roySolr <ro...@gmail.com>.
Nobody? I'm still confused about this
--
View this message in context: http://lucene.472066.n3.nabble.com/How-many-fields-can-SOLR-handle-tp3033910p3137301.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How many fields can SOLR handle?
Posted by roySolr <ro...@gmail.com>.
Yes i use something like that. I make a db connection to get the facets for
the chosen category. With this data i add facet.fields dynamically:
example:
foreach(results as result){
qStr = "facet.field=" . result;
}
I was searching for a solution that i don't need to get the facets from db.
Now i cache this list(PHP) so i don't need a request every time to the db.
Thanks for all the help!!
--
View this message in context: http://lucene.472066.n3.nabble.com/How-many-fields-can-SOLR-handle-tp3033910p3150970.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How many fields can SOLR handle?
Posted by William Bell <bi...@gmail.com>.
RoySolr,
Not sure what language your client is written in, but this is a simple
if statement.
if (category == "TV") {
qStr = "q=*:*&facet=true&facet.field=tv_size&facet.field=resolution";
elseif (category == "Computer") {
qStr = "q=*:*&facet=true&facet.field=cpu&facet.field=gpu";
}
curl "http://localhost:8983/solr/select?" + qStr;
On Thu, Jul 7, 2011 at 2:29 AM, roySolr <ro...@gmail.com> wrote:
> Hello Erik,
>
> I need the *_facets also for searching so stored must be true.
>
> "Then, and I used *_facet similar to you, kept a list of all *_facet actual
> field names and used those in all subsequent search requests. "
>
> Is this not bad for performance? I only need a few facets, not all.(only the
> facets for the chosen category)
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-many-fields-can-SOLR-handle-tp3033910p3147520.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Bill Bell
billnbell@gmail.com
cell 720-256-8076
Re: How many fields can SOLR handle?
Posted by roySolr <ro...@gmail.com>.
Hello Erik,
I need the *_facets also for searching so stored must be true.
"Then, and I used *_facet similar to you, kept a list of all *_facet actual
field names and used those in all subsequent search requests. "
Is this not bad for performance? I only need a few facets, not all.(only the
facets for the chosen category)
--
View this message in context: http://lucene.472066.n3.nabble.com/How-many-fields-can-SOLR-handle-tp3033910p3147520.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How many fields can SOLR handle?
Posted by Erik Hatcher <er...@gmail.com>.
On Jun 7, 2011, at 06:22 , roySolr wrote:
> Every product has different facets. I have something like this in my schema:
>
> <dynamicField name="*_FACET" type="facetType" indexed="true" stored="true"
> multiValued="true"/>
One optimization, if you don't need the stored values, is to set stored="false". Faceting is driven off the indexed terms, not the stored values. But maybe you need the stored values (but maybe not).
> In SOLR i have now a lot of fields: CPU_FACET, GPU_FACET etc. How many
> fields can SOLR handle?
>
> Another question: Is it possible to add the FACET fields automatically to my
> query? facet.field=*_FACET? Now i do first a request to a DB to get the
> FACET titles and add this to the request: facet.field=cpu_FACET,gpu_FACET.
> I'm affraid that *_FACET is a overkill solution.
Solr currently cannot do this automatically. But....
One thing that I built in to Solr Flare (long live Solr Flare!) was a startup request to Solr's luke request handler (/admin/luke?numTerms=0) and got the list of field names. Then, and I used *_facet similar to you, kept a list of all *_facet actual field names and used those in all subsequent search requests.
Erik