You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Dave C." <da...@hotmail.com> on 2007/12/01 01:27:27 UTC
Tips for searching
Hello,
I was wondering if someone could offer some suggestions for a problem I am having.
When I perform a query on a field using a single character, I get no results back: for example, if i search on the "description" field for the character 'a', i.e. (description:a), I get no results back. I would like to get all documents where the letter 'a' appears in the description field.
I haven't played around with any of the settings in my schema.xml, so I am wondering if someone could point out how I could configure Solr such that it will return me all documents with an 'a' (or any arbitrary character) in the description field. If this could be expanded to match any sub-word (i.e. match "the" in "weather") that would be great as well.
Thanks for your help,
David
_________________________________________________________________
Connect and share in new ways with Windows Live.
http://www.windowslive.com/connect.html?ocid=TXT_TAGLM_Wave2_newways_112007
RE: Tips for searching
Posted by "Dave C." <da...@hotmail.com>.
Actually, yes, I want to be able to match *any* single character, not just 'a'.
The StopFilterFactory wiki doesn't help with that, apparently...correct?
-David
> From: daft.davy@hotmail.com
> To: solr-user@lucene.apache.org
> Subject: RE: Tips for searching
> Date: Sat, 1 Dec 2007 00:43:21 +0000
>
>
> Thanks for the quick response Mike...
> Ideally it should match more than just a single character, i.e. "the" in "weather" or "pro" in "profile" or "000" in "18000".
>
> Would these cases be taken care of by the StopFilterFactory?
>
> Thanks,
> david
>
>
> > From: mike.klaas@gmail.com
> > Subject: Re: Tips for searching
> > Date: Fri, 30 Nov 2007 16:40:41 -0800
> > To: solr-user@lucene.apache.org
> >
> > On 30-Nov-07, at 4:27 PM, Dave C. wrote:
> >
> > > When I perform a query on a field using a single character, I get
> > > no results back: for example, if i search on the "description"
> > > field for the character 'a', i.e. (description:a), I get no results
> > > back. I would like to get all documents where the letter 'a'
> > > appears in the description field.
> > >
> > > I haven't played around with any of the settings in my schema.xml,
> > > so I am wondering if someone could point out how I could configure
> > > Solr such that it will return me all documents with an 'a' (or any
> > > arbitrary character) in the description field. If this could be
> > > expanded to match any sub-word (i.e. match "the" in "weather") that
> > > would be great as well.
> >
> > Any single character, or just 'a'? IR systems often strip out stop
> > words (extremely common) words for efficiency reasons. This is
> > configured via the "StopFilterFactory" in schema.xml: just remove it
> > from the field you are interested in and reindex.
> >
> > -Mike
>
> _________________________________________________________________
> Share life as it happens with the new Windows Live.Download today it's FREE!
> http://www.windowslive.com/share.html?ocid=TXT_TAGLM_Wave2_sharelife_112007
_________________________________________________________________
Connect and share in new ways with Windows Live.
http://www.windowslive.com/connect.html?ocid=TXT_TAGLM_Wave2_newways_112007
RE: Tips for searching
Posted by Will Johnson <wi...@gmail.com>.
If you want any letter and any possible substring you might be better off
breaking every word into single letters with special tokens between words:
ie:
the quick brown fox
Becomes
t h e ZZ q u i c k ZZ b r o w n ZZ f o x
then you can do all the single letter searches and multi letter searches
turn into phrase searches. Ie:
uic (from quick)
would be rewritten as
"u i c"
And so on. This should give you better performance and more predictable
results than wildcard searches depending on the size and complexity of your
data. Relevancy would be horrible since the tf/idf would always have a
common denominator depending on character set but there are ways around that
as well.
- will
-----Original Message-----
From: Mike Klaas [mailto:mike.klaas@gmail.com]
Sent: Friday, November 30, 2007 7:51 PM
To: solr-user@lucene.apache.org
Subject: Re: Tips for searching
On 30-Nov-07, at 4:43 PM, Dave C. wrote:
>
> Thanks for the quick response Mike...
> Ideally it should match more than just a single character, i.e.
> "the" in "weather" or "pro" in "profile" or "000" in "18000".
>
> Would these cases be taken care of by the StopFilterFactory?
No... you are looking for variant of WildcardQuery's. Prefix
wildcards are supported (pro* -> profile), but generalize wildcard
queries aren't enabled by default. There has been lots of discussion
on the list if you do a search.
-Mike
Re: Tips for searching
Posted by Mike Klaas <mi...@gmail.com>.
On 30-Nov-07, at 4:43 PM, Dave C. wrote:
>
> Thanks for the quick response Mike...
> Ideally it should match more than just a single character, i.e.
> "the" in "weather" or "pro" in "profile" or "000" in "18000".
>
> Would these cases be taken care of by the StopFilterFactory?
No... you are looking for variant of WildcardQuery's. Prefix
wildcards are supported (pro* -> profile), but generalize wildcard
queries aren't enabled by default. There has been lots of discussion
on the list if you do a search.
-Mike
RE: Tips for searching
Posted by "Dave C." <da...@hotmail.com>.
Thanks for the quick response Mike...
Ideally it should match more than just a single character, i.e. "the" in "weather" or "pro" in "profile" or "000" in "18000".
Would these cases be taken care of by the StopFilterFactory?
Thanks,
david
> From: mike.klaas@gmail.com
> Subject: Re: Tips for searching
> Date: Fri, 30 Nov 2007 16:40:41 -0800
> To: solr-user@lucene.apache.org
>
> On 30-Nov-07, at 4:27 PM, Dave C. wrote:
>
> > When I perform a query on a field using a single character, I get
> > no results back: for example, if i search on the "description"
> > field for the character 'a', i.e. (description:a), I get no results
> > back. I would like to get all documents where the letter 'a'
> > appears in the description field.
> >
> > I haven't played around with any of the settings in my schema.xml,
> > so I am wondering if someone could point out how I could configure
> > Solr such that it will return me all documents with an 'a' (or any
> > arbitrary character) in the description field. If this could be
> > expanded to match any sub-word (i.e. match "the" in "weather") that
> > would be great as well.
>
> Any single character, or just 'a'? IR systems often strip out stop
> words (extremely common) words for efficiency reasons. This is
> configured via the "StopFilterFactory" in schema.xml: just remove it
> from the field you are interested in and reindex.
>
> -Mike
_________________________________________________________________
Share life as it happens with the new Windows Live.Download today it's FREE!
http://www.windowslive.com/share.html?ocid=TXT_TAGLM_Wave2_sharelife_112007
Re: Tips for searching
Posted by Mike Klaas <mi...@gmail.com>.
On 30-Nov-07, at 4:27 PM, Dave C. wrote:
> When I perform a query on a field using a single character, I get
> no results back: for example, if i search on the "description"
> field for the character 'a', i.e. (description:a), I get no results
> back. I would like to get all documents where the letter 'a'
> appears in the description field.
>
> I haven't played around with any of the settings in my schema.xml,
> so I am wondering if someone could point out how I could configure
> Solr such that it will return me all documents with an 'a' (or any
> arbitrary character) in the description field. If this could be
> expanded to match any sub-word (i.e. match "the" in "weather") that
> would be great as well.
Any single character, or just 'a'? IR systems often strip out stop
words (extremely common) words for efficiency reasons. This is
configured via the "StopFilterFactory" in schema.xml: just remove it
from the field you are interested in and reindex.
-Mike