You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Phillip Farber <pf...@umich.edu> on 2007/12/05 16:16:54 UTC
Searching for two terms together in a multiValued TextField
Hello,
I'm still new to Solr/Lucene. I want to search documents for 2 or more
terms that must appear together on a page. I have a multiValued
TextField called "page" in a document with uniqueId called "id" that
represents a OCR'd book. My default operator is AND. My default field is
"page". My query is:
q=adhesion+ring&fl=id,score&fq=id:(1+44)&version=2.2
But this doesn't work. I get documents that contain "adhesion" on any
page and "ring" on any page even thought adhesion and ring do not appear
together in any single page field.
I guess this is not too surprising but could someone please point me to
a way to define my query or schema to support this search functionality?
A typical document resembles:
<doc>
<field name="id">44</field>
<field name="extern_id">mdp.39015015394847</field>
<field name="page">B 1,371,014 1S45 Jab of Jzvo Citie8.</field>
<field name="page">I</field>
<field name="page">4b217L. //4/ //& -%/DEDICATED TO THOMAS D.
GILBERT AND TO THE OLD RESIDENTS ASSOCIATION OF THE GRAND RIVER VALLEY.
</field>
etc.
Thanks,
Phil
Re: Searching for two terms together in a multiValued TextField
Posted by Erick Erickson <er...@gmail.com>.
Scoring isn't that simple, but don't ask me details <G>.. This link might
be useful:
http://lucene.apache.org/java/docs/scoring.html
Erick
On Dec 6, 2007 2:15 PM, Phillip Farber <pf...@umich.edu> wrote:
> Hello Hoss,
>
> I appreciate your detailed response. I think I like your second
> alternative because I'd like to score whole books rather than pages in
> books. It seems to me that the more words one has to work with in a
> "document" the better the scoring would be for the entire book.
>
> Here's a question. Am I correct that (1) book "documents" where the
> terms appear farther apart than 10,000 words would not be returned in
> the result or (2) they would be in the result but scored lower than all
> the books where the terms were within 10,000 chars?
>
> Thanks,
>
> Phil
>
>
>
> Chris Hostetter wrote:
> > : that must appear together on a page. I have a multiValued TextField
> called
> > : "page" in a document with uniqueId called "id" that represents a OCR'd
> book.
> > : My default operator is AND. My default field is "page". My query is:
> > :
> > : q=adhesion+ring&fl=id,score&fq=id:(1+44)&version=2.2
> > :
> > : But this doesn't work. I get documents that contain "adhesion" on any
> page
> > : and "ring" on any page even thought adhesion and ring do not appear
> together
> > : in any single page field.
> >
> > one option to consider is changing the granularity of your "documents"
> so
> > that each "page" is indexed as a seperate document ... this is
> > particulararly handy if your ultimate goal is to tell people "the words
> > you searched for were all found on *this* page of *this* document.
> >
> > Alternately: take a look at the comments about "positionIncrementGap" in
> > the example schema. by setting that to some value bigger then the
> number
> > of words you expect to ever be on the same page, you can then do
> > "proxmity" queries where you look for the words closer together then
> that
> > gap size...
> >
> > q="adhesion+ring"~10000&fl=id,score&fq=id:(1+44)&version=2.2
> >
> > this will let you tell people "the words you searched for were all found
> > on *a* page of *this* document"
> >
> >
> > -Hoss
> >
>
Re: Searching for two terms together in a multiValued TextField
Posted by Phillip Farber <pf...@umich.edu>.
Hello Hoss,
I appreciate your detailed response. I think I like your second
alternative because I'd like to score whole books rather than pages in
books. It seems to me that the more words one has to work with in a
"document" the better the scoring would be for the entire book.
Here's a question. Am I correct that (1) book "documents" where the
terms appear farther apart than 10,000 words would not be returned in
the result or (2) they would be in the result but scored lower than all
the books where the terms were within 10,000 chars?
Thanks,
Phil
Chris Hostetter wrote:
> : that must appear together on a page. I have a multiValued TextField called
> : "page" in a document with uniqueId called "id" that represents a OCR'd book.
> : My default operator is AND. My default field is "page". My query is:
> :
> : q=adhesion+ring&fl=id,score&fq=id:(1+44)&version=2.2
> :
> : But this doesn't work. I get documents that contain "adhesion" on any page
> : and "ring" on any page even thought adhesion and ring do not appear together
> : in any single page field.
>
> one option to consider is changing the granularity of your "documents" so
> that each "page" is indexed as a seperate document ... this is
> particulararly handy if your ultimate goal is to tell people "the words
> you searched for were all found on *this* page of *this* document.
>
> Alternately: take a look at the comments about "positionIncrementGap" in
> the example schema. by setting that to some value bigger then the number
> of words you expect to ever be on the same page, you can then do
> "proxmity" queries where you look for the words closer together then that
> gap size...
>
> q="adhesion+ring"~10000&fl=id,score&fq=id:(1+44)&version=2.2
>
> this will let you tell people "the words you searched for were all found
> on *a* page of *this* document"
>
>
> -Hoss
>
Re: Searching for two terms together in a multiValued TextField
Posted by Chris Hostetter <ho...@fucit.org>.
: that must appear together on a page. I have a multiValued TextField called
: "page" in a document with uniqueId called "id" that represents a OCR'd book.
: My default operator is AND. My default field is "page". My query is:
:
: q=adhesion+ring&fl=id,score&fq=id:(1+44)&version=2.2
:
: But this doesn't work. I get documents that contain "adhesion" on any page
: and "ring" on any page even thought adhesion and ring do not appear together
: in any single page field.
one option to consider is changing the granularity of your "documents" so
that each "page" is indexed as a seperate document ... this is
particulararly handy if your ultimate goal is to tell people "the words
you searched for were all found on *this* page of *this* document.
Alternately: take a look at the comments about "positionIncrementGap" in
the example schema. by setting that to some value bigger then the number
of words you expect to ever be on the same page, you can then do
"proxmity" queries where you look for the words closer together then that
gap size...
q="adhesion+ring"~10000&fl=id,score&fq=id:(1+44)&version=2.2
this will let you tell people "the words you searched for were all found
on *a* page of *this* document"
-Hoss