You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Ali Rouhi <sr...@yahoo.com> on 2003/07/03 01:41:05 UTC
Query across multiple fields scenario not handled by "MultiFieldQueryParser"
Hi
I need to perform a search for an expression in
multiple fields "as if" they were one field. This is
best illustrated by a simple example.
Find expression (X AND Y) in fields (F, G).
I want this to translate to:
((X in F) OR (X in G)) AND ((Y in F) OR (Y in G))
In other words I want the query to return true *not
only* if
((X AND Y) in F) OR ((X AND Y) in G)
(1)
but *also* the following should give me a true value
((X in F) AND (Y in G)) OR ((X in G) AND (Y in F))
(2)
I believe that "MultiFieldQueryParser" just gives me
(1) and leaves out the "cross terms" in(2).(If you
want a practical example of why one would want to do
such a search I would be glad to provide one).
Of course I could construct the combination of (1) and
(2) manually, but the problem is that I want the
"general solution" with a general expression being
searched for in multiple fields in the manner of the
above example.
I have a feeling that this sort of a thing is best
done by writing a custom QueryParser.jj. We use Lucene
in production code and have great java expertise, but
little JavaCC expertise. Suggestions for solving the
problem at a higher level than the QueryParser are of
course also very welcome.
Many Thanks
Ali
__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Query across multiple fields scenario not handled by "MultiFieldQueryParser"
Posted by Ali Rouhi <sr...@yahoo.com>.
--- Ali Rouhi <sr...@yahoo.com> wrote:
>
> --- Victor Hadianto <vi...@nuix.com.au> wrote:
> > > I need to perform a search for an expression in
> > > multiple fields "as if" they were one field.
> >
> > Depending on your time requirment, you can as you
> > have suggested yourself
> > write your own QueryParser. However a simple hack
> > will be to have an "all"
> > field and simple reindex the data in F and G to
> this
> > "all" field.
> >
>
> We have thought of this - but we index an enormous
> amount of data (Terabytes distributed across many
> linux boxes) and would prefer to save the disk
> space.
> However we may indeed end up doing what you suggest.
Just a little clarification to the point I made
regarding your suggestion. We do need to preserve the
indexes under F and G and be able to search
independently on them. This is why having an extra ALL
field which contains contents of both F and G would
involve extra disc usages.
Ali
>
> > > little JavaCC expertise. Suggestions for solving
> > the
> > > problem at a higher level than the QueryParser
> are
> > of
> > > course also very welcome.
> >
> > Well this is definitely not "higher" level than
> > QueryParser :)
>
> This in itself is useful to know. I can now justify
> having to learn JavaCC to my boss:).
>
> Thanks
> Ali
>
> __________________________________
> Do you Yahoo!?
> SBC Yahoo! DSL - Now only $29.95 per month!
> http://sbc.yahoo.com
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
>
__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Query across multiple fields scenario not handled by "MultiFieldQueryParser"
Posted by Ali Rouhi <sr...@yahoo.com>.
--- Victor Hadianto <vi...@nuix.com.au> wrote:
> > I need to perform a search for an expression in
> > multiple fields "as if" they were one field.
>
> Depending on your time requirment, you can as you
> have suggested yourself
> write your own QueryParser. However a simple hack
> will be to have an "all"
> field and simple reindex the data in F and G to this
> "all" field.
>
We have thought of this - but we index an enormous
amount of data (Terabytes distributed across many
linux boxes) and would prefer to save the disk space.
However we may indeed end up doing what you suggest.
> > little JavaCC expertise. Suggestions for solving
> the
> > problem at a higher level than the QueryParser are
> of
> > course also very welcome.
>
> Well this is definitely not "higher" level than
> QueryParser :)
This in itself is useful to know. I can now justify
having to learn JavaCC to my boss:).
Thanks
Ali
__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Query across multiple fields scenario not handled by "MultiFieldQueryParser"
Posted by Victor Hadianto <vi...@nuix.com.au>.
> I need to perform a search for an expression in
> multiple fields "as if" they were one field.
Depending on your time requirment, you can as you have suggested yourself
write your own QueryParser. However a simple hack will be to have an "all"
field and simple reindex the data in F and G to this "all" field.
> little JavaCC expertise. Suggestions for solving the
> problem at a higher level than the QueryParser are of
> course also very welcome.
Well this is definitely not "higher" level than QueryParser :)
HTH
victor
On Thu, 3 Jul 2003 09:41 am, Ali Rouhi wrote:
> Hi
>
This is
> best illustrated by a simple example.
>
> Find expression (X AND Y) in fields (F, G).
>
> I want this to translate to:
>
> ((X in F) OR (X in G)) AND ((Y in F) OR (Y in G))
>
> In other words I want the query to return true *not
> only* if
>
> ((X AND Y) in F) OR ((X AND Y) in G)
> (1)
>
> but *also* the following should give me a true value
>
> ((X in F) AND (Y in G)) OR ((X in G) AND (Y in F))
> (2)
>
> I believe that "MultiFieldQueryParser" just gives me
> (1) and leaves out the "cross terms" in(2).(If you
> want a practical example of why one would want to do
> such a search I would be glad to provide one).
>
> Of course I could construct the combination of (1) and
> (2) manually, but the problem is that I want the
> "general solution" with a general expression being
> searched for in multiple fields in the manner of the
> above example.
>
> I have a feeling that this sort of a thing is best
> done by writing a custom QueryParser.jj. We use Lucene
> in production code and have great java expertise, but
>
> Many Thanks
> Ali
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Newbie : Search Result Null from Tomcat (NO exceptions , just null)
Posted by alvaro z <al...@yahoo.com>.
Im trying lucene-1.2.jar .
I index the files from console ( that generated the index dir with the index files (using dir txts that has lots of txt files ).
then I tried the search (with all the methods in the sample) . It works
But when i tried to made a search or a query by a JSP using Tomcat 3.2.3, it showed me null values . not even Exceptions about IndexSearcher or a query search. just null . Im using the same index dir generated at the console.
im wondering if the prob is the access to the txts dir (that is not public).
thanks for helping,
alvaro.
---------------------------------
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
Re: Query across multiple fields scenario not handled by "MultiFieldQueryParser" (figured out how to do it)
Posted by Ali Rouhi <sr...@yahoo.com>.
Otis
This is sort of situation we needed this functionality
(and I should have given this example before to make
the original post make more sense):
Where I work, we index (SMTP compliant) email using
lucene. We need to index recipients that a "regular
user" has search access to (To, CC, Apparently-To)
under a separate field from BCC recipients (if any).
If we indexed BCC under the same field as the other
recipients, users of our system could (by accident or
deliberately) find BCC recipients of an email by
searching in the recipient field - which defeats the
purpose of BCC (just because you received an email
doesn't mean you should find out who it was bcc'd to).
To special "audit users" we want to give permission to
search bcc fields. However in our UI we don't want to
use separate text fields (it's a web UI) for BCC
recipients. If we did this, the audit user would have
to do 4 separate searches to look for recipients:
Joe AND Fred
Namely (To stands for To,cc,apparently-to which are
indexed under the same field)
1. (Joe AND Fred) in To
2. (Joe AND Fred) in BCC
3. (Joe in To) and (Fred in BCC)
4. (Fred in BCC) and (Joe in To)
For more complicated search expressions this becomes
tedious very quickly.
So basically indexing under separate fields solves an
access control problem for us - but introduces this
other problem of doing the 4 searches above as one
search.
My implementation doesn't break existing QP
functionality (we still have to support the non audit
users and other fields - "from", "subject", "content"
etc still use the old QP functionality). However the
problem we had to solve here seems specialized - I
can't easily think of situations where users may want
this, but this may be lack of imagination on my part.
BTW, I should also mention that the solution is
general and does the kind of search in the example
across and array of fields (not just 2 as in example).
If you are still interested let me know and I'll send
you the modified QueryParser.jj.
Ali
p.s. I think BCC is evil and do not use it myself
ever:)
--- Otis Gospodnetic <ot...@yahoo.com>
wrote:
> Hello Ali,
>
> I'd be interested in the QueryParser.jj unified
> diff, especially if it
> preserves the existing QP functionality.
> I'm also curious where you use such queries. I
> think it may be
> confusing to visualize this to novice users, but it
> may be nice to have
> support for this in the QueryParser that ships with
> Lucene.
>
> Thanks,
> Otis
>
>
> --- Ali Rouhi <sr...@yahoo.com> wrote:
> > Just a note to anyone interested, I figured out
> how to
> > solve to problem I posed last week by adding the
> > functionality needed to QueryParser.jj. It was a
> > pretty specialized request to begin with - but
> anyone
> > interested - I'll be glad to let them know how it
> > works.
> >
> > Ali
> >
> > --- Ali Rouhi <sr...@yahoo.com> wrote:
> > > Hi
> > >
> > > I need to perform a search for an expression in
> > > multiple fields "as if" they were one field.
> This is
> > > best illustrated by a simple example.
> > >
> > > Find expression (X AND Y) in fields (F, G).
> > >
> > > I want this to translate to:
> > >
> > > ((X in F) OR (X in G)) AND ((Y in F) OR (Y in
> G))
> > >
> > > In other words I want the query to return true
> *not
> > > only* if
> > >
> > > ((X AND Y) in F) OR ((X AND Y) in G)
>
> > > (1)
> > >
> > > but *also* the following should give me a true
> value
> > >
> > > ((X in F) AND (Y in G)) OR ((X in G) AND (Y in
> F))
> > > (2)
> > >
> > > I believe that "MultiFieldQueryParser" just
> gives me
> > > (1) and leaves out the "cross terms" in(2).(If
> you
> > > want a practical example of why one would want
> to do
> > > such a search I would be glad to provide one).
> > >
> > > Of course I could construct the combination of
> (1)
> > > and
> > > (2) manually, but the problem is that I want the
> > > "general solution" with a general expression
> being
> > > searched for in multiple fields in the manner of
> the
> > > above example.
> > >
> > > I have a feeling that this sort of a thing is
> best
> > > done by writing a custom QueryParser.jj. We use
> > > Lucene
> > > in production code and have great java
> expertise,
> > > but
> > > little JavaCC expertise. Suggestions for solving
> the
> > > problem at a higher level than the QueryParser
> are
> > > of
> > > course also very welcome.
> > >
> > > Many Thanks
> > > Ali
> > >
> > >
> > >
> > > __________________________________
> > > Do you Yahoo!?
> > > SBC Yahoo! DSL - Now only $29.95 per month!
> > > http://sbc.yahoo.com
> > >
> > >
> >
>
---------------------------------------------------------------------
> > > To unsubscribe, e-mail:
> > > lucene-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail:
> > > lucene-user-help@jakarta.apache.org
> > >
> >
> >
> > __________________________________
> > Do you Yahoo!?
> > SBC Yahoo! DSL - Now only $29.95 per month!
> > http://sbc.yahoo.com
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
> >
>
>
> __________________________________
> Do you Yahoo!?
> SBC Yahoo! DSL - Now only $29.95 per month!
> http://sbc.yahoo.com
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
>
__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Query across multiple fields scenario not handled by "MultiFieldQueryParser" (figured out how to do it)
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hello Ali,
I'd be interested in the QueryParser.jj unified diff, especially if it
preserves the existing QP functionality.
I'm also curious where you use such queries. I think it may be
confusing to visualize this to novice users, but it may be nice to have
support for this in the QueryParser that ships with Lucene.
Thanks,
Otis
--- Ali Rouhi <sr...@yahoo.com> wrote:
> Just a note to anyone interested, I figured out how to
> solve to problem I posed last week by adding the
> functionality needed to QueryParser.jj. It was a
> pretty specialized request to begin with - but anyone
> interested - I'll be glad to let them know how it
> works.
>
> Ali
>
> --- Ali Rouhi <sr...@yahoo.com> wrote:
> > Hi
> >
> > I need to perform a search for an expression in
> > multiple fields "as if" they were one field. This is
> > best illustrated by a simple example.
> >
> > Find expression (X AND Y) in fields (F, G).
> >
> > I want this to translate to:
> >
> > ((X in F) OR (X in G)) AND ((Y in F) OR (Y in G))
> >
> > In other words I want the query to return true *not
> > only* if
> >
> > ((X AND Y) in F) OR ((X AND Y) in G)
> > (1)
> >
> > but *also* the following should give me a true value
> >
> > ((X in F) AND (Y in G)) OR ((X in G) AND (Y in F))
> > (2)
> >
> > I believe that "MultiFieldQueryParser" just gives me
> > (1) and leaves out the "cross terms" in(2).(If you
> > want a practical example of why one would want to do
> > such a search I would be glad to provide one).
> >
> > Of course I could construct the combination of (1)
> > and
> > (2) manually, but the problem is that I want the
> > "general solution" with a general expression being
> > searched for in multiple fields in the manner of the
> > above example.
> >
> > I have a feeling that this sort of a thing is best
> > done by writing a custom QueryParser.jj. We use
> > Lucene
> > in production code and have great java expertise,
> > but
> > little JavaCC expertise. Suggestions for solving the
> > problem at a higher level than the QueryParser are
> > of
> > course also very welcome.
> >
> > Many Thanks
> > Ali
> >
> >
> >
> > __________________________________
> > Do you Yahoo!?
> > SBC Yahoo! DSL - Now only $29.95 per month!
> > http://sbc.yahoo.com
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > lucene-user-help@jakarta.apache.org
> >
>
>
> __________________________________
> Do you Yahoo!?
> SBC Yahoo! DSL - Now only $29.95 per month!
> http://sbc.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Query across multiple fields scenario not handled by "MultiFieldQueryParser" (figured out how to do it)
Posted by Ali Rouhi <sr...@yahoo.com>.
Just a note to anyone interested, I figured out how to
solve to problem I posed last week by adding the
functionality needed to QueryParser.jj. It was a
pretty specialized request to begin with - but anyone
interested - I'll be glad to let them know how it
works.
Ali
--- Ali Rouhi <sr...@yahoo.com> wrote:
> Hi
>
> I need to perform a search for an expression in
> multiple fields "as if" they were one field. This is
> best illustrated by a simple example.
>
> Find expression (X AND Y) in fields (F, G).
>
> I want this to translate to:
>
> ((X in F) OR (X in G)) AND ((Y in F) OR (Y in G))
>
> In other words I want the query to return true *not
> only* if
>
> ((X AND Y) in F) OR ((X AND Y) in G)
> (1)
>
> but *also* the following should give me a true value
>
> ((X in F) AND (Y in G)) OR ((X in G) AND (Y in F))
> (2)
>
> I believe that "MultiFieldQueryParser" just gives me
> (1) and leaves out the "cross terms" in(2).(If you
> want a practical example of why one would want to do
> such a search I would be glad to provide one).
>
> Of course I could construct the combination of (1)
> and
> (2) manually, but the problem is that I want the
> "general solution" with a general expression being
> searched for in multiple fields in the manner of the
> above example.
>
> I have a feeling that this sort of a thing is best
> done by writing a custom QueryParser.jj. We use
> Lucene
> in production code and have great java expertise,
> but
> little JavaCC expertise. Suggestions for solving the
> problem at a higher level than the QueryParser are
> of
> course also very welcome.
>
> Many Thanks
> Ali
>
>
>
> __________________________________
> Do you Yahoo!?
> SBC Yahoo! DSL - Now only $29.95 per month!
> http://sbc.yahoo.com
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
>
__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org