Lot of ORs in a query and De Morgan Law

[Emmanuel Espina <es...@gmail.com>]

I have queries with a big big amount of OR terms. The AND terms are much
more convenient to handle because they can be turned into several filter
queries and cached.

Thinking in innovative solutions I recalled the De Morgan Laws
http://en.wikipedia.org/wiki/De_Morgan's_laws of Boolean Algebras, and
considering that Set theory can be considered a generalized Boolean algebra
(with: intersection as . (dot),  union as +,  complement as negation,  full
set as 1 and empty set as 0) can be an innovative way of solving it

With regular filter queries, all the fq filters are merged into one (using
intersection of sets) and in the results of the query we only consider the
documents contained in that resulting set. If we had an "inverse filter
query" that merges all the filters into one (performing an intersection
among the filter sets in the regular way) and then considering in the
results only documents NOT contained in the resulting set, we could
implement multiple OR queries in a much more performant way by aplying De
Morgan law to the query.

What are your opinions on this?

Thanks
Emmanuel

Re: Lot of ORs in a query and De Morgan Law

[Chris Hostetter <ho...@fucit.org>]

: I have queries with a big big amount of OR terms. The AND terms are much
: more convenient to handle because they can be turned into several filter
: queries and cached.
: 
: Thinking in innovative solutions I recalled the De Morgan Laws
: http://en.wikipedia.org/wiki/De_Morgan's_laws of Boolean Algebras, and
	...
: documents contained in that resulting set. If we had an "inverse filter
: query" that merges all the filters into one (performing an intersection
: among the filter sets in the regular way) and then considering in the
: results only documents NOT contained in the resulting set, we could
: implement multiple OR queries in a much more performant way by aplying De
: Morgan law to the query.

I need a lot more caffiene before i could say for sure, but thinking 
through the truth tables i'm pretty sure what you are saying is true.  As 
long as your are planning ot use the individual clauses over and over 
again then caching them individually and computing the "negation of the 
intersection of the negations" and using that as a filter ... but i'm not 
sure why you would need to?

assuming this is implemented "server side" (either in solr, or in 
something else dealingwith lucene directly) it's already fairly easy to 
"union" DocSets ... so you could cache the individual clauses and then 
union them into one DocSet and use that as a filter.


-Hoss

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org