You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Jay Malaluan <ex...@yahoo.com> on 2008/12/17 06:23:22 UTC
Unique results in BooleanQuery
Hi,
Anyone knowledgeable on how to get unique hits using the BooleanQuery?
If I have 2 queries so the when the 1st query is processed then the 2nd
query will not anymore return the same results from the 1st query.
Regards,
Jay Malaluan
--
View this message in context: http://www.nabble.com/Unique-results-in-BooleanQuery-tp21047428p21047428.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Unique results in BooleanQuery
Posted by Jay Joel Malaluan <ex...@yahoo.com>.
Hi Chris,
I was just thinking that when the 1st query of q2 is run it will have its result.
Then the 2nd query of q2 will run and have its own result BUT it is now filtered that no same data from the 1st query is returned.
Results of the 1st and 2nd query have been appended.
Does the 2nd query of q2 have two process?
1. Run the query to get results.
2. For filtering
Regards,
Jay
________________________________
From: Chris Hostetter <ho...@fucit.org>
To: java-user@lucene.apache.org
Sent: Thursday, December 18, 2008 3:14:51 PM
Subject: Re: Unique results in BooleanQuery
: Let me expound more on the question. Will the q1 be run on the
: BooleanQuery q2 and append the results that are not equal to the result
: of the first query of q2?
i really have no idea what you mean by: "q1 be run on the BooleanQuery q2"
the query structure suggested will ensure that you only get results which
match q2 and do not match q1 (per your request "then the 2nd query will
not anymore return the same results from the 1st query")
: Do the first query as normal, then create a second BooleanQuery; add one clause for "colour:blue", Occur.MUST, and then add the query from the first search as a MUST_NOT.
:
: Query q1 = new TermQuery(new Term("fruit", "apple"));
: ..... run query, get back all apples ...
:
: BooleanQuery q2 = new BooleanQuery();
: q2.add(new TermQuery(new Term("colour", "blue")), Occur.MUST);
: q2.add(q1, Occur.MUST_NOT);
: ..... run query q2, get back all blue things that don't have apples ...
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Unique results in BooleanQuery
Posted by Chris Hostetter <ho...@fucit.org>.
: Let me expound more on the question. Will the q1 be run on the
: BooleanQuery q2 and append the results that are not equal to the result
: of the first query of q2?
i really have no idea what you mean by: "q1 be run on the BooleanQuery q2"
the query structure suggested will ensure that you only get results which
match q2 and do not match q1 (per your request "then the 2nd query will
not anymore return the same results from the 1st query")
: Do the first query as normal, then create a second BooleanQuery; add one clause for "colour:blue", Occur.MUST, and then add the query from the first search as a MUST_NOT.
:
: Query q1 = new TermQuery(new Term("fruit", "apple"));
: ..... run query, get back all apples ...
:
: BooleanQuery q2 = new BooleanQuery();
: q2.add(new TermQuery(new Term("colour", "blue")), Occur.MUST);
: q2.add(q1, Occur.MUST_NOT);
: ..... run query q2, get back all blue things that don't have apples ...
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: What are the best document edit options?
Posted by Chris Hostetter <ho...@fucit.org>.
: In-Reply-To: <13...@web45306.mail.sp1.yahoo.com>
: Subject: What are the best document edit options?
http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which thread
you replied to and your question is "hidden" in that thread and gets less
attention. It makes following discussions in the mailing list archives
particularly difficult.
See Also: http://en.wikipedia.org/wiki/Thread_hijacking
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: What are the best document edit options?
Posted by Erick Erickson <er...@gmail.com>.
I'll leave those details to the experts who are up to speed <G>.
On Wed, Dec 17, 2008 at 11:52 AM, Thomas J. Buhr <vi...@gmail.com>wrote:
> Erick,
>
> Thanks for the good news, my question was still lingering from months ago
> when I initially looked at an older Lucene.
>
> Now I need a bit more specific info, since much in my architecture rests on
> this ability to modify document fields dynamically. Where can I see how
> IndexWriter.updateDocument works without getting into Lucene all over again
> until this important issue is resolved? Is there a sample of its usage for
> updating specific fields in a given document? Is the update immediate or
> delayed?
>
> Thanks, Thom
>
>
>
> On 17-Dec-08, at 8:15 AM, Erick Erickson wrote:
>
> What version of Lucene are you using? The more recent ones have
>> IndexWriter.updateDocument......
>>
>> Best
>> Erick
>>
>> On Wed, Dec 17, 2008 at 2:20 AM, Thomas J. Buhr <visual.logic@gmail.com
>> >wrote:
>>
>> Hello Lucene,
>>>
>>> Looking at the document object it seems like each time I want to edit its
>>> contents I need to do the following:
>>>
>>> 1 - fetch the document
>>> 2 - dump its contents into a temp container
>>> 3 - update field values in the temp container
>>> 4 - create a new document
>>> 5 - transfer my updated field values back into the new document
>>> 6 - add document back into the index
>>>
>>> This seems like a long process, is it really necessary?
>>>
>>> An index structure does seem ideal for storing and searching a large set
>>> of
>>> symbolic strings in my app. From here I need to render these string chars
>>> as
>>> visible nodes on screen, this can be done with the new JavaFX API .
>>>
>>> JavaFX has a unique binding system that would also be great to use. Using
>>> binding I could setup two way (inverse) binding between the fields in a
>>> document and the rendered screen nodes. This way when there is a value
>>> change in the rendered node the document field values would auto update
>>> and
>>> also if there is a change in the document field value the node would auto
>>> update.
>>>
>>> This update senario via binding would be ideal. How can it be done?
>>>
>>> Information about the JavaFX binding mechanism is available at this link:
>>>
>>> http://java.sun.com/javafx/1/tutorials/core/dataBinding
>>>
>>>
>>> Hope this can work,
>>>
>>> Thom
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: What are the best document edit options?
Posted by "Thomas J. Buhr" <vi...@gmail.com>.
Steve,
Thanks for the helpful information, the addition of the new document
methods makes things much better.
One more question, is there JSON support in Lucene? JSON is more fat-
free compared to XML and would be preferred. Digester works well for
indexing XML but something along the same lines for JSON would be
even sweeter.
Cheers,
Thom
On 17-Dec-08, at 2:39 PM, Steven A Rowe wrote:
> Hi Thomas,
>
> On 12/17/2008 at 11:52 AM, Thomas J. Buhr wrote:
>> Where can I see how IndexWriter.updateDocument works without getting
>> into Lucene all over again until this important issue is resolved?
>> Is there a sample of its usage for updating specific fields in a
>> given document?
>
> The updateDocument() functionality was added to IndexWriter in
> Lucene v2.1.0 under the aegis of LUCENE-565 <http://
> issues.apache.org/jira/browse/LUCENE-565>, and the discussion there
> should help, especially the design overviews (more than one there -
> look for Ning Li's 2006-12-18 comment: <http://issues.apache.org/
> jira/browse/LUCENE-565?focusedCommentId=12459506#action_12459506>).
>
> Several of Lucene's test cases invoke updateDocument(), e.g.
> o.a.l.index.TestIndexWriterDelete.testOperationsOnDiskFull():
>
> <http://svn.apache.org/viewvc/lucene/java/tags/lucene_2_4_0/src/
> test/org/apache/lucene/index/TestIndexWriterDelete.java?view=markup>
>
>> Is the update immediate or delayed?
>
> My reading of the way this all works is that IndexWriter buffers
> deletes and only writes them out to disk on flush, which can be
> triggered by a commit. An IndexReader over this index will only
> see a single post-commit snapshot. So there will be a delay: the
> IndexWriter must perform a commit, and the IndexReader must be
> closed and then opened again (or just reopen()'d, as of Lucene
> v2.3.0).
>
> Steve
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: What are the best document edit options?
Posted by Steven A Rowe <sa...@syr.edu>.
Hi Thomas,
On 12/17/2008 at 11:52 AM, Thomas J. Buhr wrote:
> Where can I see how IndexWriter.updateDocument works without getting
> into Lucene all over again until this important issue is resolved?
> Is there a sample of its usage for updating specific fields in a
> given document?
The updateDocument() functionality was added to IndexWriter in Lucene v2.1.0 under the aegis of LUCENE-565 <http://issues.apache.org/jira/browse/LUCENE-565>, and the discussion there should help, especially the design overviews (more than one there - look for Ning Li's 2006-12-18 comment: <http://issues.apache.org/jira/browse/LUCENE-565?focusedCommentId=12459506#action_12459506>).
Several of Lucene's test cases invoke updateDocument(), e.g. o.a.l.index.TestIndexWriterDelete.testOperationsOnDiskFull():
<http://svn.apache.org/viewvc/lucene/java/tags/lucene_2_4_0/src/test/org/apache/lucene/index/TestIndexWriterDelete.java?view=markup>
> Is the update immediate or delayed?
My reading of the way this all works is that IndexWriter buffers deletes and only writes them out to disk on flush, which can be triggered by a commit. An IndexReader over this index will only see a single post-commit snapshot. So there will be a delay: the IndexWriter must perform a commit, and the IndexReader must be closed and then opened again (or just reopen()'d, as of Lucene v2.3.0).
Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: What are the best document edit options?
Posted by "Thomas J. Buhr" <vi...@gmail.com>.
Erick,
Thanks for the good news, my question was still lingering from months
ago when I initially looked at an older Lucene.
Now I need a bit more specific info, since much in my architecture
rests on this ability to modify document fields dynamically. Where
can I see how IndexWriter.updateDocument works without getting into
Lucene all over again until this important issue is resolved? Is
there a sample of its usage for updating specific fields in a given
document? Is the update immediate or delayed?
Thanks, Thom
On 17-Dec-08, at 8:15 AM, Erick Erickson wrote:
> What version of Lucene are you using? The more recent ones have
> IndexWriter.updateDocument......
>
> Best
> Erick
>
> On Wed, Dec 17, 2008 at 2:20 AM, Thomas J. Buhr
> <vi...@gmail.com>wrote:
>
>> Hello Lucene,
>>
>> Looking at the document object it seems like each time I want to
>> edit its
>> contents I need to do the following:
>>
>> 1 - fetch the document
>> 2 - dump its contents into a temp container
>> 3 - update field values in the temp container
>> 4 - create a new document
>> 5 - transfer my updated field values back into the new document
>> 6 - add document back into the index
>>
>> This seems like a long process, is it really necessary?
>>
>> An index structure does seem ideal for storing and searching a
>> large set of
>> symbolic strings in my app. From here I need to render these
>> string chars as
>> visible nodes on screen, this can be done with the new JavaFX API .
>>
>> JavaFX has a unique binding system that would also be great to
>> use. Using
>> binding I could setup two way (inverse) binding between the fields
>> in a
>> document and the rendered screen nodes. This way when there is a
>> value
>> change in the rendered node the document field values would auto
>> update and
>> also if there is a change in the document field value the node
>> would auto
>> update.
>>
>> This update senario via binding would be ideal. How can it be done?
>>
>> Information about the JavaFX binding mechanism is available at
>> this link:
>>
>> http://java.sun.com/javafx/1/tutorials/core/dataBinding
>>
>>
>> Hope this can work,
>>
>> Thom
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: What are the best document edit options?
Posted by Erick Erickson <er...@gmail.com>.
What version of Lucene are you using? The more recent ones have
IndexWriter.updateDocument......
Best
Erick
On Wed, Dec 17, 2008 at 2:20 AM, Thomas J. Buhr <vi...@gmail.com>wrote:
> Hello Lucene,
>
> Looking at the document object it seems like each time I want to edit its
> contents I need to do the following:
>
> 1 - fetch the document
> 2 - dump its contents into a temp container
> 3 - update field values in the temp container
> 4 - create a new document
> 5 - transfer my updated field values back into the new document
> 6 - add document back into the index
>
> This seems like a long process, is it really necessary?
>
> An index structure does seem ideal for storing and searching a large set of
> symbolic strings in my app. From here I need to render these string chars as
> visible nodes on screen, this can be done with the new JavaFX API .
>
> JavaFX has a unique binding system that would also be great to use. Using
> binding I could setup two way (inverse) binding between the fields in a
> document and the rendered screen nodes. This way when there is a value
> change in the rendered node the document field values would auto update and
> also if there is a change in the document field value the node would auto
> update.
>
> This update senario via binding would be ideal. How can it be done?
>
> Information about the JavaFX binding mechanism is available at this link:
>
> http://java.sun.com/javafx/1/tutorials/core/dataBinding
>
>
> Hope this can work,
>
> Thom
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
What are the best document edit options?
Posted by "Thomas J. Buhr" <vi...@gmail.com>.
Hello Lucene,
Looking at the document object it seems like each time I want to edit
its contents I need to do the following:
1 - fetch the document
2 - dump its contents into a temp container
3 - update field values in the temp container
4 - create a new document
5 - transfer my updated field values back into the new document
6 - add document back into the index
This seems like a long process, is it really necessary?
An index structure does seem ideal for storing and searching a large
set of symbolic strings in my app. From here I need to render these
string chars as visible nodes on screen, this can be done with the
new JavaFX API .
JavaFX has a unique binding system that would also be great to use.
Using binding I could setup two way (inverse) binding between the
fields in a document and the rendered screen nodes. This way when
there is a value change in the rendered node the document field
values would auto update and also if there is a change in the
document field value the node would auto update.
This update senario via binding would be ideal. How can it be done?
Information about the JavaFX binding mechanism is available at this
link:
http://java.sun.com/javafx/1/tutorials/core/dataBinding
Hope this can work,
Thom
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Unique results in BooleanQuery
Posted by Jay Joel Malaluan <ex...@yahoo.com>.
Let me expound more on the question. Will the q1 be run on the BooleanQuery q2 and append the results that are not equal to the result of the first query of q2?
________________________________
From: Jay Joel Malaluan <ex...@yahoo.com>
To: java-user@lucene.apache.org
Sent: Wednesday, December 17, 2008 2:42:06 PM
Subject: Re: Unique results in BooleanQuery
Hi Paul,
But will the q1 be run on the BooleanQuery q2 or q1 is just used for filtering?
Regards,
Jay Malaluan
________________________________
From: Paul Cowan <co...@aconex.com>
To: java-user@lucene.apache.org
Sent: Wednesday, December 17, 2008 1:37:15 PM
Subject: Re: Unique results in BooleanQuery
Hi Jay,
> Anyone knowledgeable on how to get unique hits using the BooleanQuery?
> If I have 2 queries so the when the 1st query is processed then the 2nd
> query will not anymore return the same results from the 1st query.
Do you mean you want to run two separate queries -- get all the results from query 1 first, then get the results which match query 2 but also DON'T match query 1?
e.g. search for "fruit:apple", get back x records
then search for "colour:blue", get back records with 'blue' as a colour but WITHOUT 'apple' as a fruit?
Do the first query as normal, then create a second BooleanQuery; add one clause for "colour:blue", Occur.MUST, and then add the query from the first search as a MUST_NOT.
Query q1 = new TermQuery(new Term("fruit", "apple"));
..... run query, get back all apples ...
BooleanQuery q2 = new BooleanQuery();
q2.add(new TermQuery(new Term("colour", "blue")), Occur.MUST);
q2.add(q1, Occur.MUST_NOT);
..... run query q2, get back all blue things that don't have apples ...
Cheers,
Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Unique results in BooleanQuery
Posted by Jay Joel Malaluan <ex...@yahoo.com>.
Hi Paul,
But will the q1 be run on the BooleanQuery q2 or q1 is just used for filtering?
Regards,
Jay Malaluan
________________________________
From: Paul Cowan <co...@aconex.com>
To: java-user@lucene.apache.org
Sent: Wednesday, December 17, 2008 1:37:15 PM
Subject: Re: Unique results in BooleanQuery
Hi Jay,
> Anyone knowledgeable on how to get unique hits using the BooleanQuery?
> If I have 2 queries so the when the 1st query is processed then the 2nd
> query will not anymore return the same results from the 1st query.
Do you mean you want to run two separate queries -- get all the results from query 1 first, then get the results which match query 2 but also DON'T match query 1?
e.g. search for "fruit:apple", get back x records
then search for "colour:blue", get back records with 'blue' as a colour but WITHOUT 'apple' as a fruit?
Do the first query as normal, then create a second BooleanQuery; add one clause for "colour:blue", Occur.MUST, and then add the query from the first search as a MUST_NOT.
Query q1 = new TermQuery(new Term("fruit", "apple"));
.... run query, get back all apples ...
BooleanQuery q2 = new BooleanQuery();
q2.add(new TermQuery(new Term("colour", "blue")), Occur.MUST);
q2.add(q1, Occur.MUST_NOT);
.... run query q2, get back all blue things that don't have apples ...
Cheers,
Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Unique results in BooleanQuery
Posted by Paul Cowan <co...@aconex.com>.
Hi Jay,
> Anyone knowledgeable on how to get unique hits using the BooleanQuery?
> If I have 2 queries so the when the 1st query is processed then the 2nd
> query will not anymore return the same results from the 1st query.
Do you mean you want to run two separate queries -- get all the results
from query 1 first, then get the results which match query 2 but also
DON'T match query 1?
e.g. search for "fruit:apple", get back x records
then search for "colour:blue", get back records with 'blue' as a colour
but WITHOUT 'apple' as a fruit?
Do the first query as normal, then create a second BooleanQuery; add one
clause for "colour:blue", Occur.MUST, and then add the query from the
first search as a MUST_NOT.
Query q1 = new TermQuery(new Term("fruit", "apple"));
... run query, get back all apples ...
BooleanQuery q2 = new BooleanQuery();
q2.add(new TermQuery(new Term("colour", "blue")), Occur.MUST);
q2.add(q1, Occur.MUST_NOT);
... run query q2, get back all blue things that don't have apples ...
Cheers,
Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Unique results in BooleanQuery
Posted by Erick Erickson <er...@gmail.com>.
You could also think about a filter. Just run q1 as a regular query.
Use one of the Collector methods to create a Filter. At the end,
invert the Filter and use it as a parameter for your second query.
Best
Erick
On Wed, Dec 17, 2008 at 12:23 AM, Jay Malaluan <ex...@yahoo.com>wrote:
>
> Hi,
>
> Anyone knowledgeable on how to get unique hits using the BooleanQuery?
> If I have 2 queries so the when the 1st query is processed then the 2nd
> query will not anymore return the same results from the 1st query.
>
>
> Regards,
> Jay Malaluan
> --
> View this message in context:
> http://www.nabble.com/Unique-results-in-BooleanQuery-tp21047428p21047428.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>