You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Mark Miller <ma...@gmail.com> on 2010/01/10 19:28:56 UTC
update doc by query
Any reason we don't offer update doc by query along with term?
Its easy enough to implement in the same manner - is there some sort of
gotchya with this, or is it just because there has been no demand yet?
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: update doc by query
Posted by Sanne Grinovero <s....@sourcesense.com>.
Then I wouldn't need it and can still improve performance by using
periodic commits, nice!
thanks for explaining this,
Sanne
On Mon, Jan 11, 2010 at 10:57 AM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> On Sun, Jan 10, 2010 at 6:13 PM, Sanne Grinovero
> <s....@sourcesense.com> wrote:
>> Even if it's not strictly needed anymore, could it improve performance?
>
> I think there should be no real performance gains/losses one way or another.
>
> The current updateDocument call basically boils down to delete then add.
>
>> Right now I need to use commit() right after this dual operation to
>> make sure no reader is ever going to miss it
>
> You don't need to use commit() right after -- you can use commit any
> time later and both the del & add will be present.
>
>> but if it was atomic I
>> could have avoided the commit and just trust that "at some time later"
>> it will be auto-committed: exact moment would be out of my control,
>> but even so the view on index wouldn't have a chance to miss some
>> documents.
>
> Lucene no longer auto-commits -- your app completely controls when to
> commit, so, I think the atomic-ness is unecessary?
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
--
Sanne Grinovero
http://in.relation.to/Bloggers/Sanne
Sourcesense - making sense of Open Source: http://www.sourcesense.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: update doc by query
Posted by Michael McCandless <lu...@mikemccandless.com>.
Also, if the only reason why you're committing is so a reader can see
the changes (ie, you don't need so much "safety"), you should use
IndexWriter.getReader instead.
commit is really only needed for safety (ie known recovery points on
crash), or, for cases where the reader must be opened in a different
JVM than the writer.
Mike
On Mon, Jan 11, 2010 at 4:57 AM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> On Sun, Jan 10, 2010 at 6:13 PM, Sanne Grinovero
> <s....@sourcesense.com> wrote:
>> Even if it's not strictly needed anymore, could it improve performance?
>
> I think there should be no real performance gains/losses one way or another.
>
> The current updateDocument call basically boils down to delete then add.
>
>> Right now I need to use commit() right after this dual operation to
>> make sure no reader is ever going to miss it
>
> You don't need to use commit() right after -- you can use commit any
> time later and both the del & add will be present.
>
>> but if it was atomic I
>> could have avoided the commit and just trust that "at some time later"
>> it will be auto-committed: exact moment would be out of my control,
>> but even so the view on index wouldn't have a chance to miss some
>> documents.
>
> Lucene no longer auto-commits -- your app completely controls when to
> commit, so, I think the atomic-ness is unecessary?
>
> Mike
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: update doc by query
Posted by Michael McCandless <lu...@mikemccandless.com>.
On Sun, Jan 10, 2010 at 6:13 PM, Sanne Grinovero
<s....@sourcesense.com> wrote:
> Even if it's not strictly needed anymore, could it improve performance?
I think there should be no real performance gains/losses one way or another.
The current updateDocument call basically boils down to delete then add.
> Right now I need to use commit() right after this dual operation to
> make sure no reader is ever going to miss it
You don't need to use commit() right after -- you can use commit any
time later and both the del & add will be present.
> but if it was atomic I
> could have avoided the commit and just trust that "at some time later"
> it will be auto-committed: exact moment would be out of my control,
> but even so the view on index wouldn't have a chance to miss some
> documents.
Lucene no longer auto-commits -- your app completely controls when to
commit, so, I think the atomic-ness is unecessary?
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: update doc by query
Posted by Sanne Grinovero <s....@sourcesense.com>.
Even if it's not strictly needed anymore, could it improve performance?
Right now I need to use commit() right after this dual operation to
make sure no reader is ever going to miss it, but if it was atomic I
could have avoided the commit and just trust that "at some time later"
it will be auto-committed: exact moment would be out of my control,
but even so the view on index wouldn't have a chance to miss some
documents.
Regards,
Sanne
On Sun, Jan 10, 2010 at 10:04 PM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> I think there no particular demand...
>
> But: why not just separately delete by query, then add?
>
> Back when IW had autoCommit=true, it was compelling to have an atomic
> update, but now with only autoCommit=false, the app has full control
> over visibility to readers, so do we even need update-by-term anymore?
>
> Mike
>
> On Sun, Jan 10, 2010 at 2:13 PM, Sanne Grinovero
> <sa...@gmail.com> wrote:
>> If the demand is the problem:
>> I would really love that: in most scenarios a single term is not
>> enough to identify a Document: I need at least two so I use usually
>> remove-by-query first and then add again.
>> This sometimes needs some application level lock to make the changes consistent.
>>
>> Sanne
>>
>> 2010/1/10 Mark Miller <ma...@gmail.com>:
>>> Any reason we don't offer update doc by query along with term?
>>>
>>> Its easy enough to implement in the same manner - is there some sort of
>>> gotchya with this, or is it just because there has been no demand yet?
>>>
>>> --
>>> - Mark
>>>
>>> http://www.lucidimagination.com
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
--
Sanne Grinovero
http://in.relation.to/Bloggers/Sanne
Sourcesense - making sense of Open Source: http://www.sourcesense.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: update doc by query
Posted by Michael McCandless <lu...@mikemccandless.com>.
I think there no particular demand...
But: why not just separately delete by query, then add?
Back when IW had autoCommit=true, it was compelling to have an atomic
update, but now with only autoCommit=false, the app has full control
over visibility to readers, so do we even need update-by-term anymore?
Mike
On Sun, Jan 10, 2010 at 2:13 PM, Sanne Grinovero
<sa...@gmail.com> wrote:
> If the demand is the problem:
> I would really love that: in most scenarios a single term is not
> enough to identify a Document: I need at least two so I use usually
> remove-by-query first and then add again.
> This sometimes needs some application level lock to make the changes consistent.
>
> Sanne
>
> 2010/1/10 Mark Miller <ma...@gmail.com>:
>> Any reason we don't offer update doc by query along with term?
>>
>> Its easy enough to implement in the same manner - is there some sort of
>> gotchya with this, or is it just because there has been no demand yet?
>>
>> --
>> - Mark
>>
>> http://www.lucidimagination.com
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: update doc by query
Posted by Sanne Grinovero <sa...@gmail.com>.
If the demand is the problem:
I would really love that: in most scenarios a single term is not
enough to identify a Document: I need at least two so I use usually
remove-by-query first and then add again.
This sometimes needs some application level lock to make the changes consistent.
Sanne
2010/1/10 Mark Miller <ma...@gmail.com>:
> Any reason we don't offer update doc by query along with term?
>
> Its easy enough to implement in the same manner - is there some sort of
> gotchya with this, or is it just because there has been no demand yet?
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org