You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jamie Johnson <je...@gmail.com> on 2012/07/05 22:29:43 UTC
deleteById commitWithin question
I am running off of a snapshot taken 5/3/2012 of solr 4.0 and am
noticing some issues around deleteById when a commitWithin parameter
is included using SolrJ, specifically commit isn't executed. If I
later just call commit on the solr instance I see the item is deleted
though. Is anyone aware if this should work in that snapshot?
Re: deleteById commitWithin question
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Thu, Jul 5, 2012 at 4:29 PM, Jamie Johnson <je...@gmail.com> wrote:
> I am running off of a snapshot taken 5/3/2012 of solr 4.0 and am
> noticing some issues around deleteById when a commitWithin parameter
> is included using SolrJ, specifically commit isn't executed. If I
> later just call commit on the solr instance I see the item is deleted
> though. Is anyone aware if this should work in that snapshot?
I thought I remembered something like this... but looking at the
commit log for DUH2, I don't see it.
/opt/code/lusolr4$ svn log
./solr/core/src/java/org/apache/solr/update/DirectUpdateHandler2.java
| less
------------------------------------------------------------------------
r1357332 | yonik | 2012-07-04 12:23:09 -0400 (Wed, 04 Jul 2012) | 1 line
log DBQ reordering events
------------------------------------------------------------------------
r1356858 | markrmiller | 2012-07-03 14:18:48 -0400 (Tue, 03 Jul 2012) | 1 line
SOLR-3587: After reloading a SolrCore, the original Analyzer is still
used rather than a new one
------------------------------------------------------------------------
r1356845 | yonik | 2012-07-03 13:47:56 -0400 (Tue, 03 Jul 2012) | 1 line
SOLR-3559: DBQ reorder support
------------------------------------------------------------------------
r1355088 | sarowe | 2012-06-28 13:51:38 -0400 (Thu, 28 Jun 2012) | 1 line
LUCENE-4172: clean up redundant throws clauses (merge from trunk)
------------------------------------------------------------------------
r1348984 | hossman | 2012-06-11 15:46:14 -0400 (Mon, 11 Jun 2012) | 1 line
LUCENE-3949: fix license headers to not be javadoc style comments
------------------------------------------------------------------------
r1343813 | rmuir | 2012-05-29 12:16:38 -0400 (Tue, 29 May 2012) | 1 line
create stable branch for 4.x releases
------------------------------------------------------------------------
r1328890 | yonik | 2012-04-22 11:01:55 -0400 (Sun, 22 Apr 2012) | 1 line
SOLR-3392: fix search leak when openSearcher=false
------------------------------------------------------------------------
r1328883 | yonik | 2012-04-22 09:58:00 -0400 (Sun, 22 Apr 2012) | 1 line
SOLR-3391: Make explicit commits cancel pending autocommits.
I'll try out trunk quick and see if it currently works.
-Yonik
http://lucidimagination.com
Re: deleteById commitWithin question
Posted by Jamie Johnson <je...@gmail.com>.
Thanks Yonik, glad that it is not an issue on Trunk. I'll see if our
group is interested in updating now to take advantage of this. In the
mean time we just issue a commit after we do deletes and that seems to
be a sufficient workaround.
On Thu, Jul 5, 2012 at 6:05 PM, Yonik Seeley <yo...@lucidimagination.com> wrote:
> On Thu, Jul 5, 2012 at 5:12 PM, Jamie Johnson <je...@gmail.com> wrote:
>> Ok, so some more context, hopefully this is useful.
>>
>> I didn't think this was a SolrCloud issue but it appears to be. I
>> have a simple 2 shard set up, I add 1 document which goes to shard 1.
>> I then issue a delete to shard 2. The delete gets there and I see the
>> commit eventually being run in the log, but it doesn't appear to be
>> being distributed to shard 1. If I manually call a server.commit()
>> the item is removed.
>
> OK, I just verified this for trunk / 4x. It appears to work fine now,
> but I'm not exactly sure why it may not with the snapshot you have
> (much has changed since).
>
> I followed the solr cloud wiki except that the path now needs to have
> collection1 in it, and I set the number of shards to 1 (with 2
> replicas). I indexed the example docs, then queried both shards to
> verify the doc existed on both. Then I deleted the doc through the
> non-leader via:
> curl "http://localhost:7574/solr/update?commitWithin=60000" -H
> 'Content-type:application/json' -d '{"delete": { "id":"SOLR1000"}}'
>
> This was to test both the forwarding to the leader and the fowarding
> from the leader to the replica.
> After 60 seconds, the document was gone on both replicas.
>
> aside: If you use deleteByQuery, be aware that full distributed
> indexing support was just finished yesterday by SOLR-3559
>
> -Yonik
> http://lucidimagination.com
Re: deleteById commitWithin question
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Thu, Jul 5, 2012 at 5:12 PM, Jamie Johnson <je...@gmail.com> wrote:
> Ok, so some more context, hopefully this is useful.
>
> I didn't think this was a SolrCloud issue but it appears to be. I
> have a simple 2 shard set up, I add 1 document which goes to shard 1.
> I then issue a delete to shard 2. The delete gets there and I see the
> commit eventually being run in the log, but it doesn't appear to be
> being distributed to shard 1. If I manually call a server.commit()
> the item is removed.
OK, I just verified this for trunk / 4x. It appears to work fine now,
but I'm not exactly sure why it may not with the snapshot you have
(much has changed since).
I followed the solr cloud wiki except that the path now needs to have
collection1 in it, and I set the number of shards to 1 (with 2
replicas). I indexed the example docs, then queried both shards to
verify the doc existed on both. Then I deleted the doc through the
non-leader via:
curl "http://localhost:7574/solr/update?commitWithin=60000" -H
'Content-type:application/json' -d '{"delete": { "id":"SOLR1000"}}'
This was to test both the forwarding to the leader and the fowarding
from the leader to the replica.
After 60 seconds, the document was gone on both replicas.
aside: If you use deleteByQuery, be aware that full distributed
indexing support was just finished yesterday by SOLR-3559
-Yonik
http://lucidimagination.com
Re: deleteById commitWithin question
Posted by Jamie Johnson <je...@gmail.com>.
Ok, so some more context, hopefully this is useful.
I didn't think this was a SolrCloud issue but it appears to be. I
have a simple 2 shard set up, I add 1 document which goes to shard 1.
I then issue a delete to shard 2. The delete gets there and I see the
commit eventually being run in the log, but it doesn't appear to be
being distributed to shard 1. If I manually call a server.commit()
the item is removed.
On Thu, Jul 5, 2012 at 4:56 PM, Jamie Johnson <je...@gmail.com> wrote:
> Thanks for the fast response. Yeah, it could be in either place
> either SolrJ or on the Server side.
>
> On Thu, Jul 5, 2012 at 4:47 PM, Yonik Seeley <yo...@lucidimagination.com> wrote:
>> On Thu, Jul 5, 2012 at 4:29 PM, Jamie Johnson <je...@gmail.com> wrote:
>>> I am running off of a snapshot taken 5/3/2012 of solr 4.0 and am
>>> noticing some issues around deleteById when a commitWithin parameter
>>> is included using SolrJ
>>
>> Oh wait... I just realized you were talking about SolrJ specifically -
>> so the issue may be there.
>>
>> -Yonik
>> http://lucidimagination.com
Re: deleteById commitWithin question
Posted by Jamie Johnson <je...@gmail.com>.
Thanks for the fast response. Yeah, it could be in either place
either SolrJ or on the Server side.
On Thu, Jul 5, 2012 at 4:47 PM, Yonik Seeley <yo...@lucidimagination.com> wrote:
> On Thu, Jul 5, 2012 at 4:29 PM, Jamie Johnson <je...@gmail.com> wrote:
>> I am running off of a snapshot taken 5/3/2012 of solr 4.0 and am
>> noticing some issues around deleteById when a commitWithin parameter
>> is included using SolrJ
>
> Oh wait... I just realized you were talking about SolrJ specifically -
> so the issue may be there.
>
> -Yonik
> http://lucidimagination.com
Re: deleteById commitWithin question
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Thu, Jul 5, 2012 at 4:29 PM, Jamie Johnson <je...@gmail.com> wrote:
> I am running off of a snapshot taken 5/3/2012 of solr 4.0 and am
> noticing some issues around deleteById when a commitWithin parameter
> is included using SolrJ
Oh wait... I just realized you were talking about SolrJ specifically -
so the issue may be there.
-Yonik
http://lucidimagination.com