Delete by query discrepancy

[Mat Brown <ma...@patch.com>]

Hi all,

Trying to debug a very sneaky bug in a small Solr extension that I
wrote, and I've come across an odd situation. Here's what my test
suite does:

deleteByQuery("*:*");
// add some documents
commit();
// test the search

This works fine. The test suite that exposed the error (which is
actually for a Ruby client library I maintain) was doing almost the
exact same thing, with one exception - the deleteByQuery() passed the
query "type:[* TO *]" instead of "*:*" (in an attempt to isolate the
error, I made sure that the input document and search parameters were
identical between the two test suites).

In the schema, the "type" field has at least one value for every
document (in practice it has more than one for all the documents in
this test suite). Changing the test setup code to pass "type:[* TO *]"
to deleteByQuery() causes it to fail.

So I'm a bit confused - wouldn't deleteByQuery("type:[* TO *]") have
the same effect as deleteByQuery("*:*"), assuming every document has a
value for the "type" field? Or is there something subtler going on in
the internals - perhaps optimizing the "*:*" deleteByQuery() to just
tear down the whole index and build a new one from scratch? Something
that might have some subtle side effect? Now that I'm finally able to
reproduce the error in my extension's test suite, I can start actually
figuring out what's causing it, but I was surprised to find out that
the deleteByQuery() query is what makes the difference between passing
and failing.

Any insight much appreciated!

Mat Brown

Re: Delete by query discrepancy

[Mat Brown <ma...@patch.com>]

Cool, thanks - just wanted to make sure I'm not insane. Makes sense
that there would be a difference if the index is built fresh in that
case.

On Tue, Feb 16, 2010 at 11:59, Mark Miller <ma...@gmail.com> wrote:
> Mat Brown wrote:
>> Hi all,
>>
>> Trying to debug a very sneaky bug in a small Solr extension that I
>> wrote, and I've come across an odd situation. Here's what my test
>> suite does:
>>
>> deleteByQuery("*:*");
>> // add some documents
>> commit();
>> // test the search
>>
>> This works fine. The test suite that exposed the error (which is
>> actually for a Ruby client library I maintain) was doing almost the
>> exact same thing, with one exception - the deleteByQuery() passed the
>> query "type:[* TO *]" instead of "*:*" (in an attempt to isolate the
>> error, I made sure that the input document and search parameters were
>> identical between the two test suites).
>>
>> In the schema, the "type" field has at least one value for every
>> document (in practice it has more than one for all the documents in
>> this test suite). Changing the test setup code to pass "type:[* TO *]"
>> to deleteByQuery() causes it to fail.
>>
>> So I'm a bit confused - wouldn't deleteByQuery("type:[* TO *]") have
>> the same effect as deleteByQuery("*:*"), assuming every document has a
>> value for the "type" field? Or is there something subtler going on in
>> the internals - perhaps optimizing the "*:*" deleteByQuery() to just
>> tear down the whole index and build a new one from scratch? Something
>> that might have some subtle side effect? Now that I'm finally able to
>> reproduce the error in my extension's test suite, I can start actually
>> figuring out what's causing it, but I was surprised to find out that
>> the deleteByQuery() query is what makes the difference between passing
>> and failing.
>>
>> Any insight much appreciated!
>>
>> Mat Brown
>>
> Not sure why the tests would would be affected, but yes, Solr detects a
> delete of *:* and just creates a new
> index instead of deleting every document.
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>

Re: Delete by query discrepancy

[Mark Miller <ma...@gmail.com>]

Mat Brown wrote:
> Hi all,
>
> Trying to debug a very sneaky bug in a small Solr extension that I
> wrote, and I've come across an odd situation. Here's what my test
> suite does:
>
> deleteByQuery("*:*");
> // add some documents
> commit();
> // test the search
>
> This works fine. The test suite that exposed the error (which is
> actually for a Ruby client library I maintain) was doing almost the
> exact same thing, with one exception - the deleteByQuery() passed the
> query "type:[* TO *]" instead of "*:*" (in an attempt to isolate the
> error, I made sure that the input document and search parameters were
> identical between the two test suites).
>
> In the schema, the "type" field has at least one value for every
> document (in practice it has more than one for all the documents in
> this test suite). Changing the test setup code to pass "type:[* TO *]"
> to deleteByQuery() causes it to fail.
>
> So I'm a bit confused - wouldn't deleteByQuery("type:[* TO *]") have
> the same effect as deleteByQuery("*:*"), assuming every document has a
> value for the "type" field? Or is there something subtler going on in
> the internals - perhaps optimizing the "*:*" deleteByQuery() to just
> tear down the whole index and build a new one from scratch? Something
> that might have some subtle side effect? Now that I'm finally able to
> reproduce the error in my extension's test suite, I can start actually
> figuring out what's causing it, but I was surprised to find out that
> the deleteByQuery() query is what makes the difference between passing
> and failing.
>
> Any insight much appreciated!
>
> Mat Brown
>   
Not sure why the tests would would be affected, but yes, Solr detects a
delete of *:* and just creates a new
index instead of deleting every document.

-- 
- Mark

http://www.lucidimagination.com