You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Saksham Gupta <sa...@indiamart.com.INVALID> on 2023/12/19 04:42:50 UTC

Deletion on Solr Failing due to _version_ conflict

Hi All,

I have written a small script in python to delete a product from solr cloud
collection on the basis of unique id, _route_ and _version_.

I have extracted the exact values of unique id, _route_ and _version_ from
solr index and used them to delete the product. But my script gives an
error which indicates that the _version_ I used is incorrect. The error
message looks like:

"msg":"Async exception during distributed update: Error from server at
http://solr_ip:port/solr/collection_name/: null\n\n\n\nrequest:
http://solr_ip:port/solr/collection_name/\nRemote error message: version
conflict for {unique id} expected=1784635392440402000
actual=1784635392440401920",

    "code":409}}

1. How could this be the case where actual _version_ is different from the
one visible on querying.


2. How to get the actual _version_ of a document?

3. Any other ways to delete the solr document using _version_?

Re: Deletion on Solr Failing due to _version_ conflict

Posted by Shawn Heisey <ap...@elyograg.org.INVALID>.
On 12/19/2023 23:38, Saksham Gupta wrote:
> I have extracted the_version_ field using query:
> *fq=uniqueid:{some_value}&fl=_version_*, but upon running filter query
> using this same version I extracted; fq=_version_:{extracted_version} it is
> unable to find the document.

This is the definition of the _version_ field in my index.  I believe 
this is how the field is defined in all Solr example configs:

   <field name="_version_" type="long" indexed="false" stored="false"/>

The 'indexed="false"' in that definition means that querying on that 
field is impossible.  I suspect that it is not indexed in the sample 
configs because querying is not needed for the field's function and not 
indexing it makes the index a little bit smaller.

You would think that having stored as false would mean it wouldn't be 
returned in search results ... but a field of that type has docValues 
defined by default, and docValues functionality provides a value for 
search results by default.

I do not know why you are running into a discrepancy on the _version_ 
field.  It seems very odd.

Thanks,
Shawn


Re: Deletion on Solr Failing due to _version_ conflict

Posted by Saksham Gupta <sa...@indiamart.com.INVALID>.
Thanks Shawn for your insights,

I had already considered the concerns shared by you, still couldn't find a
way to use _version_ while deleting or even querying a document.

Is there anything else running that might update the document between
the query request and the delete request?
No, there was no indexing running. I had even committed all the changes
before checking the version of documents. So, I am pretty sure there was no
chance the document was updated b/w the time I queried and deleted the
document.

Do you possibly have the same uniqueKey value in more than one shard?
No, we do not documents with same unique key on more than one shard. I
verified that the document I was trying to delete was on single shard (by
routing request on each shard one by one; *eg. _route_:shard1*; and
checking if document is present on any) before trying to delete the
document.

Just do any query that would match that document and the _version_ field
should be in the search results.  If you have documents with the same
uniqueKey value in multiple shards, then which document gets returned
cannot be predicted.  Solr will eliminate duplicates when returning results
so there is only one document for each uniqueKey field value.
I have extracted the _version_ field using query:
*fq=uniqueid:{some_value}&fl=_version_*, but upon running filter query
using this same version I extracted; fq=_version_:{extracted_version} it is
unable to find the document.

When using the implicit router, if you simply use deleteById and give it
the value in the uniqueKey field, I am reasonably certain that the delete
request is sent to all shards so it won't matter what shard it's in or what
_version_ it has.
Yes, we can delete documents without _version_, but I am still curious as
to why is delete request with _version_ is failing.

On Tue, Dec 19, 2023 at 10:05 PM Shawn Heisey <ap...@elyograg.org.invalid>
wrote:

> On 12/18/23 21:42, Saksham Gupta wrote:
> > I have written a small script in python to delete a product from solr
> cloud
> > collection on the basis of unique id, _route_ and _version_.
> >
> > I have extracted the exact values of unique id, _route_ and _version_
> from
> > solr index and used them to delete the product. But my script gives an
> > error which indicates that the _version_ I used is incorrect. The error
> > message looks like:
> >
> > "msg":"Async exception during distributed update: Error from server at
> > http://solr_ip:port/solr/collection_name/: null\n\n\n\nrequest:
> > http://solr_ip:port/solr/collection_name/\nRemote error message: version
> > conflict for {unique id} expected=1784635392440402000
> > actual=1784635392440401920",
> >
> >      "code":409}}
> >
> > 1. How could this be the case where actual _version_ is different from
> the
> > one visible on querying.
>
> Is there anything else running that might update the document between
> the query request and the delete request?
>
> Do you possibly have the same uniqueKey value in more than one shard?
>
> If either of those conditions is true, then automatically generated
> fields like _version_ are subject to change.
>
> > 2. How to get the actual _version_ of a document?
>
> Just do any query that would match that document and the _version_ field
> should be in the search results.  If you have documents with the same
> uniqueKey value in multiple shards, then which document gets returned
> cannot be predicted.  Solr will eliminate duplicates when returning
> results so there is only one document for each uniqueKey field value.
>
> > 3. Any other ways to delete the solr document using _version_?
>
> When using the implicit router, if you simply use deleteById and give it
> the value in the uniqueKey field, I am reasonably certain that the
> delete request is sent to all shards so it won't matter what shard it's
> in or what _version_ it has.
>
> Thanks,
> Shawn
>
>

Re: Deletion on Solr Failing due to _version_ conflict

Posted by Shawn Heisey <ap...@elyograg.org.INVALID>.
On 12/18/23 21:42, Saksham Gupta wrote:
> I have written a small script in python to delete a product from solr cloud
> collection on the basis of unique id, _route_ and _version_.
> 
> I have extracted the exact values of unique id, _route_ and _version_ from
> solr index and used them to delete the product. But my script gives an
> error which indicates that the _version_ I used is incorrect. The error
> message looks like:
> 
> "msg":"Async exception during distributed update: Error from server at
> http://solr_ip:port/solr/collection_name/: null\n\n\n\nrequest:
> http://solr_ip:port/solr/collection_name/\nRemote error message: version
> conflict for {unique id} expected=1784635392440402000
> actual=1784635392440401920",
> 
>      "code":409}}
> 
> 1. How could this be the case where actual _version_ is different from the
> one visible on querying.

Is there anything else running that might update the document between 
the query request and the delete request?

Do you possibly have the same uniqueKey value in more than one shard?

If either of those conditions is true, then automatically generated 
fields like _version_ are subject to change.

> 2. How to get the actual _version_ of a document?

Just do any query that would match that document and the _version_ field 
should be in the search results.  If you have documents with the same 
uniqueKey value in multiple shards, then which document gets returned 
cannot be predicted.  Solr will eliminate duplicates when returning 
results so there is only one document for each uniqueKey field value.

> 3. Any other ways to delete the solr document using _version_?

When using the implicit router, if you simply use deleteById and give it 
the value in the uniqueKey field, I am reasonably certain that the 
delete request is sent to all shards so it won't matter what shard it's 
in or what _version_ it has.

Thanks,
Shawn


Re: Deletion on Solr Failing due to _version_ conflict

Posted by Saksham Gupta <sa...@indiamart.com.INVALID>.
Hi All,
Pinging again for some assistance.

On Tue, Dec 19, 2023 at 10:12 AM Saksham Gupta <sa...@indiamart.com>
wrote:

> Hi All,
>
> I have written a small script in python to delete a product from solr
> cloud collection on the basis of unique id, _route_ and _version_.
>
> I have extracted the exact values of unique id, _route_ and _version_ from
> solr index and used them to delete the product. But my script gives an
> error which indicates that the _version_ I used is incorrect. The error
> message looks like:
>
> "msg":"Async exception during distributed update: Error from server at
> http://solr_ip:port/solr/collection_name/: null\n\n\n\nrequest:
> http://solr_ip:port/solr/collection_name/\nRemote error message: version
> conflict for {unique id} expected=1784635392440402000
> actual=1784635392440401920",
>
>     "code":409}}
>
> 1. How could this be the case where actual _version_ is different from the
> one visible on querying.
>
>
> 2. How to get the actual _version_ of a document?
>
> 3. Any other ways to delete the solr document using _version_?
>