You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Ishan Chattopadhyaya (Jira)" <ji...@apache.org> on 2021/10/23 20:37:00 UTC
[jira] [Commented] (SOLR-15705) Distribute a DeleteById to all
shards when using the CompositeId router with a router field defined and
field value is missing in request
[ https://issues.apache.org/jira/browse/SOLR-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17433363#comment-17433363 ]
Ishan Chattopadhyaya commented on SOLR-15705:
---------------------------------------------
I tried to reproduce some of these scenarios, and here's what I did:
h2. Scenario 1: Implicit routing
1. Created a collection with implicit router
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/admin/collections?action=CREATE&name=abc&numShards=2&router.name=implicit&shards=shard1%2Cshard2'
{code}
2. Added a document to shard1:
{code:java}
curl --request POST \
--url 'http://localhost:18983/solr/abc/update/json/docs?_route_=shard1' \
--header 'Content-Type: application/json' \
--data '{
"id": "usa1",
"title_t": "hello how are you"
}'
{code}
3. Added a second document to shard2 & performed a commit
{code:java}
curl --request POST \
--url 'http://localhost:18983/solr/abc/update/json/docs?_route_=shard2&commit=true' \
--header 'Content-Type: application/json' \
--data '{
"id": "usa1",
"title_t": "i am fine"
}'
{code}
4. Sent a DBI for usa1 to shard2
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/update?commit=true' \
--header 'Content-Type: application/xml' \
--data '<delete>
<id>usa1</id>
</delete>'
{code}
5. Searched for all documents:
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/select?q=*%3A*'
{code}
Here, shard1 was placed on another node, so these requests were handled by shard2. This document was successfully deleted.
5. Sent a DBI for india1 to shard2
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/update?commit=true' \
--header 'Content-Type: application/xml' \
--data '<delete>
<id>india1</id>
</delete>'
{code}
6. Searched for all documents:
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/select?q=*%3A*'
{code}
Here, we see that india1 was not deleted, even though the DBI query returned a 0 status (successful).
h2. Scenario 2: Composite routing
1. Create a collection
{code}
curl --request GET \
--url 'http://localhost:18983/solr/admin/collections?action=CREATE&name=com1&numShards=2'
{code}
2. Added two documents:
{code}
curl --request POST \
--url http://localhost:18983/solr/com1/update/json/docs \
--header 'Content-Type: application/json' \
--data '{
"id": "india:1",
"title_t": "hello how are you"
}'
curl --request POST \
--url http://localhost:18983/solr/com1/update/json/docs&commit=true \
--header 'Content-Type: application/json' \
--data '{
"id": "russia:1",
"title_t": "hi"
}'
{code}
3. Deleted usa:1
{code}
curl --request GET \
--url 'http://localhost:18983/solr/com1/update?commit=true' \
--header 'Content-Type: application/xml' \
--data '<delete>
<id>india:1</id>
</delete>'
{code}
4. Searched for all documents:
{code}
curl --request GET \
--url 'http://localhost:18983/solr/com1/select?q=*%3A*&shards=shard2'
{code}
Here, Solr was able to delete the document successfully.
I repeated the step 3 & 4 with "russia:1" and that document was also deleted successfully.
h2. Conclusion
1. I can see that in implicit router scenario, the bug exists. However, looking at the code [0], it seemed to me that the compositeIdRouter situation was addressed, but not the implicit router.
2. Judging by the scenario 2 experiment I did (and please point out what I may have missed), this problem doesn't exist for compositeId router.
[0] - https://github.com/apache/solr/pull/288/files#diff-dc18a65054da5e402a69909f9ee134b56dc496200d64dc8293e941d14d9961bdR324-R331
[~makosten], can you please suggest what I could be missing? Thanks!
> Distribute a DeleteById to all shards when using the CompositeId router with a router field defined and field value is missing in request
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-15705
> URL: https://issues.apache.org/jira/browse/SOLR-15705
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Reporter: Michael Kosten
> Assignee: David Eric Pugh
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When issuing a DeleteById command for a collection using the CompositeId router with a router field defined and the route value is missing from the request, the DeleteById fails silently, even if it happens to have been sent to the correct shard. Instead of failing silently, the request could be forwarded to all shard leaders and from there to all replicas. Another required change is that the deletion would need to be performed even though the route value is missing. The deletion would be a no-op on the shards where the document does not exist.
> One use case for this feature is when the routing key value is volatile and you need to guarantee no duplicates exist when updating a document if it happens to change shards. Another use case is if you want to delete a document without referencing a database to retrieve the route field value.
> A work-around is to use a DeleteByQuery. However, my testing has found that DeleteById performs significantly better, even when sent to all shards. Additionally, I've found that a heavy mixed load of DeleteByQuery commands and Add commands can lead to node failures when there are multiple replicas.
> Additional comments are in SOLR-6910, however, that JIRA is specific to the implicit router and I mistakenly added them there.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org