You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Yonik Seeley (JIRA)" <ji...@apache.org> on 2013/10/27 17:22:42 UTC
[jira] [Comment Edited] (SOLR-5374) Support user configured
doc-centric versioning rules
[ https://issues.apache.org/jira/browse/SOLR-5374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13806370#comment-13806370 ]
Yonik Seeley edited comment on SOLR-5374 at 10/27/13 4:22 PM:
--------------------------------------------------------------
Hmmm, testConcurrentAdds fails even if I change the executor size to 1 thread... not sure why at this point.
edit: found it - a test bug where sometimes the winner was chosen for deletion when it wasn't supposed to be. Test now passes with single-thread executor and fails with more threads. Next up: optimistic concurrency to fix the concurrency issues.
Oh, another random note: we must have this processor run before the distributed update processor that handles internal solr versions. This is because that processor could end up dropping a winning update (i.e. a doc that has a higher external version but was assigned a lower internal version).
was (Author: yseeley@gmail.com):
Hmmm, testConcurrentAdds fails even if I change the executor size to 1 thread... not sure why at this point.
> Support user configured doc-centric versioning rules
> ----------------------------------------------------
>
> Key: SOLR-5374
> URL: https://issues.apache.org/jira/browse/SOLR-5374
> Project: Solr
> Issue Type: Improvement
> Reporter: Hoss Man
> Assignee: Hoss Man
> Attachments: SOLR-5374.patch, SOLR-5374.patch
>
>
> The existing optimistic concurrency features of Solr can be very handy for ensuring that you are only updating/replacing the version of the doc you think you are updating/replacing, w/o the risk of someone else adding/removing the doc in the mean time -- but I've recently encountered some situations where I really wanted to be able to let the client specify an arbitrary version, on a per document basis, (ie: generated by an external system, or perhaps a timestamp of when a file was last modified) and ensure that the corresponding document update was processed only if the "new" version is greater then the "old" version -- w/o needing to check exactly which version is currently in Solr. (ie: If a client wants to index version 101 of a doc, that update should fail if version 102 is already in the index, but succeed if the currently indexed version is 99 -- w/o the client needing to ask Solr what the current version)
> The idea Yonik brought up in SOLR-5298 (letting the client specify a {{\_new\_version\_}} that would be used by the existing optimistic concurrency code to control the assignment of the {{\_version\_}} field for documents) looked like a good direction to go -- but after digging into the way {{\_version\_}} is used internally I realized it requires a uniqueness constraint across all update commands, that would make it impossible to allow multiple independent documents to have the same {{\_version\_}}.
> So instead I've tackled the problem in a different way, using an UpdateProcessor that is configured with user defined field to track a "DocBasedVersion" and uses the RTG logic to figure out if the update is allowed.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org