You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by "Hoss Man (Confluence)" <co...@apache.org> on 2013/11/19 20:17:00 UTC
[CONF] Apache Solr Reference Guide > Updating Parts of Documents
Space: Apache Solr Reference Guide (https://cwiki.apache.org/confluence/display/solr)
Page: Updating Parts of Documents (https://cwiki.apache.org/confluence/display/solr/Updating+Parts+of+Documents)
Change Comment:
---------------------------------------------------------------------
SOLR-5374
Edited by Hoss Man:
---------------------------------------------------------------------
Once you have indexed the content you need in your Solr index, you will want to start thinking about your strategy for dealing with changes to those documents. Solr supports two approaches to updating documents that have only partially changed.
The first is _atomic updates_. This approach allows changing only one or more fields of a document without having to re-index the entire document.
The second approach is known as _optimistic concurrency_ or _optimistic locking_. It is a feature of many NoSQL databases, and allows conditional updating a document based on it's version. This approach includes semantics and rules for how to deal with version matches or mis-matches.
Atomic Updates and Optimistic Concurrency may be used as independent strategies for managing changes to documents, or they may be combined: you can use optimistic concurrency to conditionally apply an atomic update.
h2. Atomic Updates
Solr supports several modifiers that atomically update values of a document. This allows updating only specific fields, which can help speed indexing processes in an environment where speed of index additions is critical to the application.
To use atomic updates, add a modifier to the field that needs to be updated. The content can be updated, added to, or incrementally increased if a number.
|| Modifier || Usage ||
| set | Set or replace a particular value, or remove the value if 'null' is specified as the new value. |
| add | Adds an additional value to a list. |
| inc | Increments a numeric value by a specific amount. |
{note}All original source fields must be stored for field modifiers to work correctly, which is the Solr default.{note}
For example:
{code:language=none|borderStyle=solid|borderColor=#666666}
{"id":"mydoc", "f1"{"set":10}, "f2"{"add":20}}
{code}
This example results in field {{f1}} being set to "10", and field {{f2}} having an additional value of "20" added. All other existing fields from the original document remain unchanged.
h2. Optimistic Concurrency
Optimistic Concurrency is a feature of Solr that can be used by client applications which update/replace documents to ensure that the document they are replacing/updating has not been concurrently modified by another client application. This feature works by requiring a {{\_version_}} field on all documents in the index, and comparing that to a {{\_version_}} specified as part of the update command. By default, Solr's {{schema.xml}} includes a {{\_version_}} field, and this field is automatically added to each new document.
In general, using optimistic concurrency involves the following work flow:
# A client reads a document. In Solr, one might retrieve the document with the {{/get}} handler to be sure to have the latest version.
# A client changes the document locally.
# The client resubmits the changed document to Solr, for example, perhaps with the {{/update}} handler.
# If there is a version conflict (HTTP error code 409), the client starts the process over.
When the client resubmits a changed document to Solr, the {{\_version_}} can be included with the update to invoke optimistic concurrency control. Specific semantics are used to define when the document should be updated or when to report a conflict.
* If the content in the {{\_version_}} field is greater than '1' (i.e., '12345'), then the {{\_version_}} in the document must match the {{\_version_}} in the index.
* If the content in the {{\_version_}} field is equal to '1', then the document must simply exist. In this case, no version matching occurs, but if the document does not exist, the updates will be rejected.
* If the content in the {{\_version_}} field is less than '0' (i.e., '-1'), then the document must *not* exist. In this case, no version matching occurs, but if the document exists, the updates will be rejected.
* If the content in the {{\_version_}} field is equal to '0', then it doesn't matter if the versions match or if the document exists or not. If it exists, it will be overwritten; if it does not exist, it will be added.
If the document being updated does not include the {{\_version_}} field, and atomic updates are not being used, the document will be treated by normal Solr rules, which is usually to discard it
For more information, please also see [Yonik Seeley's presentation on NoSQL features in Solr 4|https://www.youtube.com/watch?v=WYVM6Wz-XTw] from Apache Lucene EuroCon 2012.
h2. Document Centric Versioning Constraints
Optimistic Concurrency is extremely powerful, and works very efficiently because it uses an internally assigned, globally unique values for the {{\_version_}} field. However, In some situations users may want to configure their own document specific version field, where the version values are assigned on a per-document basis by an external system, and have Solr reject updates that attempt to replace a document with an "older" version. In situations like this the {{[DocBasedVersionConstraintsProcessorFactory|https://lucene.apache.org/solr/api/solr-core/org/apache/solr/update/processor/DocBasedVersionConstraintsProcessorFactory.html]}} can be useful.
The basic usage of {{DocBasedVersionConstraintsProcessorFactory}} is to configure it in {{solrconfig.xml}} as part of the [UpdateRequestProcessorChain|http://wiki.apache.org/solr/UpdateRequestProcessor] and specify the name of the {{versionField}} in your schema that should be checked when validating updates:
{code:xml|borderStyle=solid|borderColor=#666666}
<processor class="solr.DocBasedVersionConstraintsProcessorFactory">
<str name="versionField">my_version_l</str>
</processor>
{code}
Once configured, this update processor will reject (HTTP error code 409) any attempt to update an existing document where the value of the {{my_version_l}} field in the "new" document is not greater then the value of that field in the existing document.
{{DocBasedVersionConstraintsProcessorFactory}} supports two additional configuration params which are optional:
* {{ignoreOldUpdates}} - A boolean option which defaults to {{false}}. If set to {{true}} then instead of rejecting updates where the {{versionField}} is too low, the update will be silently ignored (and return a status 200 to the client).
* {{deleteVersionParam}} - A String parameter that can be specified to indicate that this processor should also inspect Delete By Id commands. The value of this configuration option should be the name of a request parameter that the processor will now consider mandatory for all attempts to Delete By Id, and must be be used by clients to specify a value for the {{versionField}} which is greater then the existing value of the document to be deleted. When using this request param, any Delete By Id command with a high enough document version number to succeed will be internally converted into an Add Document command that replaces the existing document with a new one which is empty except for the Unique Key and {{versionField}} to keeping a record of the deleted version so future Add Document commands will fail if their "new" version is not high enough.
Please consult the [processor javadocs|https://lucene.apache.org/solr/api/solr-core/org/apache/solr/update/processor/DocBasedVersionConstraintsProcessorFactory.html] and [test configs|https://svn.apache.org/viewvc/lucene/dev/trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-externalversionconstraint.xml?view=markup] for additional information and example usages.
{scrollbar}
Stop watching space: https://cwiki.apache.org/confluence/users/removespacenotification.action?spaceKey=solr
Change email notification preferences: https://cwiki.apache.org/confluence/users/editmyemailsettings.action