You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Hendrik Haddorp <he...@gmx.net> on 2017/07/21 16:20:13 UTC
atomic updates in conjunction with optimistic concurrency
Hi,
when I try to use an atomic update in conjunction with optimistic
concurrency Solr sometimes complains that the version I passed in does
not match. The version in my request however match to what is stored and
what the exception states as the actual version does not exist in the
collection at all. Strangely this does only happen sometimes but once it
happens for a collection it seems to stay like that. Any idea why that
might happen?
I'm using Solr 6.3 in Cloud mode with SolrJ.
regards,
Hendrik
Re: atomic updates in conjunction with optimistic concurrency
Posted by Susheel Kumar <su...@gmail.com>.
luceneMatchVersion is something that cause differences. You may want to
try 6.3 version to be on same page as Amrit's code.
On Fri, Jul 21, 2017 at 6:08 PM, Hendrik Haddorp <he...@gmx.net>
wrote:
> Thanks for trying to reproduce my issue.
>
> I'm using a Solr Cloud, my collection was quite small, only a 50-500
> documents, with one shard and a replication factor of 3. I updated all of
> the documents in one request. Beside that the flow is pretty much like
> yours.
>
> The goal of my code was to add a field to all documents that did not
> contain the field already, which should actually have been every document
> in the collection. So the code does a query and gets a few fields for each
> document. Then I use an atomic update to add two fields. In the end I send
> all fields to Solr in one request. The whole thing did then loop over a few
> hundred collections. In most cases it worked just fine but in some it
> failed. If so I could reproduce the issue on that collection constantly. In
> an earlier version I accidentally tried to add a field instead of setting
> it and that had caused an exception as the field only allowed a single
> value. At least in one case the version conflict later on happened on a
> collection that had this issue before. don't really think that causes it so
> just stating for the sake of completeness. I was not using a document
> centric version field but the normal _version_ field.
>
> Even though I'm running on Solr 6.3 my config still contains
> "<luceneMatchVersion>6.2.1</luceneMatchVersion>". Not sure if that might
> have an effect.
>
> The main code is this:
> SolrQuery query = new SolrQuery();
> query.setQuery(documentsMatchingQueryQ);
> query.setRows(Integer.MAX_VALUE);
> query.setFields(documentsMatchingQueryFL);
>
> QueryResponse queryResponse =
> solr.query(collectionName, query);
> if (queryResponse.getStatus() != 0) {
> LOGGER.error("request failed,
> status: ", queryResponse);
> throw new
> IllegalStateException("request failed");
> }
> LOGGER.info("{} | documents to
> process: {}", collectionName, queryResponse.getResults().getNumFound());
> if (queryResponse.getResults().getNumFound()
> > 0) {
> LOGGER.trace("{} | start
> processing", collectionName);
> List<SolrInputDocument> docs = queryResponse.getResults().stream()
> .map(documentGenerator::apply)
> .peek(doc ->
> LOGGER.trace("{} doc: {}", collectionName, doc))
> .collect(Collectors.toList());
> if (!reportOnly) {
> LOGGER.trace("{} | updating
> solr", collectionName);
> solr.add(collectionName, docs);
> solr.commit(collectionName, false, false, true);
>
> }
> LOGGER.trace("{} | processing
> done", collectionName);
> }
>
> The documentGenerator looks like this:
> originalDoc -> {
> SolrInputDocument doc = new SolrInputDocument();
>
> doc.addField("id", originalDoc.getFieldValue("id"));
> doc.addField("__docId__",
> Collections.singletonMap("set", originalDoc.getFieldValue("classification")
> + ":" + originalDoc.getFieldValue("id")));
> doc.addField("systemModified",
> Collections.singletonMap("set", originalDoc.getFieldValue("las
> tModified")));
> doc.addField("_version_",
> originalDoc.getFieldValue("_version_"));
>
> return doc;
>
> }
>
> On 21.07.2017 22:33, Amrit Sarkar wrote:
>
>> Hendrik,
>>
>> Ran a little test on 6.3, with infinite atomic updates with optimistic
>> concurrency,
>> cannot *reproduce*:
>>
>> List<SolrInputDocument> docs = new ArrayList<>();
>>
>>> SolrInputDocument document = new SolrInputDocument();
>>> document.addField("id", String.valueOf(1));
>>> document.addField("external_version_field_s",
>>> System.currentTimeMillis()); // normal update
>>> docs.add(document);
>>> UpdateRequest updateRequest = new UpdateRequest();
>>> updateRequest.add(docs);
>>> client.request(updateRequest, collection);
>>> updateRequest = new UpdateRequest();
>>> updateRequest.commit(client, collection);
>>>
>>> while (true) {
>>> QueryResponse response = client.query(new
>>> ModifiableSolrParams().add("q", "id:1"));
>>> System.out.println(response.getResults().get(0).get("_version_"));
>>> docs = new ArrayList<>();
>>> document = new SolrInputDocument();
>>> document.addField("id", String.valueOf(1));
>>> Map<String, String> map = new HashMap<>();
>>> map.put("set", createSentance(1)); // atomic map value
>>> document.addField("external_version_field_s", map);
>>> document.addField("_version_", response.getResults().get(0).g
>>> et("_version_"));
>>> docs.add(document);
>>> updateRequest = new UpdateRequest();
>>> updateRequest.add(docs);
>>> client.request(updateRequest, collection);
>>> updateRequest = new UpdateRequest();
>>> updateRequest.commit(client, collection);
>>> }
>>>
>>> Maybe you can let us know more details how the update been made?
>>>
>> Amrit Sarkar
>> Search Engineer
>> Lucidworks, Inc.
>> 415-589-9269
>> www.lucidworks.com
>> Twitter http://twitter.com/lucidworks
>> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>>
>> On Fri, Jul 21, 2017 at 10:36 PM, Hendrik Haddorp <
>> hendrik.haddorp@gmx.net>
>> wrote:
>>
>> Hi,
>>>
>>> I can't find anything about this in the Solr logs. On the caller side I
>>> have this:
>>> Error from server at http://xxxxx_shard1_replica2: version conflict for
>>> xxxxx expected=1573538179623944192 actual=1573546159565176832
>>> org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error
>>> from server at http://xxxxx_shard1_replica2: version conflict for xxxxx
>>> expected=1573538179623944192 actual=1573546159565176832
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpda
>>> te(CloudSolrClient.java:765)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.sendReques
>>> t(CloudSolrClient.java:1173)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWit
>>> hRetryOnStaleState(CloudSolrClient.java:1062)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.request(Cl
>>> oudSolrClient.java:1004)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest
>>> .java:149)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> ...
>>> Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrE
>>> xception:
>>> Error from server at http://xxxxx_shard1_replica2: version conflict for
>>> xxxxx expected=1573538179623944192 actual=1573546159565176832
>>> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMeth
>>> od(HttpSolrClient.java:593)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>>> pSolrClient.java:262)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>>> pSolrClient.java:251)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest
>>> (LBHttpSolrClient.java:435)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(L
>>> BHttpSolrClient.java:387)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$dir
>>> ectUpdate$0(CloudSolrClient.java:742) ~[solr-solrj-6.3.0.jar:6.3.0
>>> a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:52:43]
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> ~[?:1.8.0_131]
>>> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolE
>>> xecutor.lambda$execute$0(ExecutorUtil.java:229)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>> Executor.java:1142)
>>> ~[?:1.8.0_131]
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>> lExecutor.java:617)
>>> ~[?:1.8.0_131]
>>> at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
>>>
>>> The version "1573546159565176832" does not exist. It looks a bit like the
>>> update was first creating a new value and then checks against it.
>>>
>>> regards,
>>> Hendrik
>>>
>>>
>>> On 21.07.2017 18:21, Amrit Sarkar wrote:
>>>
>>> Hendrik,
>>>>
>>>> Can you list down the error snippet so that we can refer the code where
>>>> exactly that is happening.
>>>>
>>>>
>>>> Amrit Sarkar
>>>> Search Engineer
>>>> Lucidworks, Inc.
>>>> 415-589-9269
>>>> www.lucidworks.com
>>>> Twitter http://twitter.com/lucidworks
>>>> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>>>>
>>>> On Fri, Jul 21, 2017 at 9:50 PM, Hendrik Haddorp <
>>>> hendrik.haddorp@gmx.net
>>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>>> when I try to use an atomic update in conjunction with optimistic
>>>>> concurrency Solr sometimes complains that the version I passed in does
>>>>> not
>>>>> match. The version in my request however match to what is stored and
>>>>> what
>>>>> the exception states as the actual version does not exist in the
>>>>> collection
>>>>> at all. Strangely this does only happen sometimes but once it happens
>>>>> for a
>>>>> collection it seems to stay like that. Any idea why that might happen?
>>>>>
>>>>> I'm using Solr 6.3 in Cloud mode with SolrJ.
>>>>>
>>>>> regards,
>>>>> Hendrik
>>>>>
>>>>>
>>>>>
>
Re: atomic updates in conjunction with optimistic concurrency
Posted by Hendrik Haddorp <he...@gmx.net>.
Thanks for trying to reproduce my issue.
I'm using a Solr Cloud, my collection was quite small, only a 50-500
documents, with one shard and a replication factor of 3. I updated all
of the documents in one request. Beside that the flow is pretty much
like yours.
The goal of my code was to add a field to all documents that did not
contain the field already, which should actually have been every
document in the collection. So the code does a query and gets a few
fields for each document. Then I use an atomic update to add two fields.
In the end I send all fields to Solr in one request. The whole thing did
then loop over a few hundred collections. In most cases it worked just
fine but in some it failed. If so I could reproduce the issue on that
collection constantly. In an earlier version I accidentally tried to add
a field instead of setting it and that had caused an exception as the
field only allowed a single value. At least in one case the version
conflict later on happened on a collection that had this issue before.
don't really think that causes it so just stating for the sake of
completeness. I was not using a document centric version field but the
normal _version_ field.
Even though I'm running on Solr 6.3 my config still contains
"<luceneMatchVersion>6.2.1</luceneMatchVersion>". Not sure if that might
have an effect.
The main code is this:
SolrQuery query = new SolrQuery();
query.setQuery(documentsMatchingQueryQ);
query.setRows(Integer.MAX_VALUE);
query.setFields(documentsMatchingQueryFL);
QueryResponse queryResponse =
solr.query(collectionName, query);
if (queryResponse.getStatus() != 0) {
LOGGER.error("request failed,
status: ", queryResponse);
throw new
IllegalStateException("request failed");
}
LOGGER.info("{} | documents to
process: {}", collectionName, queryResponse.getResults().getNumFound());
if
(queryResponse.getResults().getNumFound() > 0) {
LOGGER.trace("{} | start
processing", collectionName);
List<SolrInputDocument> docs = queryResponse.getResults().stream()
.map(documentGenerator::apply)
.peek(doc ->
LOGGER.trace("{} doc: {}", collectionName, doc))
.collect(Collectors.toList());
if (!reportOnly) {
LOGGER.trace("{} | updating
solr", collectionName);
solr.add(collectionName, docs);
solr.commit(collectionName, false, false, true);
}
LOGGER.trace("{} | processing
done", collectionName);
}
The documentGenerator looks like this:
originalDoc -> {
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", originalDoc.getFieldValue("id"));
doc.addField("__docId__",
Collections.singletonMap("set",
originalDoc.getFieldValue("classification") + ":" +
originalDoc.getFieldValue("id")));
doc.addField("systemModified",
Collections.singletonMap("set", originalDoc.getFieldValue("lastModified")));
doc.addField("_version_",
originalDoc.getFieldValue("_version_"));
return doc;
}
On 21.07.2017 22:33, Amrit Sarkar wrote:
> Hendrik,
>
> Ran a little test on 6.3, with infinite atomic updates with optimistic
> concurrency,
> cannot *reproduce*:
>
> List<SolrInputDocument> docs = new ArrayList<>();
>> SolrInputDocument document = new SolrInputDocument();
>> document.addField("id", String.valueOf(1));
>> document.addField("external_version_field_s", System.currentTimeMillis()); // normal update
>> docs.add(document);
>> UpdateRequest updateRequest = new UpdateRequest();
>> updateRequest.add(docs);
>> client.request(updateRequest, collection);
>> updateRequest = new UpdateRequest();
>> updateRequest.commit(client, collection);
>>
>> while (true) {
>> QueryResponse response = client.query(new ModifiableSolrParams().add("q", "id:1"));
>> System.out.println(response.getResults().get(0).get("_version_"));
>> docs = new ArrayList<>();
>> document = new SolrInputDocument();
>> document.addField("id", String.valueOf(1));
>> Map<String, String> map = new HashMap<>();
>> map.put("set", createSentance(1)); // atomic map value
>> document.addField("external_version_field_s", map);
>> document.addField("_version_", response.getResults().get(0).get("_version_"));
>> docs.add(document);
>> updateRequest = new UpdateRequest();
>> updateRequest.add(docs);
>> client.request(updateRequest, collection);
>> updateRequest = new UpdateRequest();
>> updateRequest.commit(client, collection);
>> }
>>
>> Maybe you can let us know more details how the update been made?
> Amrit Sarkar
> Search Engineer
> Lucidworks, Inc.
> 415-589-9269
> www.lucidworks.com
> Twitter http://twitter.com/lucidworks
> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>
> On Fri, Jul 21, 2017 at 10:36 PM, Hendrik Haddorp <he...@gmx.net>
> wrote:
>
>> Hi,
>>
>> I can't find anything about this in the Solr logs. On the caller side I
>> have this:
>> Error from server at http://xxxxx_shard1_replica2: version conflict for
>> xxxxx expected=1573538179623944192 actual=1573546159565176832
>> org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error
>> from server at http://xxxxx_shard1_replica2: version conflict for xxxxx
>> expected=1573538179623944192 actual=1573546159565176832
>> at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:765)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1173)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWit
>> hRetryOnStaleState(CloudSolrClient.java:1062)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> ...
>> Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>> Error from server at http://xxxxx_shard1_replica2: version conflict for
>> xxxxx expected=1573538179623944192 actual=1573546159565176832
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:593)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$dir
>> ectUpdate$0(CloudSolrClient.java:742) ~[solr-solrj-6.3.0.jar:6.3.0
>> a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:52:43]
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> ~[?:1.8.0_131]
>> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolE
>> xecutor.lambda$execute$0(ExecutorUtil.java:229)
>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>> shalin - 2016-11-02 19:52:43]
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> ~[?:1.8.0_131]
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> ~[?:1.8.0_131]
>> at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
>>
>> The version "1573546159565176832" does not exist. It looks a bit like the
>> update was first creating a new value and then checks against it.
>>
>> regards,
>> Hendrik
>>
>>
>> On 21.07.2017 18:21, Amrit Sarkar wrote:
>>
>>> Hendrik,
>>>
>>> Can you list down the error snippet so that we can refer the code where
>>> exactly that is happening.
>>>
>>>
>>> Amrit Sarkar
>>> Search Engineer
>>> Lucidworks, Inc.
>>> 415-589-9269
>>> www.lucidworks.com
>>> Twitter http://twitter.com/lucidworks
>>> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>>>
>>> On Fri, Jul 21, 2017 at 9:50 PM, Hendrik Haddorp <hendrik.haddorp@gmx.net
>>> wrote:
>>>
>>> Hi,
>>>> when I try to use an atomic update in conjunction with optimistic
>>>> concurrency Solr sometimes complains that the version I passed in does
>>>> not
>>>> match. The version in my request however match to what is stored and what
>>>> the exception states as the actual version does not exist in the
>>>> collection
>>>> at all. Strangely this does only happen sometimes but once it happens
>>>> for a
>>>> collection it seems to stay like that. Any idea why that might happen?
>>>>
>>>> I'm using Solr 6.3 in Cloud mode with SolrJ.
>>>>
>>>> regards,
>>>> Hendrik
>>>>
>>>>
Re: atomic updates in conjunction with optimistic concurrency
Posted by Amrit Sarkar <sa...@gmail.com>.
Hendrik,
Ran a little test on 6.3, with infinite atomic updates with optimistic
concurrency,
cannot *reproduce*:
List<SolrInputDocument> docs = new ArrayList<>();
> SolrInputDocument document = new SolrInputDocument();
> document.addField("id", String.valueOf(1));
> document.addField("external_version_field_s", System.currentTimeMillis()); // normal update
> docs.add(document);
> UpdateRequest updateRequest = new UpdateRequest();
> updateRequest.add(docs);
> client.request(updateRequest, collection);
> updateRequest = new UpdateRequest();
> updateRequest.commit(client, collection);
>
> while (true) {
> QueryResponse response = client.query(new ModifiableSolrParams().add("q", "id:1"));
> System.out.println(response.getResults().get(0).get("_version_"));
> docs = new ArrayList<>();
> document = new SolrInputDocument();
> document.addField("id", String.valueOf(1));
> Map<String, String> map = new HashMap<>();
> map.put("set", createSentance(1)); // atomic map value
> document.addField("external_version_field_s", map);
> document.addField("_version_", response.getResults().get(0).get("_version_"));
> docs.add(document);
> updateRequest = new UpdateRequest();
> updateRequest.add(docs);
> client.request(updateRequest, collection);
> updateRequest = new UpdateRequest();
> updateRequest.commit(client, collection);
> }
>
> Maybe you can let us know more details how the update been made?
Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2
On Fri, Jul 21, 2017 at 10:36 PM, Hendrik Haddorp <he...@gmx.net>
wrote:
> Hi,
>
> I can't find anything about this in the Solr logs. On the caller side I
> have this:
> Error from server at http://xxxxx_shard1_replica2: version conflict for
> xxxxx expected=1573538179623944192 actual=1573546159565176832
> org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error
> from server at http://xxxxx_shard1_replica2: version conflict for xxxxx
> expected=1573538179623944192 actual=1573546159565176832
> at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:765)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1173)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWit
> hRetryOnStaleState(CloudSolrClient.java:1062)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> ...
> Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at http://xxxxx_shard1_replica2: version conflict for
> xxxxx expected=1573538179623944192 actual=1573546159565176832
> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:593)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$dir
> ectUpdate$0(CloudSolrClient.java:742) ~[solr-solrj-6.3.0.jar:6.3.0
> a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:52:43]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_131]
> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolE
> xecutor.lambda$execute$0(ExecutorUtil.java:229)
> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
> shalin - 2016-11-02 19:52:43]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[?:1.8.0_131]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[?:1.8.0_131]
> at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
>
> The version "1573546159565176832" does not exist. It looks a bit like the
> update was first creating a new value and then checks against it.
>
> regards,
> Hendrik
>
>
> On 21.07.2017 18:21, Amrit Sarkar wrote:
>
>> Hendrik,
>>
>> Can you list down the error snippet so that we can refer the code where
>> exactly that is happening.
>>
>>
>> Amrit Sarkar
>> Search Engineer
>> Lucidworks, Inc.
>> 415-589-9269
>> www.lucidworks.com
>> Twitter http://twitter.com/lucidworks
>> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>>
>> On Fri, Jul 21, 2017 at 9:50 PM, Hendrik Haddorp <hendrik.haddorp@gmx.net
>> >
>> wrote:
>>
>> Hi,
>>>
>>> when I try to use an atomic update in conjunction with optimistic
>>> concurrency Solr sometimes complains that the version I passed in does
>>> not
>>> match. The version in my request however match to what is stored and what
>>> the exception states as the actual version does not exist in the
>>> collection
>>> at all. Strangely this does only happen sometimes but once it happens
>>> for a
>>> collection it seems to stay like that. Any idea why that might happen?
>>>
>>> I'm using Solr 6.3 in Cloud mode with SolrJ.
>>>
>>> regards,
>>> Hendrik
>>>
>>>
>
Re: atomic updates in conjunction with optimistic concurrency
Posted by Hendrik Haddorp <he...@gmx.net>.
Hi,
I can't find anything about this in the Solr logs. On the caller side I
have this:
Error from server at http://xxxxx_shard1_replica2: version conflict for
xxxxx expected=1573538179623944192 actual=1573546159565176832
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error
from server at http://xxxxx_shard1_replica2: version conflict for xxxxx
expected=1573538179623944192 actual=1573546159565176832
at
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:765)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1173)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
...
Caused by:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://xxxxx_shard1_replica2: version conflict for
xxxxx expected=1573538179623944192 actual=1573546159565176832
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:593)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:742)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_131]
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[?:1.8.0_131]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
The version "1573546159565176832" does not exist. It looks a bit like
the update was first creating a new value and then checks against it.
regards,
Hendrik
On 21.07.2017 18:21, Amrit Sarkar wrote:
> Hendrik,
>
> Can you list down the error snippet so that we can refer the code where
> exactly that is happening.
>
>
> Amrit Sarkar
> Search Engineer
> Lucidworks, Inc.
> 415-589-9269
> www.lucidworks.com
> Twitter http://twitter.com/lucidworks
> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>
> On Fri, Jul 21, 2017 at 9:50 PM, Hendrik Haddorp <he...@gmx.net>
> wrote:
>
>> Hi,
>>
>> when I try to use an atomic update in conjunction with optimistic
>> concurrency Solr sometimes complains that the version I passed in does not
>> match. The version in my request however match to what is stored and what
>> the exception states as the actual version does not exist in the collection
>> at all. Strangely this does only happen sometimes but once it happens for a
>> collection it seems to stay like that. Any idea why that might happen?
>>
>> I'm using Solr 6.3 in Cloud mode with SolrJ.
>>
>> regards,
>> Hendrik
>>
Re: atomic updates in conjunction with optimistic concurrency
Posted by Amrit Sarkar <sa...@gmail.com>.
Hendrik,
Can you list down the error snippet so that we can refer the code where
exactly that is happening.
Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2
On Fri, Jul 21, 2017 at 9:50 PM, Hendrik Haddorp <he...@gmx.net>
wrote:
> Hi,
>
> when I try to use an atomic update in conjunction with optimistic
> concurrency Solr sometimes complains that the version I passed in does not
> match. The version in my request however match to what is stored and what
> the exception states as the actual version does not exist in the collection
> at all. Strangely this does only happen sometimes but once it happens for a
> collection it seems to stay like that. Any idea why that might happen?
>
> I'm using Solr 6.3 in Cloud mode with SolrJ.
>
> regards,
> Hendrik
>