[jira] [Commented] (SOLR-5850) Race condition in ConcurrentUpdateSolrServer

["Vijay Sekhri (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14566209#comment-14566209 ] 

Vijay Sekhri commented on SOLR-5850:
------------------------------------

One more observation here is if we increase the poll time to 10 second and also add a rertry logic (to retry 3 times before finally giving up)  that the leaders some times ends up having around 4k live threads and results in JVM error or unable to create more threads. https://plumbr.eu/outofmemoryerror/unable-to-create-new-native-thread


> Race condition in ConcurrentUpdateSolrServer
> --------------------------------------------
>
>                 Key: SOLR-5850
>                 URL: https://issues.apache.org/jira/browse/SOLR-5850
>             Project: Solr
>          Issue Type: Bug
>          Components: clients - java, search, SolrCloud, update
>    Affects Versions: 4.6
>            Reporter: Devansh Dhutia
>            Assignee: Timothy Potter
>            Priority: Critical
>              Labels: 500, cloud, difficulty-medium, error, impact-medium, update
>         Attachments: SOLR-5850.patch, SOLR-5850.patch, SOLR-5850.patch, SOLR-5850.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Possibly related to SOLR-2308, we are seeing a Queue Full error message when issuing writes to Solr Cloud
> Each Update has 200 documents, and a commit is issued after 2000 documents have been added. 
> The writes are spread out to all the servers in the cloud (2 in this case) and following is the stack trace from Solr: 
> {code:xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <response>
> <lst name="responseHeader"><int name="status">500</int><int name="QTime">101</int></lst><lst name="error"><str name="msg">Queue full</str><str name="t
> race">java.lang.IllegalStateException: Queue full
>         at java.util.AbstractQueue.add(Unknown Source)
>         at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner$1.writeTo(ConcurrentUpdateSolrServer.java:181)
>         at org.apache.http.entity.EntityTemplate.writeTo(EntityTemplate.java:72)
>         at org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:98)
>         at org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:108)
>         at org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:122)
>         at org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:271)
>         at org.apache.http.impl.conn.ManagedClientConnectionImpl.sendRequestEntity(ManagedClientConnectionImpl.java:197)
>         at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:257)
>         at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
>         at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
>         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
>         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
>         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
>         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
>         at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:232)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
> </str><int name="code">500</int></lst>
> </response>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org