You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by aparana bhatt <ap...@gmail.com> on 2020/10/13 04:18:32 UTC
Solr Document Update issues
Hi ,
I have been facing lot of issues in using solr update functionality .
Multitude of requests respond with
* org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://192.169.33.86/solr/cms
<http://192.169.33.86/solr/cms>: Expected mime type
application/octet-stream but got text/html. <!DOCTYPE HTML PUBLIC
"-//IETF//DTD HTML 2.0//EN"><html><head><title>502 Proxy
Error</title></head><body><h1>Proxy Error</h1><p>The proxy server received
an invalid^Mresponse from an upstream server.<br />^MThe proxy server could
not handle the request <em><a
href="/solr/cms/update">POST /solr/cms/update</a></em>.<p>Reason:
<strong>Error reading from remote server</strong></p></p></body></html>*
Used solr version -> 6.5.0 Type -> master/Slave config
Error in solr.log ->
*2020-10-07 05:43:50.639 WARN (qtp142261320-27831) [ x:cms]
o.a.s.c.SolrCore slow: [cms] webapp=/solr path=/update
params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}
status=0 QTime=443272020-10-07 05:43:50.640 WARN (qtp142261320-27837) [
x:cms] o.a.s.u.DefaultSolrCoreState WARNING - Dangerous
interruptjava.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:871)
at
org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:167)
at
org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:112)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:618)
at
org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)
at
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:68)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1895)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1872)
at
org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:72)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)*
The rate of update query on master solr is 6 request per min only .
Solr also slows down and search becomes really slow .
I don't understand where to look for an issue .
I have tried to check various parameters in update request , if I do
softcommit=true and commit =false then updates do not reflect , so i have
set below options ->
UpdateRequest updateRequest = new UpdateRequest();
updateRequest.setAction( UpdateRequest.ACTION.COMMIT, true,
true);
waitsearcher=true ,
waitflush=true .
I do not get what is causing the issue . Kindly suggest .
Also I could not find much help from internet about given issues as well .
--
Regards
Aparana Bhatt
Re: Solr Document Update issues
Posted by Radu Gheorghe <ra...@sematext.com>.
Hi,
I wouldn’t commit on every update. The general practice is to use autoCommit and autoSoftCommit, so this work is done in background depending on how quickly you want data persisted and available for search: https://lucene.apache.org/solr/guide/6_6/updatehandlers-in-solrconfig.html#UpdateHandlersinSolrConfig-Commits
Best regards,
Radu
--
Sematext Cloud - Full Stack Observability - https://sematext.com
Solr and Elasticsearch Consulting, Training and Production Support
> On 13 Oct 2020, at 07:18, aparana bhatt <ap...@gmail.com> wrote:
>
> Hi ,
>
> I have been facing lot of issues in using solr update functionality .
> Multitude of requests respond with
>
>
>
>
>
>
>
>
>
>
> * org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at http://192.169.33.86/solr/cms
> <http://192.169.33.86/solr/cms>: Expected mime type
> application/octet-stream but got text/html. <!DOCTYPE HTML PUBLIC
> "-//IETF//DTD HTML 2.0//EN"><html><head><title>502 Proxy
> Error</title></head><body><h1>Proxy Error</h1><p>The proxy server received
> an invalid^Mresponse from an upstream server.<br />^MThe proxy server could
> not handle the request <em><a
> href="/solr/cms/update">POST /solr/cms/update</a></em>.<p>Reason:
> <strong>Error reading from remote server</strong></p></p></body></html>*
>
> Used solr version -> 6.5.0 Type -> master/Slave config
> Error in solr.log ->
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *2020-10-07 05:43:50.639 WARN (qtp142261320-27831) [ x:cms]
> o.a.s.c.SolrCore slow: [cms] webapp=/solr path=/update
> params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}
> status=0 QTime=443272020-10-07 05:43:50.640 WARN (qtp142261320-27837) [
> x:cms] o.a.s.u.DefaultSolrCoreState WARNING - Dangerous
> interruptjava.lang.InterruptedException at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:871)
> at
> org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:167)
> at
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:112)
> at
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:618)
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:68)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1895)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1872)
> at
> org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:72)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)*
>
>
>
> The rate of update query on master solr is 6 request per min only .
> Solr also slows down and search becomes really slow .
> I don't understand where to look for an issue .
> I have tried to check various parameters in update request , if I do
> softcommit=true and commit =false then updates do not reflect , so i have
> set below options ->
>
> UpdateRequest updateRequest = new UpdateRequest();
> updateRequest.setAction( UpdateRequest.ACTION.COMMIT, true,
> true);
> waitsearcher=true ,
> waitflush=true .
>
> I do not get what is causing the issue . Kindly suggest .
> Also I could not find much help from internet about given issues as well .
>
>
> --
> Regards
>
> Aparana Bhatt