You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Patrick Sauts <pa...@gmail.com> on 2009/12/31 11:07:24 UTC
Invalid CRLF - StreamingUpdateSolrServer ?
I'm using solr 1.4 on tomcat 5.0.28, with client
StreamingUpdateSolrServer with 10threads and xml communication via Post
method.
Is there a way to avoid this error (data lost)?
And is StreamingUpdateSolrServer reliable ?
GRAVE: org.apache.solr.common.SolrException: Invalid CRLF
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:72)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:619)
Caused by: com.ctc.wstx.exc.WstxIOException: Invalid CRLF
Re: Invalid CRLF - StreamingUpdateSolrServer ?
Posted by Patrick Sauts <pa...@gmail.com>.
I've patched the solrj release(tag) 1.4 with SOLR-1595, it's online for
about two weeks now and It's working just fine.
Thanks a lot.
Patrick.
P.S.: It's a pity there is no plan for a 1.4.1 release
Yonik Seeley a écrit :
> It could be this bug, fixed in trunk:
>
> * SOLR-1595: StreamingUpdateSolrServer used the platform default character
> set when streaming updates, rather than using UTF-8 as the HTTP headers
> indicated, leading to an encoding mismatch. (hossman, yonik)
>
> Could you try a recent nightly build (or build your own from trunk)
> and see if it fixes it?
>
> -Yonik
> http://www.lucidimagination.com
>
>
>
> On Thu, Dec 31, 2009 at 5:07 AM, Patrick Sauts <pa...@gmail.com> wrote:
>
>> I'm using solr 1.4 on tomcat 5.0.28, with client StreamingUpdateSolrServer
>> with 10threads and xml communication via Post method.
>>
>> Is there a way to avoid this error (data lost)?
>> And is StreamingUpdateSolrServer reliable ?
>>
>> GRAVE: org.apache.solr.common.SolrException: Invalid CRLF
>> at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:72)
>> at
>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>> at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
>> at
>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>> at
>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>> at
>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>> at
>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>> at java.lang.Thread.run(Thread.java:619)
>> Caused by: com.ctc.wstx.exc.WstxIOException: Invalid CRLF
>>
>>
>>
>
>
Re: Invalid CRLF - StreamingUpdateSolrServer ?
Posted by Patrick Sauts <pa...@gmail.com>.
>> The issue was sometimes null result during facet navigation or simple
>> search, results were back after a refresh, we tried to changed the cache to
>> <httpCaching never304="true"/>. But same behaviour.
>>
>>
>>
> That is strange. Just to make sure, you were using the same LBHttpSolrServer
> instance for all requests, weren't you?
>
Yes it was a single static instance for all request on the same
core/index. We have 6 differents indexes on a tomcat.
When testing localy I had no problem, but the dysfunction happened on
real traffic application server.
That's why I think it might not be thread safe.
>> *My implementation was :* (maybe wrong ?)
>> LBHttpSolrServer solrServer = new LBHttpSolrServer(new HttpClient(), new
>> XMLResponseParser(), solrServerUrl.split(","));
>> solrServer.setConnectionManagerTimeout(CONNECTION_TIMEOUT);
>> solrServer.setConnectionTimeout(CONNECTION_TIMEOUT);
>> solrServer.setSoTimeout(READ_TIMEOUT);
>> solrServer.setAliveCheckInterval(CHECK_HEALTH_INTERVAL_MS);
>>
>> *What I was suggesting :*
>> As a LBHttpSolrServer is a wrapper to CommonsHttpSolrServer
>>
>>
>>
> I think that is a good idea. Can you open a jira issue?
>
I have opened : SOLR-1700.
I hope it is precise enough.
Re: Invalid CRLF - StreamingUpdateSolrServer ?
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Mon, Jan 4, 2010 at 7:13 PM, Patrick Sauts <pa...@gmail.com>wrote:
> The issue was sometimes null result during facet navigation or simple
> search, results were back after a refresh, we tried to changed the cache to
> <httpCaching never304="true"/>. But same behaviour.
>
>
That is strange. Just to make sure, you were using the same LBHttpSolrServer
instance for all requests, weren't you?
> *My implementation was :* (maybe wrong ?)
> LBHttpSolrServer solrServer = new LBHttpSolrServer(new HttpClient(), new
> XMLResponseParser(), solrServerUrl.split(","));
> solrServer.setConnectionManagerTimeout(CONNECTION_TIMEOUT);
> solrServer.setConnectionTimeout(CONNECTION_TIMEOUT);
> solrServer.setSoTimeout(READ_TIMEOUT);
> solrServer.setAliveCheckInterval(CHECK_HEALTH_INTERVAL_MS);
>
> *What I was suggesting :*
> As a LBHttpSolrServer is a wrapper to CommonsHttpSolrServer
>
>
I think that is a good idea. Can you open a jira issue?
--
Regards,
Shalin Shekhar Mangar.
Re: Invalid CRLF - StreamingUpdateSolrServer ?
Posted by Patrick Sauts <pa...@gmail.com>.
The issue was sometimes null result during facet navigation or simple
search, results were back after a refresh, we tried to changed the cache
to <httpCaching never304="true"/>. But same behaviour.
*My implementation was :* (maybe wrong ?)
LBHttpSolrServer solrServer = new LBHttpSolrServer(new HttpClient(), new
XMLResponseParser(), solrServerUrl.split(","));
solrServer.setConnectionManagerTimeout(CONNECTION_TIMEOUT);
solrServer.setConnectionTimeout(CONNECTION_TIMEOUT);
solrServer.setSoTimeout(READ_TIMEOUT);
solrServer.setAliveCheckInterval(CHECK_HEALTH_INTERVAL_MS);
*What I was suggesting :*
As a LBHttpSolrServer is a wrapper to CommonsHttpSolrServer
CommonsHttpSolrServer search1 = new
CommonsHttpSolrServer("http://mysearch1");
search1.setConnectionTimeout(CONNECTION_TIMEOUT);
search1.setSoTimeout(READ_TIMEOUT);
search1.setConnectionManagerTimeout(solr.CONNECTION_MANAGER_TIMEOUT);
search1.setDefaultMaxConnectionsPerHost(MAX_CONNECTIONS_PER_HOST1);
search1.setMaxTotalConnections(MAX_TOTAL_CONNECTIONS1);
search1.setParser(new XMLResponseParser());
CommonsHttpSolrServer search2 = new
CommonsHttpSolrServer("http://mysearch1");
search2.setConnectionTimeout(CONNECTION_TIMEOUT);
search2.setSoTimeout(READ_TIMEOUT);
search2.setConnectionManagerTimeout(solr.CONNECTION_MANAGER_TIMEOUT);
search2.setDefaultMaxConnectionsPerHost(MAX_CONNECTIONS_PER_HOST1);
search2.setMaxTotalConnections(MAX_TOTAL_CONNECTIONS1);
search2.setParser(new XMLResponseParser());
*LBHttpSolrServer solrServers = new LBHttpSolrServer(search1, search2);*
So we can manage the parameters per server.
Thank you for your time.
Patrick.
Shalin Shekhar Mangar a écrit :
> On Mon, Jan 4, 2010 at 6:11 PM, Patrick Sauts <pa...@gmail.com>wrote:
>
>
>> I've also tested LBHttpSolrServer (We wanted to have it as a "backup" for
>> HAproxy) and it appears not to be thread safe ( what is also curious about
>> it, is that there's no way to manage the connections' pool ). If you're
>> interresting in the logs, I can send those to you.
>>
>>
>
> What is the issue that you are facing? What is it exactly that you want to
> change?
>
>
Re: Invalid CRLF - StreamingUpdateSolrServer ?
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Mon, Jan 4, 2010 at 6:11 PM, Patrick Sauts <pa...@gmail.com>wrote:
>
> I've also tested LBHttpSolrServer (We wanted to have it as a "backup" for
> HAproxy) and it appears not to be thread safe ( what is also curious about
> it, is that there's no way to manage the connections' pool ). If you're
> interresting in the logs, I can send those to you.
>
What is the issue that you are facing? What is it exactly that you want to
change?
--
Regards,
Shalin Shekhar Mangar.
Re: Invalid CRLF - StreamingUpdateSolrServer ?
Posted by Patrick Sauts <pa...@gmail.com>.
Thank you Yonik for your answer.
The platform encoding is "fr_FR.UTF-8", so it's still UTF-8, it should
be I guess "en_US.UTF-8" ?
I've also tested LBHttpSolrServer (We wanted to have it as a "backup"
for HAproxy) and it appears not to be thread safe ( what is also curious
about it, is that there's no way to manage the connections' pool ). If
you're interresting in the logs, I can send those to you.
*Will there be a Solr 1.4.1 that'll fix those problems ?*
Cause using a SNAPSHOT doesn't seem a good idea to me.
I have another question but I don't know if I have to make a new post :
Can I use "-Dmaster=disabled" in JAVA_OPTS for a server that is slave
and repeater ?
Patrick.
Yonik Seeley a écrit :
> It could be this bug, fixed in trunk:
>
> * SOLR-1595: StreamingUpdateSolrServer used the platform default character
> set when streaming updates, rather than using UTF-8 as the HTTP headers
> indicated, leading to an encoding mismatch. (hossman, yonik)
>
> Could you try a recent nightly build (or build your own from trunk)
> and see if it fixes it?
>
> -Yonik
> http://www.lucidimagination.com
>
>
>
> On Thu, Dec 31, 2009 at 5:07 AM, Patrick Sauts <pa...@gmail.com> wrote:
>
>> I'm using solr 1.4 on tomcat 5.0.28, with client StreamingUpdateSolrServer
>> with 10threads and xml communication via Post method.
>>
>> Is there a way to avoid this error (data lost)?
>> And is StreamingUpdateSolrServer reliable ?
>>
>> GRAVE: org.apache.solr.common.SolrException: Invalid CRLF
>> at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:72)
>> at
>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>> at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
>> at
>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>> at
>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>> at
>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>> at
>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>> at java.lang.Thread.run(Thread.java:619)
>> Caused by: com.ctc.wstx.exc.WstxIOException: Invalid CRLF
>>
>>
>>
>
>
Re: Invalid CRLF - StreamingUpdateSolrServer ?
Posted by Yonik Seeley <yo...@lucidimagination.com>.
It could be this bug, fixed in trunk:
* SOLR-1595: StreamingUpdateSolrServer used the platform default character
set when streaming updates, rather than using UTF-8 as the HTTP headers
indicated, leading to an encoding mismatch. (hossman, yonik)
Could you try a recent nightly build (or build your own from trunk)
and see if it fixes it?
-Yonik
http://www.lucidimagination.com
On Thu, Dec 31, 2009 at 5:07 AM, Patrick Sauts <pa...@gmail.com> wrote:
> I'm using solr 1.4 on tomcat 5.0.28, with client StreamingUpdateSolrServer
> with 10threads and xml communication via Post method.
>
> Is there a way to avoid this error (data lost)?
> And is StreamingUpdateSolrServer reliable ?
>
> GRAVE: org.apache.solr.common.SolrException: Invalid CRLF
> at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:72)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
> at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: com.ctc.wstx.exc.WstxIOException: Invalid CRLF
>
>