You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Walter Closenfleight <wa...@gmail.com> on 2011/09/23 14:55:39 UTC
Solrj - when a request fails
*
I have a java program which sends thousands of Solr XML files up to Solr
using the following code. It works fine until there is a problem with one of
the Solr XML files. The code fails on the solrServer.request(up) line, but
it does not throw an exception, my application therefore cannot catch it and
recover, and my whole application dies.
I've fixed this individual file that made it fail, but want to better trap
these so my application does not die.
Thanks for any insight you can provide. Java code and log below-
// ... start of a loop to process each file removed ...
try {
String xml = read(filename);
DirectXmlRequest up = new DirectXmlRequest( "/update", xml );
solrServer.request( up );
solrServer.commit();
} catch (SolrServerException e) {
log.warn("Exception: "+ e.toString());
throw new MyException(e);
} catch (IOException e) {
log.warn("Exception: "+ e.toString());
throw new MyException(e);
}
DEBUG >> "[\n]" - (Wire.java:70)
DEBUG Request body sent - (EntityEnclosingMethod.java:508)
DEBUG << "HTTP/1.1 400 Bad Request[\r][\n]" - (Wire.java:70)
DEBUG << "HTTP/1.1 400 Bad Request[\r][\n]" - (Wire.java:70)
DEBUG << "Server: Apache-Coyote/1.1[\r][\n]" - (Wire.java:70)
DEBUG << "Content-Type: text/html;charset=utf-8[\r][\n]" - (Wire.java:70)
DEBUG << "Content-Length: 1271[\r][\n]" - (Wire.java:70)
DEBUG << "Date: Fri, 23 Sep 2011 12:08:05 GMT[\r][\n]" - (Wire.java:70)
DEBUG << "Connection: close[\r][\n]" - (Wire.java:70)
DEBUG << "[\r][\n]" - (Wire.java:70)
DEBUG << "<html><head><title>Apache Tomcat/6.0.29 - Error
report</title><style><!--H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
H2
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
H3
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
BODY
{font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;}
P
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A
{color : black;}A.name {color : black;}HR {color : #525D76;}--></style>
</head><body><h1>HTTP Status 400 - Unexpected character 'x' (code 120) in
prolog; expected '<'[\n]" - (Wire.java:70)
DEBUG << " at [row,col {unknown-source}]: [3,1]</h1><HR size="1"
noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b>
<u>Unexpected character 'x' (code 120) in prolog; expected '<'[\n]" -
(Wire.java:70)
DEBUG << " at [row,col {unknown-source}]: [3,1]</u></p><p><b>description</b>
" - (Wire.java:84)
DEBUG << "<u>The request sent by the client was syntactically incorrect
(Unexpected character 'x' (code 120) in prolog; expected '<'[\n]" -
(Wire.java:70)
DEBUG << " at [row,col {unknown-source}]: [3,1]).</u></p><HR size="1"
noshade="noshade"><h3>Apache Tomcat/6.0.29</h3></body></html>" -
(Wire.java:84)
DEBUG Should close connection in response to directive: close -
(HttpMethodBase.java:1008)
*
Re: Solrj - when a request fails
Posted by Erick Erickson <er...@gmail.com>.
Hmmm. I'm a little confused. Are you sure your log is going
somewhere and that you are NOT seeing any stack traces?
Because it looks like you *are* seeing them. In which case
re-throwing an error breaks your file fetch loop and stops
your processing.....
I'd actually expect that you're losing some files before as well, since
even if you do trap those errors you're not doing a commit operation,
unless perhaps autocommit is saving you.
BTW, committing after every document is probably a bad idea, it'll create
lots and lots of segments unnecessarily. I'd rely on the autocommit
features and optionally commit after the run is completed.
Best
Erick
On Fri, Sep 23, 2011 at 5:55 AM, Walter Closenfleight
<wa...@gmail.com> wrote:
> *
> I have a java program which sends thousands of Solr XML files up to Solr
> using the following code. It works fine until there is a problem with one of
> the Solr XML files. The code fails on the solrServer.request(up) line, but
> it does not throw an exception, my application therefore cannot catch it and
> recover, and my whole application dies.
>
> I've fixed this individual file that made it fail, but want to better trap
> these so my application does not die.
>
> Thanks for any insight you can provide. Java code and log below-
>
>
> // ... start of a loop to process each file removed ...
>
> try {
>
> String xml = read(filename);
> DirectXmlRequest up = new DirectXmlRequest( "/update", xml );
>
> solrServer.request( up );
> solrServer.commit();
>
> } catch (SolrServerException e) {
> log.warn("Exception: "+ e.toString());
> throw new MyException(e);
> } catch (IOException e) {
> log.warn("Exception: "+ e.toString());
> throw new MyException(e);
> }
> DEBUG >> "[\n]" - (Wire.java:70)
> DEBUG Request body sent - (EntityEnclosingMethod.java:508)
> DEBUG << "HTTP/1.1 400 Bad Request[\r][\n]" - (Wire.java:70)
> DEBUG << "HTTP/1.1 400 Bad Request[\r][\n]" - (Wire.java:70)
> DEBUG << "Server: Apache-Coyote/1.1[\r][\n]" - (Wire.java:70)
> DEBUG << "Content-Type: text/html;charset=utf-8[\r][\n]" - (Wire.java:70)
> DEBUG << "Content-Length: 1271[\r][\n]" - (Wire.java:70)
> DEBUG << "Date: Fri, 23 Sep 2011 12:08:05 GMT[\r][\n]" - (Wire.java:70)
> DEBUG << "Connection: close[\r][\n]" - (Wire.java:70)
> DEBUG << "[\r][\n]" - (Wire.java:70)
> DEBUG << "<html><head><title>Apache Tomcat/6.0.29 - Error
> report</title><style><!--H1
> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
> H2
> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
> H3
> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
> BODY
> {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B
> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;}
> P
> {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A
> {color : black;}A.name {color : black;}HR {color : #525D76;}--></style>
> </head><body><h1>HTTP Status 400 - Unexpected character 'x' (code 120) in
> prolog; expected '<'[\n]" - (Wire.java:70)
> DEBUG << " at [row,col {unknown-source}]: [3,1]</h1><HR size="1"
> noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b>
> <u>Unexpected character 'x' (code 120) in prolog; expected '<'[\n]" -
> (Wire.java:70)
> DEBUG << " at [row,col {unknown-source}]: [3,1]</u></p><p><b>description</b>
> " - (Wire.java:84)
> DEBUG << "<u>The request sent by the client was syntactically incorrect
> (Unexpected character 'x' (code 120) in prolog; expected '<'[\n]" -
> (Wire.java:70)
> DEBUG << " at [row,col {unknown-source}]: [3,1]).</u></p><HR size="1"
> noshade="noshade"><h3>Apache Tomcat/6.0.29</h3></body></html>" -
> (Wire.java:84)
> DEBUG Should close connection in response to directive: close -
> (HttpMethodBase.java:1008)
> *
>
Re: Solrj - when a request fails
Posted by Walter Closenfleight <wa...@gmail.com>.
I tried that with the same results. You would think I would get the
exception back from Solr so I could trap it, instead I lose all other
requests after it.
On Fri, Sep 23, 2011 at 8:33 AM, Gunther, Andrew <Gu...@si.edu> wrote:
> All the solr methods look like they should throw those 2 exceptions.
> Have you tried the DirectXmlRequest method?
>
> up.process(solrServer);
>
> public UpdateResponse process( SolrServer server ) throws
> SolrServerException, IOException
> {
> long startTime = System.currentTimeMillis();
> UpdateResponse res = new UpdateResponse();
> res.setResponse( server.request( this ) );
> res.setElapsedTime( System.currentTimeMillis()-startTime );
> return res;
> }
> ________________________________________
> From: Walter Closenfleight [walter.p.closenfleight@gmail.com]
> Sent: Friday, September 23, 2011 8:55 AM
> To: solr-user@lucene.apache.org
> Subject: Solrj - when a request fails
>
> *
> I have a java program which sends thousands of Solr XML files up to Solr
> using the following code. It works fine until there is a problem with one
> of
> the Solr XML files. The code fails on the solrServer.request(up) line, but
> it does not throw an exception, my application therefore cannot catch it
> and
> recover, and my whole application dies.
>
> I've fixed this individual file that made it fail, but want to better trap
> these so my application does not die.
>
> Thanks for any insight you can provide. Java code and log below-
>
>
> // ... start of a loop to process each file removed ...
>
> try {
>
> String xml = read(filename);
> DirectXmlRequest up = new DirectXmlRequest( "/update", xml );
>
> solrServer.request( up );
> solrServer.commit();
>
> } catch (SolrServerException e) {
> log.warn("Exception: "+ e.toString());
> throw new MyException(e);
> } catch (IOException e) {
> log.warn("Exception: "+ e.toString());
> throw new MyException(e);
> }
> DEBUG >> "[\n]" - (Wire.java:70)
> DEBUG Request body sent - (EntityEnclosingMethod.java:508)
> DEBUG << "HTTP/1.1 400 Bad Request[\r][\n]" - (Wire.java:70)
> DEBUG << "HTTP/1.1 400 Bad Request[\r][\n]" - (Wire.java:70)
> DEBUG << "Server: Apache-Coyote/1.1[\r][\n]" - (Wire.java:70)
> DEBUG << "Content-Type: text/html;charset=utf-8[\r][\n]" - (Wire.java:70)
> DEBUG << "Content-Length: 1271[\r][\n]" - (Wire.java:70)
> DEBUG << "Date: Fri, 23 Sep 2011 12:08:05 GMT[\r][\n]" - (Wire.java:70)
> DEBUG << "Connection: close[\r][\n]" - (Wire.java:70)
> DEBUG << "[\r][\n]" - (Wire.java:70)
> DEBUG << "<html><head><title>Apache Tomcat/6.0.29 - Error
> report</title><style><!--H1
>
> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
> H2
>
> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
> H3
>
> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
> BODY
> {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B
> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;}
> P
>
> {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A
> {color : black;}A.name {color : black;}HR {color : #525D76;}--></style>
> </head><body><h1>HTTP Status 400 - Unexpected character 'x' (code 120) in
> prolog; expected '<'[\n]" - (Wire.java:70)
> DEBUG << " at [row,col {unknown-source}]: [3,1]</h1><HR size="1"
> noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b>
> <u>Unexpected character 'x' (code 120) in prolog; expected '<'[\n]" -
> (Wire.java:70)
> DEBUG << " at [row,col {unknown-source}]:
> [3,1]</u></p><p><b>description</b>
> " - (Wire.java:84)
> DEBUG << "<u>The request sent by the client was syntactically incorrect
> (Unexpected character 'x' (code 120) in prolog; expected '<'[\n]" -
> (Wire.java:70)
> DEBUG << " at [row,col {unknown-source}]: [3,1]).</u></p><HR size="1"
> noshade="noshade"><h3>Apache Tomcat/6.0.29</h3></body></html>" -
> (Wire.java:84)
> DEBUG Should close connection in response to directive: close -
> (HttpMethodBase.java:1008)
> *
>
RE: Solrj - when a request fails
Posted by "Gunther, Andrew" <Gu...@si.edu>.
All the solr methods look like they should throw those 2 exceptions.
Have you tried the DirectXmlRequest method?
up.process(solrServer);
public UpdateResponse process( SolrServer server ) throws SolrServerException, IOException
{
long startTime = System.currentTimeMillis();
UpdateResponse res = new UpdateResponse();
res.setResponse( server.request( this ) );
res.setElapsedTime( System.currentTimeMillis()-startTime );
return res;
}
________________________________________
From: Walter Closenfleight [walter.p.closenfleight@gmail.com]
Sent: Friday, September 23, 2011 8:55 AM
To: solr-user@lucene.apache.org
Subject: Solrj - when a request fails
*
I have a java program which sends thousands of Solr XML files up to Solr
using the following code. It works fine until there is a problem with one of
the Solr XML files. The code fails on the solrServer.request(up) line, but
it does not throw an exception, my application therefore cannot catch it and
recover, and my whole application dies.
I've fixed this individual file that made it fail, but want to better trap
these so my application does not die.
Thanks for any insight you can provide. Java code and log below-
// ... start of a loop to process each file removed ...
try {
String xml = read(filename);
DirectXmlRequest up = new DirectXmlRequest( "/update", xml );
solrServer.request( up );
solrServer.commit();
} catch (SolrServerException e) {
log.warn("Exception: "+ e.toString());
throw new MyException(e);
} catch (IOException e) {
log.warn("Exception: "+ e.toString());
throw new MyException(e);
}
DEBUG >> "[\n]" - (Wire.java:70)
DEBUG Request body sent - (EntityEnclosingMethod.java:508)
DEBUG << "HTTP/1.1 400 Bad Request[\r][\n]" - (Wire.java:70)
DEBUG << "HTTP/1.1 400 Bad Request[\r][\n]" - (Wire.java:70)
DEBUG << "Server: Apache-Coyote/1.1[\r][\n]" - (Wire.java:70)
DEBUG << "Content-Type: text/html;charset=utf-8[\r][\n]" - (Wire.java:70)
DEBUG << "Content-Length: 1271[\r][\n]" - (Wire.java:70)
DEBUG << "Date: Fri, 23 Sep 2011 12:08:05 GMT[\r][\n]" - (Wire.java:70)
DEBUG << "Connection: close[\r][\n]" - (Wire.java:70)
DEBUG << "[\r][\n]" - (Wire.java:70)
DEBUG << "<html><head><title>Apache Tomcat/6.0.29 - Error
report</title><style><!--H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
H2
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
H3
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
BODY
{font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;}
P
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A
{color : black;}A.name {color : black;}HR {color : #525D76;}--></style>
</head><body><h1>HTTP Status 400 - Unexpected character 'x' (code 120) in
prolog; expected '<'[\n]" - (Wire.java:70)
DEBUG << " at [row,col {unknown-source}]: [3,1]</h1><HR size="1"
noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b>
<u>Unexpected character 'x' (code 120) in prolog; expected '<'[\n]" -
(Wire.java:70)
DEBUG << " at [row,col {unknown-source}]: [3,1]</u></p><p><b>description</b>
" - (Wire.java:84)
DEBUG << "<u>The request sent by the client was syntactically incorrect
(Unexpected character 'x' (code 120) in prolog; expected '<'[\n]" -
(Wire.java:70)
DEBUG << " at [row,col {unknown-source}]: [3,1]).</u></p><HR size="1"
noshade="noshade"><h3>Apache Tomcat/6.0.29</h3></body></html>" -
(Wire.java:84)
DEBUG Should close connection in response to directive: close -
(HttpMethodBase.java:1008)
*