You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Daniel Miller <dm...@amfes.com> on 2017/04/24 21:55:29 UTC
Troubleshooting solr errors
I'm running Solr 6.4.2 to index my mail server (Dovecot). Searching is
great - but periodically I have Solr errors. Previously, when an error
would occur Solr would terminate. I now have it running as a systemd
service so it would auto-restart - but it seems like that doesn't solve it.
Some of the log lines include:
2017-04-24 18:18:31.101 ERROR (qtp594427726-30) [ x:dovecot]
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException:
Exception writing document id
17697/7db132200dd2df4d2f7b00003bc41c5f/dmiller@amfes.com to the index;
possible analysis error.
2017-04-24 18:18:31.125 ERROR (qtp594427726-32) [ x:dovecot]
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error
opening new searcher
I don't know what else to provide to try to troubleshoot this.
--
Daniel
Re: Troubleshooting solr errors
Posted by Erick Erickson <er...@gmail.com>.
Solr likes memory. A lot. Even 2G is quite small by recent
installations I have seen.
There is an "oom killer script" that can be specified to kill Solr if
it gets an OOM, at least then you have something to warn you.
After an OOM, Java is in an indeterminate state so all bets are off.
Best,
Erick
On Tue, Apr 25, 2017 at 11:05 AM, Daniel Miller <dm...@amfes.com> wrote:
> The problem isn't a particular email message - I get a cascade of those
> errors (every time a new message is received) once the server "breaks". The
> fix is to restart the server. I did find a Java heap error in the log - so
> I've increased the memory allocation (now to -Xms512m -Xmx2048m). I had
> thought that a heap failure would result in "simple" termination - and that
> systemd would restart it appropriately - but obviously I'm missing
> something.
>
> I was hoping to be able to help find whatever the bug might be - if indeed
> there is one - or if the problem is simply not enough memory available to
> Solr.
>
> As for the Dovecot specifics, if you'll check the Dovecot user mailing list
> archives for "Solr 6.4.1 config" you should find my post, including my
> config. If you need it I'll be happy to re-post that message here.
> Searching is performed by the IMAP clients via Dovecot - so no manual Solr
> queries are performed. I simply use the search function of my mail clients
> that support server-side searches (Thunderbird for Windoze/Linux, AquaMail
> for Android).
>
> --
> Daniel
>
>
> On 4/24/2017 5:43 PM, Rick Leir wrote:
>>
>> Daniel,
>> Would it be too much trouble to get some text out of that particular email
>> message, and try it in the Solr Admin Analysis tool?
>>
>> By the way, I also have my email in Dovecot. Would you be able to describe
>> how you index it and how you query to find an email? Perhaps with scripts in
>> a github project?
>> Thanks -- Rick
>>
>> On April 24, 2017 5:55:29 PM EDT, Daniel Miller <dm...@amfes.com> wrote:
>>>
>>> I'm running Solr 6.4.2 to index my mail server (Dovecot). Searching is
>>> great - but periodically I have Solr errors. Previously, when an error
>>> would occur Solr would terminate. I now have it running as a systemd
>>> service so it would auto-restart - but it seems like that doesn't solve
>>> it.
>>>
>>> Some of the log lines include:
>>>
>>> 2017-04-24 18:18:31.101 ERROR (qtp594427726-30) [ x:dovecot]
>>> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException:
>>> Exception writing document id
>>> 17697/7db132200dd2df4d2f7b00003bc41c5f/dmiller@amfes.com to the index;
>>> possible analysis error.
>>>
>>> 2017-04-24 18:18:31.125 ERROR (qtp594427726-32) [ x:dovecot]
>>> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error
>>> opening new searcher
>>>
>>> I don't know what else to provide to try to troubleshoot this.
>>>
>>> --
>>> Daniel
>
>
Re: Troubleshooting solr errors
Posted by Shawn Heisey <ap...@elyograg.org>.
On 4/25/2017 12:05 PM, Daniel Miller wrote:
> The problem isn't a particular email message - I get a cascade of
> those errors (every time a new message is received) once the server
> "breaks". The fix is to restart the server. I did find a Java heap
> error in the log - so I've increased the memory allocation (now to
> -Xms512m -Xmx2048m). I had thought that a heap failure would result
> in "simple" termination - and that systemd would restart it
> appropriately - but obviously I'm missing something.
Erick covered some of this already:
The init script that the service installer script installs on a
non-windows system can start Solr, but it will not automatically restart
it if it dies. That would require you to write something special,
probably a very custom systemd service specification, rather than use
the init script. Automatically restarting on death is not a good idea
-- it is VERY likely that whatever caused the death is going to happen
again.
Another detail, at least on non-windows systems, is that recent Solr
versions include a script that kills the process on OutOfMemoryError
(OOME). This is done because program operation is completely
unpredictable after that error occurs -- we have no way of knowing what
Solr will do. There's an issue in Jira to add OOME killing to the
Windows script.
FYI, the stacktrace from an OutOfMemoryError regarding the heap is
highly unlikely to give you anything useful about why the process ran
out of memory, since *any* memory allocation in any software running in
the JVM can trigger the error.
Other errors besides OOME should never terminate Solr unless there's an
enormous bug somewhere. That bug might be in Java itself, or even the OS.
Thanks,
Shawn
Re: Troubleshooting solr errors
Posted by Daniel Miller <dm...@amfes.com>.
The problem isn't a particular email message - I get a cascade of those
errors (every time a new message is received) once the server "breaks".
The fix is to restart the server. I did find a Java heap error in the
log - so I've increased the memory allocation (now to -Xms512m
-Xmx2048m). I had thought that a heap failure would result in "simple"
termination - and that systemd would restart it appropriately - but
obviously I'm missing something.
I was hoping to be able to help find whatever the bug might be - if
indeed there is one - or if the problem is simply not enough memory
available to Solr.
As for the Dovecot specifics, if you'll check the Dovecot user mailing
list archives for "Solr 6.4.1 config" you should find my post, including
my config. If you need it I'll be happy to re-post that message here.
Searching is performed by the IMAP clients via Dovecot - so no manual
Solr queries are performed. I simply use the search function of my mail
clients that support server-side searches (Thunderbird for
Windoze/Linux, AquaMail for Android).
--
Daniel
On 4/24/2017 5:43 PM, Rick Leir wrote:
> Daniel,
> Would it be too much trouble to get some text out of that particular email message, and try it in the Solr Admin Analysis tool?
>
> By the way, I also have my email in Dovecot. Would you be able to describe how you index it and how you query to find an email? Perhaps with scripts in a github project?
> Thanks -- Rick
>
> On April 24, 2017 5:55:29 PM EDT, Daniel Miller <dm...@amfes.com> wrote:
>> I'm running Solr 6.4.2 to index my mail server (Dovecot). Searching is
>> great - but periodically I have Solr errors. Previously, when an error
>> would occur Solr would terminate. I now have it running as a systemd
>> service so it would auto-restart - but it seems like that doesn't solve
>> it.
>>
>> Some of the log lines include:
>>
>> 2017-04-24 18:18:31.101 ERROR (qtp594427726-30) [ x:dovecot]
>> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException:
>> Exception writing document id
>> 17697/7db132200dd2df4d2f7b00003bc41c5f/dmiller@amfes.com to the index;
>> possible analysis error.
>>
>> 2017-04-24 18:18:31.125 ERROR (qtp594427726-32) [ x:dovecot]
>> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error
>> opening new searcher
>>
>> I don't know what else to provide to try to troubleshoot this.
>>
>> --
>> Daniel
Re: Troubleshooting solr errors
Posted by Rick Leir <rl...@leirtech.com>.
Daniel,
Would it be too much trouble to get some text out of that particular email message, and try it in the Solr Admin Analysis tool?
By the way, I also have my email in Dovecot. Would you be able to describe how you index it and how you query to find an email? Perhaps with scripts in a github project?
Thanks -- Rick
On April 24, 2017 5:55:29 PM EDT, Daniel Miller <dm...@amfes.com> wrote:
>I'm running Solr 6.4.2 to index my mail server (Dovecot). Searching is
>great - but periodically I have Solr errors. Previously, when an error
>would occur Solr would terminate. I now have it running as a systemd
>service so it would auto-restart - but it seems like that doesn't solve
>it.
>
>Some of the log lines include:
>
>2017-04-24 18:18:31.101 ERROR (qtp594427726-30) [ x:dovecot]
>o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException:
>Exception writing document id
>17697/7db132200dd2df4d2f7b00003bc41c5f/dmiller@amfes.com to the index;
>possible analysis error.
>
>2017-04-24 18:18:31.125 ERROR (qtp594427726-32) [ x:dovecot]
>o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error
>opening new searcher
>
>I don't know what else to provide to try to troubleshoot this.
>
>--
>Daniel
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com