You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Hoffman, Chase" <Ho...@advisory.com> on 2012/02/08 21:13:00 UTC
Index Start Question
Please forgive me if this is a dumb question. I've never dealt with SOLR before, and I'm being asked to determine from the logs when a SOLR index is kicked off (it is a Windows server). The TOMCAT service runs continually, so no love there. In parsing the logs, I think "org.apache.solr.core.SolrResourceLoader <init>" is the indicator, since "org.apache.solr.core.SolrCore execute" seems to occur even when I know an index has not been started.
Any advice you could give me would be wonderful.
Best,
--Chase
Chase Hoffman
Infrastructure Systems Administrator, Performance Technologies
The Advisory Board Company
512-681-2190 direct | 512-609-1150 fax
hoffmanc@advisory.com<ma...@advisory.com> | www.advisory.com<http://www.advisory.com>
Don't miss out-log in now
Unlock thousands of members-only tools, events, best practices, and more at www.advisory.com.
Get started<http://www.advisory.com/reasons-to-log-in-now/?WT.mc_id=eMail|SignatureLine|Other|ABC|Login+8+Reasons|Nov212011>
RE: Index Start Question
Posted by "Hoffman, Chase" <Ho...@advisory.com>.
Erick,
Thanks for the suggestion. I think we're going to go that route.
Best,
--Chase
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Thursday, February 09, 2012 12:30 PM
To: solr-user@lucene.apache.org
Subject: Re: Index Start Question
Hmmm. You say:
"The DBA opens a command line prompt and initiates an index build/rebuild"
How? By issuing a curl command? Running a program? It seems to me that the
easiest thing to do here would be to create a small program that kicks
off the indexing process and have *that* program send the e-mails when
it starts and perhaps a completion e-mail after it's done.
Seems a lot surer than trying to infer the action from the Solr logs...
Best
Erick
On Thu, Feb 9, 2012 at 10:43 AM, Hoffman, Chase <Ho...@advisory.com> wrote:
> Erick,
>
> My understanding of the process is this:
>
> 1. The DBA opens a command line prompt and initiates an index build/rebuild
> 2. SOLR performs said index build/rebuild
> 3. Index finishes
>
> I don't think we're appending documents to the SOLR index - it's indexing MSSQL tables. The servers these are running on aren't beefy enough to run multiple SOLR index builds at the same time. So the hope is to find some key in the logs that shows the start of the index rebuild so that I can put in some automation to blast out an email saying "Server X is currently running an index, do not kick off an index run on Server X".
>
> Thanks so much for your help.
>
> Best,
>
> --Chase
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Thursday, February 09, 2012 9:39 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Index Start Question
>
> OK, what do you mean by "index is kicked off"? You mean starting Solr or actually adding a document to a running Solr?
>
> If the latter, you're probably looking for something like this:
> Feb 9, 2012 10:34:26 AM
> org.apache.solr.update.processor.LogUpdateProcessor finish
> INFO: {add=[eoe32]} 0 6
>
> The important bits are solr.update.processor and the add=blahblah bit where the stuff after the = will be a list of <uniqueKey>s for the document(s) added.
>
> However, this will be somewhat fragile, the format of the logged messages is not guaranteed in future versions.
>
> Although this is happening, I think, after the doc has been added to the index, so it may be too late for your problem.
>
> Best
> Erick
>
> On Wed, Feb 8, 2012 at 3:13 PM, Hoffman, Chase <Ho...@advisory.com> wrote:
>> Please forgive me if this is a dumb question. I've never dealt with SOLR before, and I'm being asked to determine from the logs when a SOLR index is kicked off (it is a Windows server). The TOMCAT service runs continually, so no love there. In parsing the logs, I think "org.apache.solr.core.SolrResourceLoader <init>" is the indicator, since "org.apache.solr.core.SolrCore execute" seems to occur even when I know an index has not been started.
>>
>> Any advice you could give me would be wonderful.
>>
>> Best,
>>
>> --Chase
>>
>> Chase Hoffman
>> Infrastructure Systems Administrator, Performance Technologies The
>> Advisory Board Company
>> 512-681-2190 direct | 512-609-1150 fax
>> hoffmanc@advisory.com<ma...@advisory.com> |
>> www.advisory.com<http://www.advisory.com>
>>
>> Don't miss out-log in now
>> Unlock thousands of members-only tools, events, best practices, and more at www.advisory.com.
>> Get
>> started<http://www.advisory.com/reasons-to-log-in-now/?WT.mc_id=eMail|
>> SignatureLine|Other|ABC|Login+8+Reasons|Nov212011>
>
Re: Index Start Question
Posted by Erick Erickson <er...@gmail.com>.
Hmmm. You say:
"The DBA opens a command line prompt and initiates an index build/rebuild"
How? By issuing a curl command? Running a program? It seems to me that the
easiest thing to do here would be to create a small program that kicks
off the indexing process and have *that* program send the e-mails when
it starts and perhaps a completion e-mail after it's done.
Seems a lot surer than trying to infer the action from the Solr logs...
Best
Erick
On Thu, Feb 9, 2012 at 10:43 AM, Hoffman, Chase <Ho...@advisory.com> wrote:
> Erick,
>
> My understanding of the process is this:
>
> 1. The DBA opens a command line prompt and initiates an index build/rebuild
> 2. SOLR performs said index build/rebuild
> 3. Index finishes
>
> I don't think we're appending documents to the SOLR index - it's indexing MSSQL tables. The servers these are running on aren't beefy enough to run multiple SOLR index builds at the same time. So the hope is to find some key in the logs that shows the start of the index rebuild so that I can put in some automation to blast out an email saying "Server X is currently running an index, do not kick off an index run on Server X".
>
> Thanks so much for your help.
>
> Best,
>
> --Chase
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Thursday, February 09, 2012 9:39 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Index Start Question
>
> OK, what do you mean by "index is kicked off"? You mean starting Solr or actually adding a document to a running Solr?
>
> If the latter, you're probably looking for something like this:
> Feb 9, 2012 10:34:26 AM
> org.apache.solr.update.processor.LogUpdateProcessor finish
> INFO: {add=[eoe32]} 0 6
>
> The important bits are solr.update.processor and the add=blahblah bit where the stuff after the = will be a list of <uniqueKey>s for the document(s) added.
>
> However, this will be somewhat fragile, the format of the logged messages is not guaranteed in future versions.
>
> Although this is happening, I think, after the doc has been added to the index, so it may be too late for your problem.
>
> Best
> Erick
>
> On Wed, Feb 8, 2012 at 3:13 PM, Hoffman, Chase <Ho...@advisory.com> wrote:
>> Please forgive me if this is a dumb question. I've never dealt with SOLR before, and I'm being asked to determine from the logs when a SOLR index is kicked off (it is a Windows server). The TOMCAT service runs continually, so no love there. In parsing the logs, I think "org.apache.solr.core.SolrResourceLoader <init>" is the indicator, since "org.apache.solr.core.SolrCore execute" seems to occur even when I know an index has not been started.
>>
>> Any advice you could give me would be wonderful.
>>
>> Best,
>>
>> --Chase
>>
>> Chase Hoffman
>> Infrastructure Systems Administrator, Performance Technologies The
>> Advisory Board Company
>> 512-681-2190 direct | 512-609-1150 fax
>> hoffmanc@advisory.com<ma...@advisory.com> |
>> www.advisory.com<http://www.advisory.com>
>>
>> Don't miss out-log in now
>> Unlock thousands of members-only tools, events, best practices, and more at www.advisory.com.
>> Get
>> started<http://www.advisory.com/reasons-to-log-in-now/?WT.mc_id=eMail|
>> SignatureLine|Other|ABC|Login+8+Reasons|Nov212011>
>
RE: Index Start Question
Posted by "Hoffman, Chase" <Ho...@advisory.com>.
Erick,
My understanding of the process is this:
1. The DBA opens a command line prompt and initiates an index build/rebuild
2. SOLR performs said index build/rebuild
3. Index finishes
I don't think we're appending documents to the SOLR index - it's indexing MSSQL tables. The servers these are running on aren't beefy enough to run multiple SOLR index builds at the same time. So the hope is to find some key in the logs that shows the start of the index rebuild so that I can put in some automation to blast out an email saying "Server X is currently running an index, do not kick off an index run on Server X".
Thanks so much for your help.
Best,
--Chase
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Thursday, February 09, 2012 9:39 AM
To: solr-user@lucene.apache.org
Subject: Re: Index Start Question
OK, what do you mean by "index is kicked off"? You mean starting Solr or actually adding a document to a running Solr?
If the latter, you're probably looking for something like this:
Feb 9, 2012 10:34:26 AM
org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: {add=[eoe32]} 0 6
The important bits are solr.update.processor and the add=blahblah bit where the stuff after the = will be a list of <uniqueKey>s for the document(s) added.
However, this will be somewhat fragile, the format of the logged messages is not guaranteed in future versions.
Although this is happening, I think, after the doc has been added to the index, so it may be too late for your problem.
Best
Erick
On Wed, Feb 8, 2012 at 3:13 PM, Hoffman, Chase <Ho...@advisory.com> wrote:
> Please forgive me if this is a dumb question. I've never dealt with SOLR before, and I'm being asked to determine from the logs when a SOLR index is kicked off (it is a Windows server). The TOMCAT service runs continually, so no love there. In parsing the logs, I think "org.apache.solr.core.SolrResourceLoader <init>" is the indicator, since "org.apache.solr.core.SolrCore execute" seems to occur even when I know an index has not been started.
>
> Any advice you could give me would be wonderful.
>
> Best,
>
> --Chase
>
> Chase Hoffman
> Infrastructure Systems Administrator, Performance Technologies The
> Advisory Board Company
> 512-681-2190 direct | 512-609-1150 fax
> hoffmanc@advisory.com<ma...@advisory.com> |
> www.advisory.com<http://www.advisory.com>
>
> Don't miss out-log in now
> Unlock thousands of members-only tools, events, best practices, and more at www.advisory.com.
> Get
> started<http://www.advisory.com/reasons-to-log-in-now/?WT.mc_id=eMail|
> SignatureLine|Other|ABC|Login+8+Reasons|Nov212011>
Re: Index Start Question
Posted by Erick Erickson <er...@gmail.com>.
OK, what do you mean by "index is kicked off"? You mean starting Solr
or actually adding a document to a running Solr?
If the latter, you're probably looking for something like this:
Feb 9, 2012 10:34:26 AM
org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: {add=[eoe32]} 0 6
The important bits are solr.update.processor and the add=blahblah bit
where the stuff after the =
will be a list of <uniqueKey>s for the document(s) added.
However, this will be somewhat fragile, the format of the logged messages is not
guaranteed in future versions.
Although this is happening, I think, after the doc has been added to
the index, so it
may be too late for your problem.
Best
Erick
On Wed, Feb 8, 2012 at 3:13 PM, Hoffman, Chase <Ho...@advisory.com> wrote:
> Please forgive me if this is a dumb question. I've never dealt with SOLR before, and I'm being asked to determine from the logs when a SOLR index is kicked off (it is a Windows server). The TOMCAT service runs continually, so no love there. In parsing the logs, I think "org.apache.solr.core.SolrResourceLoader <init>" is the indicator, since "org.apache.solr.core.SolrCore execute" seems to occur even when I know an index has not been started.
>
> Any advice you could give me would be wonderful.
>
> Best,
>
> --Chase
>
> Chase Hoffman
> Infrastructure Systems Administrator, Performance Technologies
> The Advisory Board Company
> 512-681-2190 direct | 512-609-1150 fax
> hoffmanc@advisory.com<ma...@advisory.com> | www.advisory.com<http://www.advisory.com>
>
> Don't miss out-log in now
> Unlock thousands of members-only tools, events, best practices, and more at www.advisory.com.
> Get started<http://www.advisory.com/reasons-to-log-in-now/?WT.mc_id=eMail|SignatureLine|Other|ABC|Login+8+Reasons|Nov212011>