You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jerome L Quinn <jl...@us.ibm.com> on 2010/01/15 03:59:12 UTC
[1.3] help with update timeout issue?
Hi, folks,
I am using Solr 1.3 pretty successfully, but am running into an issue that
hits once in a long while. I'm still using 1.3 since I have some custom
code I will have to port forward to 1.4.
My basic setup is that I have data sources continually pushing data into
Solr, around 20K adds per day. The index is currently around 100G, stored
on local disk on a fast linux server. I'm trying to make new docs
searchable as quickly as possible, so I currently have autocommit set to
15s. I originally had 3s but that seems to be a little too unstable. I
never optimize the index since optimize will lock things up solid for 2
hours, dropping docs until the optimize completes. I'm using the default
segment merging settings.
Every once in a while I'm getting a socket timeout when trying to add a
document. I traced it to a 20s timeout and then found the corresponding
point in the Solr log.
Jan 13, 2010 2:59:15 PM org.apache.solr.core.SolrCore execute
INFO: [tales] webapp=/solr path=/update params={} status=0 QTime=2
Jan 13, 2010 2:59:15 PM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit(optimize=false,waitFlush=true,waitSearcher=true)
Jan 13, 2010 2:59:56 PM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@26e926e9 main
Jan 13, 2010 2:59:56 PM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
Solr locked up for 41 seconds here while doing some of the commit work.
So, I have a few questions.
Is this related to GC?
Does Solr always lock up when merging segments and I just have to live with
losing the doc I want to add?
Is there a timeout that would guarantee me a write success?
Should I just retry in this situation? If so, how do I distinguish between
this and Solr just being down?
I already have had issues in the past with too many files open, so
increasing the merge factor isn't an option.
On a related note, I had previously asked about optimizing and was told
that segment merging would take care of cleaning up deleted docs. However,
I have the following stats for my index:
numDocs : 2791091
maxDoc : 4811416
My understanding is that numDocs is the docs being searched and maxDoc is
the number of docs including ones that will disappear after optimization.
How do I get this cleanup without using optimize, since it locks up Solr
for multiple hours. I'm deleting old docs daily as well.
Thanks for all the help,
Jerry
Re: [1.3] help with update timeout issue?
Posted by Jerome L Quinn <jl...@us.ibm.com>.
Lance Norskog <go...@gmail.com> wrote on 01/16/2010 12:43:09 AM:
> If your indexing software does not have the ability to retry after a
> failure, you might with to change the timeout from 20 seconds to, say,
> 5 minutes.
I can make it retry, but I have somewhat real-time processes doing these
updates. Does anyone
push updates into a temporary file and then have an async process push the
updates so that it
can survive the lockups without worry? This seems like a real hack, but I
don't want a
long timeout like that in the program that currently pushes the data.
One thing that worries me is that solr may not respond to searches in these
windows. I'm basing
that on the observation that search does not respond when solr is
optimizing.
Can anyone offer me insight on why these delays happen?
Thanks,
Jerry
Re: [1.3] help with update timeout issue?
Posted by Lance Norskog <go...@gmail.com>.
If your indexing software does not have the ability to retry after a
failure, you might with to change the timeout from 20 seconds to, say,
5 minutes.
On Fri, Jan 15, 2010 at 1:20 PM, Jerome L Quinn <jl...@us.ibm.com> wrote:
> Otis Gospodnetic <ot...@yahoo.com> wrote on 01/14/2010 10:07:15
> PM:
>
>> See those "waitFlush=true,waitSearcher=true" ? Do things improve if
>> you make them false? (not sure how with autocommit without looking
>> at the config and not sure if this makes a difference when
>> autocommit triggers commits)
>
> Looking at DirectUpdateHandler2, it appears that those values are hardwired
> to true for autocommit. Unless there's another mechanism for changing
> that.
>
>> Re deleted docs, they are probably getting expunged, it's just that
>> you always have more deleted docs, so those 2 numbers will never be
>> the same without optimize.
>
> I can accept that they will always be different, but that's a large
> difference.
> Hmm, a couple weeks ago, I manually deleted a bunch of docs that had
> associated
> data get corrupted. Normally, I'd only be deleting a day's worth of docs
> at
> a time. Is there a time I could expect the old stuff to get cleaned up by
> without optimizing?
>
> Thanks,
> Jerry
--
Lance Norskog
goksron@gmail.com
Re: [1.3] help with update timeout issue?
Posted by Jerome L Quinn <jl...@us.ibm.com>.
Otis Gospodnetic <ot...@yahoo.com> wrote on 01/14/2010 10:07:15
PM:
> See those "waitFlush=true,waitSearcher=true" ? Do things improve if
> you make them false? (not sure how with autocommit without looking
> at the config and not sure if this makes a difference when
> autocommit triggers commits)
Looking at DirectUpdateHandler2, it appears that those values are hardwired
to true for autocommit. Unless there's another mechanism for changing
that.
> Re deleted docs, they are probably getting expunged, it's just that
> you always have more deleted docs, so those 2 numbers will never be
> the same without optimize.
I can accept that they will always be different, but that's a large
difference.
Hmm, a couple weeks ago, I manually deleted a bunch of docs that had
associated
data get corrupted. Normally, I'd only be deleting a day's worth of docs
at
a time. Is there a time I could expect the old stuff to get cleaned up by
without optimizing?
Thanks,
Jerry
Re: [1.3] help with update timeout issue?
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Jerome,
See those "waitFlush=true,waitSearcher=true" ? Do things improve if you make them false? (not sure how with autocommit without looking at the config and not sure if this makes a difference when autocommit triggers commits)
Re deleted docs, they are probably getting expunged, it's just that you always have more deleted docs, so those 2 numbers will never be the same without optimize.
Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
----- Original Message ----
> From: Jerome L Quinn <jl...@us.ibm.com>
> To: solr-user@lucene.apache.org
> Sent: Thu, January 14, 2010 9:59:12 PM
> Subject: [1.3] help with update timeout issue?
>
>
>
> Hi, folks,
>
> I am using Solr 1.3 pretty successfully, but am running into an issue that
> hits once in a long while. I'm still using 1.3 since I have some custom
> code I will have to port forward to 1.4.
>
> My basic setup is that I have data sources continually pushing data into
> Solr, around 20K adds per day. The index is currently around 100G, stored
> on local disk on a fast linux server. I'm trying to make new docs
> searchable as quickly as possible, so I currently have autocommit set to
> 15s. I originally had 3s but that seems to be a little too unstable. I
> never optimize the index since optimize will lock things up solid for 2
> hours, dropping docs until the optimize completes. I'm using the default
> segment merging settings.
>
> Every once in a while I'm getting a socket timeout when trying to add a
> document. I traced it to a 20s timeout and then found the corresponding
> point in the Solr log.
>
> Jan 13, 2010 2:59:15 PM org.apache.solr.core.SolrCore execute
> INFO: [tales] webapp=/solr path=/update params={} status=0 QTime=2
> Jan 13, 2010 2:59:15 PM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: start commit(optimize=false,waitFlush=true,waitSearcher=true)
> Jan 13, 2010 2:59:56 PM org.apache.solr.search.SolrIndexSearcher
> INFO: Opening Searcher@26e926e9 main
> Jan 13, 2010 2:59:56 PM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: end_commit_flush
>
> Solr locked up for 41 seconds here while doing some of the commit work.
> So, I have a few questions.
>
> Is this related to GC?
> Does Solr always lock up when merging segments and I just have to live with
> losing the doc I want to add?
> Is there a timeout that would guarantee me a write success?
> Should I just retry in this situation? If so, how do I distinguish between
> this and Solr just being down?
> I already have had issues in the past with too many files open, so
> increasing the merge factor isn't an option.
>
>
> On a related note, I had previously asked about optimizing and was told
> that segment merging would take care of cleaning up deleted docs. However,
> I have the following stats for my index:
>
> numDocs : 2791091
> maxDoc : 4811416
>
> My understanding is that numDocs is the docs being searched and maxDoc is
> the number of docs including ones that will disappear after optimization.
> How do I get this cleanup without using optimize, since it locks up Solr
> for multiple hours. I'm deleting old docs daily as well.
>
> Thanks for all the help,
> Jerry
Re: [1.3] help with update timeout issue?
Posted by Andre Parodi <an...@gmail.com>.
add these to your JAVA_OPTS when you start your jvm.
-verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails
-Xloggc:/your/path/verbose-gc.log
tail the verbose gc log to see if the timing of your pause corresponds
with a full gc.
On 15/01/10 03:59, Jerome L Quinn wrote:
> Is this related to GC?
>
Re: Stripping Punctuation in a fieldType
Posted by Robert Muir <rc...@gmail.com>.
hello, RTL languages like arabic are encoded "logically" in
left-to-right order, too. only your display swaps the order.
On Fri, Jan 15, 2010 at 2:07 PM, David Seltzer <ds...@tveyes.com> wrote:
> Hi Erik,
>
> Thanks for your thoughtful reply!
>
>> It's actually quite rare for simple tokenizers like these to be
> satisfactory
>> unless it's a field you can guarantee is indexed/searched in a very
>> controlled manner, say part numbers or words from a list. In your
>> example above, none of the three variants would get a hit if the
>> user searched for "nation". Is that what you want?
>
> Yes, this is what I want. The reason for this behavior is that the
> output of SOLR needs to closely match the search results provided by a
> different legacy system. Our user have rigidly defined queries. A user
> who was interested in "nation's" is required either to search for
> "nations" or "nation*".
>
>> But no, Standard* don't have any stemming built in. And
>> what do you mean by "language specific functionality"?
>> They do NOT fold accents for instance if that's what
>> you're getting at.
>
> I asked that because I'm not super comfortable I know what's going on
> under the hood inside these tokenizers. Do they work the same on
> RightToLeft languages (such as Arabic) as they do in LeftToRight
> languages? (My assumption regarding the WhiteSpaceTokenizer is that it
> would be very language/direction neutral)
>
>> Could you explain a bit about *why* you want this behavior?
> In short we have to support multiple languages and match the behavior of
> an existing non-solr system.
>
> -Dave
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Friday, January 15, 2010 1:42 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Stripping Punctuation in a fieldType
>
> If you haven't seen it, this page is invaluable for this kind of
> question:
> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LetterT
> okenizerFactory
> <http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.Letter
> TokenizerFactory>
>
> LetterTokenizerFactory may well be your friend here, followed by
> LowerCaserFilterFactory. There is a problem that it would
> split "nation's" up into "nation" and "s", so searching on "nations"
> wouldn't get a hit.
>
> But you have equally ugly stuff with WhiteSpaceTokenizerFactory
> as you're finding out.
>
> It's actually quite rare for simple tokenizers like these to be
> satisfactory
> unless it's a field you can guarantee is indexed/searched in a very
> controlled manner, say part numbers or words from a list. In your
> example above, none of the three variants would get a hit if the
> user searched for "nation". Is that what you want?
>
> But no, Standard* don't have any stemming built in. And
> what do you mean by "language specific functionality"?
> They do NOT fold accents for instance if that's what
> you're getting at.
>
> Could you explain a bit about *why* you want this behavior?
>
> HTH
> Erick
>
> On Fri, Jan 15, 2010 at 1:17 PM, David Seltzer <ds...@tveyes.com>
> wrote:
>
>> I'm hesitant to change Tokenizers at the moment because what we have
> is
>> working so nicely - or so I thought.
>>
>> What I'm looking for is case-insensitive search for words and numbers
>> without any of the stemming features turned on. The new requirement is
>> that we take punctuation out of the mix.
>>
>> Right now when I search for "Obama" I'm not getting any hits on
> "Obama."
>>
>> So I'm basically looking to strip punctuation. The consequence would
> be
>> that "nation's", "nations" and "nations," would all be represented the
>> same way.
>>
>> Would the StandardTokenizerFactory accomplish this?
>> Does it have any language specific functionality?
>> Does it do anything with stemming?
>>
>> Thanks for everyone's input!
>>
>> -Dave
>>
>>
>>
>> -----Original Message-----
>> From: Ahmet Arslan [mailto:iorixxx@yahoo.com]
>> Sent: Friday, January 15, 2010 12:42 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Stripping Punctuation in a fieldType
>>
>> > I'm trying to find the best way to set up a fieldType that
>> > strips punctuation.
>>
>> Use solr.StandardTokenizerFactory that strips punctuations.
>>
>> Or if you do not care about alphanumeric or numeric queries use
>> solr.LowerCaseTokenizerFactory that uses LetterTokenizer.
>>
>> I think the right way to do this is using a
>> > CharacterFilter
>> > of some type, but I can't seem to find any examples of how
>> > to set this
>> > up in a schema.xml file.
>>
>> If you want to use solr.MappingCharFilterFactory you need to write all
>> punctiation characters to a text file manually. e.g. "," => ""
>>
>>
>>
>>
>
--
Robert Muir
rcmuir@gmail.com
Re: Stripping Punctuation in a fieldType
Posted by Robert Muir <rc...@gmail.com>.
also, if you are really concerned about different languages, but can
use solr 1.5 then take a look at Unicode Collation.
you can simply add
<filter class="solr.CollationKeyFilterFactory"
language=""
strength="primary"
/>
after your tokenizer and ignore case,accents,punctuation in a
reasonable way for all languages.
http://wiki.apache.org/solr/UnicodeCollation
On Fri, Jan 15, 2010 at 2:31 PM, Erick Erickson <er...@gmail.com> wrote:
> Ah, ok, your approach makes sense. Mostly I was trying
> to insure that you weren't flying blind.
>
> Perhaps you would find some joy with
> PatternReplaceCharFilterFactory, replacing
> all non-alphanum with empty string?
>
> HTH
> Erick
>
> On Fri, Jan 15, 2010 at 2:07 PM, David Seltzer <ds...@tveyes.com> wrote:
>
>> Hi Erik,
>>
>> Thanks for your thoughtful reply!
>>
>> > It's actually quite rare for simple tokenizers like these to be
>> satisfactory
>> > unless it's a field you can guarantee is indexed/searched in a very
>> > controlled manner, say part numbers or words from a list. In your
>> > example above, none of the three variants would get a hit if the
>> > user searched for "nation". Is that what you want?
>>
>> Yes, this is what I want. The reason for this behavior is that the
>> output of SOLR needs to closely match the search results provided by a
>> different legacy system. Our user have rigidly defined queries. A user
>> who was interested in "nation's" is required either to search for
>> "nations" or "nation*".
>>
>> > But no, Standard* don't have any stemming built in. And
>> > what do you mean by "language specific functionality"?
>> > They do NOT fold accents for instance if that's what
>> > you're getting at.
>>
>> I asked that because I'm not super comfortable I know what's going on
>> under the hood inside these tokenizers. Do they work the same on
>> RightToLeft languages (such as Arabic) as they do in LeftToRight
>> languages? (My assumption regarding the WhiteSpaceTokenizer is that it
>> would be very language/direction neutral)
>>
>> > Could you explain a bit about *why* you want this behavior?
>> In short we have to support multiple languages and match the behavior of
>> an existing non-solr system.
>>
>> -Dave
>>
>> -----Original Message-----
>> From: Erick Erickson [mailto:erickerickson@gmail.com]
>> Sent: Friday, January 15, 2010 1:42 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Stripping Punctuation in a fieldType
>>
>> If you haven't seen it, this page is invaluable for this kind of
>> question:
>> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LetterT
>> okenizerFactory
>> <http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.Letter
>> TokenizerFactory>
>>
>> LetterTokenizerFactory may well be your friend here, followed by
>> LowerCaserFilterFactory. There is a problem that it would
>> split "nation's" up into "nation" and "s", so searching on "nations"
>> wouldn't get a hit.
>>
>> But you have equally ugly stuff with WhiteSpaceTokenizerFactory
>> as you're finding out.
>>
>> It's actually quite rare for simple tokenizers like these to be
>> satisfactory
>> unless it's a field you can guarantee is indexed/searched in a very
>> controlled manner, say part numbers or words from a list. In your
>> example above, none of the three variants would get a hit if the
>> user searched for "nation". Is that what you want?
>>
>> But no, Standard* don't have any stemming built in. And
>> what do you mean by "language specific functionality"?
>> They do NOT fold accents for instance if that's what
>> you're getting at.
>>
>> Could you explain a bit about *why* you want this behavior?
>>
>> HTH
>> Erick
>>
>> On Fri, Jan 15, 2010 at 1:17 PM, David Seltzer <ds...@tveyes.com>
>> wrote:
>>
>> > I'm hesitant to change Tokenizers at the moment because what we have
>> is
>> > working so nicely - or so I thought.
>> >
>> > What I'm looking for is case-insensitive search for words and numbers
>> > without any of the stemming features turned on. The new requirement is
>> > that we take punctuation out of the mix.
>> >
>> > Right now when I search for "Obama" I'm not getting any hits on
>> "Obama."
>> >
>> > So I'm basically looking to strip punctuation. The consequence would
>> be
>> > that "nation's", "nations" and "nations," would all be represented the
>> > same way.
>> >
>> > Would the StandardTokenizerFactory accomplish this?
>> > Does it have any language specific functionality?
>> > Does it do anything with stemming?
>> >
>> > Thanks for everyone's input!
>> >
>> > -Dave
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Ahmet Arslan [mailto:iorixxx@yahoo.com]
>> > Sent: Friday, January 15, 2010 12:42 PM
>> > To: solr-user@lucene.apache.org
>> > Subject: Re: Stripping Punctuation in a fieldType
>> >
>> > > I'm trying to find the best way to set up a fieldType that
>> > > strips punctuation.
>> >
>> > Use solr.StandardTokenizerFactory that strips punctuations.
>> >
>> > Or if you do not care about alphanumeric or numeric queries use
>> > solr.LowerCaseTokenizerFactory that uses LetterTokenizer.
>> >
>> > I think the right way to do this is using a
>> > > CharacterFilter
>> > > of some type, but I can't seem to find any examples of how
>> > > to set this
>> > > up in a schema.xml file.
>> >
>> > If you want to use solr.MappingCharFilterFactory you need to write all
>> > punctiation characters to a text file manually. e.g. "," => ""
>> >
>> >
>> >
>> >
>>
>
--
Robert Muir
rcmuir@gmail.com
Re: Stripping Punctuation in a fieldType
Posted by Lance Norskog <go...@gmail.com>.
If your user query syntax has a small number of features, you could
write your own query parser.
On Fri, Jan 15, 2010 at 4:18 PM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:
> David,
>
> PatternReplaceCharFilterFactory accepts pattern and replacement args.
> Please read PatternReplaceCharFilter javadoc to see few samples:
>
> http://lucene.apache.org/solr/api/org/apache/solr/analysis/PatternReplaceCharFilter.html
>
> Koji
>
> --
> http://www.rondhuit.com/en/
>
>
>
> David Seltzer wrote:
>>
>> Does anyone out there know how to use PatternReplaceCharFilterFactory?
>>
>> The closest think to an example I see is in the default schema.xml:
>> <!--
>> The PatternReplaceFilter gives you the flexibility to use
>> Java Regular expression to replace any sequence of
>> characters
>> matching a pattern with an arbitrary replacement string,
>> which may include back references to portions of the
>> original
>> string matched by the pattern.
>> See the Java Regular Expression documentation for
>> more
>> information on pattern and replacement string syntax.
>>
>> http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/package-summary.
>> html
>> -->
>> <filter class="solr.PatternReplaceFilterFactory" pattern="([^a-z])"
>> replacement="" replace="all"/>
>>
>> I'm not sure how the PatternReplaceCharFilterFactory differs from the
>> PatternReplaceFilterFactory. Can anyone give me an example of how to
>> strip all commas for example using this technique?
>>
>> Thanks!
>>
>> -Dave
>>
>
>
>
--
Lance Norskog
goksron@gmail.com
Re: Stripping Punctuation in a fieldType
Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
David,
PatternReplaceCharFilterFactory accepts pattern and replacement args.
Please read PatternReplaceCharFilter javadoc to see few samples:
http://lucene.apache.org/solr/api/org/apache/solr/analysis/PatternReplaceCharFilter.html
Koji
--
http://www.rondhuit.com/en/
David Seltzer wrote:
> Does anyone out there know how to use PatternReplaceCharFilterFactory?
>
> The closest think to an example I see is in the default schema.xml:
> <!--
> The PatternReplaceFilter gives you the flexibility to use
> Java Regular expression to replace any sequence of
> characters
> matching a pattern with an arbitrary replacement string,
> which may include back references to portions of the
> original
> string matched by the pattern.
>
> See the Java Regular Expression documentation for more
> information on pattern and replacement string syntax.
>
>
> http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/package-summary.
> html
>
> -->
> <filter class="solr.PatternReplaceFilterFactory" pattern="([^a-z])"
> replacement="" replace="all"/>
>
> I'm not sure how the PatternReplaceCharFilterFactory differs from the
> PatternReplaceFilterFactory. Can anyone give me an example of how to
> strip all commas for example using this technique?
>
> Thanks!
>
> -Dave
>
Re: Index Courruption after replication by new Solr 1.4 Replication
Posted by Chris Hostetter <ho...@fucit.org>.
: Subject: Index Courruption after replication by new Solr 1.4 Replication
: References: <3C...@mail02.tveyes.com>
: <66...@web52905.mail.re2.yahoo.com>
: <3C...@mail02.tveyes.com>
: <35...@mail.gmail.com>
: <3C...@mail02.tveyes.com>
: <35...@mail.gmail.com>
: <3C...@mail02.tveyes.com>
: In-Reply-To: <3C...@mail02.tveyes.com>
http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which thread
you replied to and your question is "hidden" in that thread and gets less
attention. It makes following discussions in the mailing list archives
particularly difficult.
See Also: http://en.wikipedia.org/wiki/User:DonDiego/Thread_hijacking
-Hoss
RE: Index Courruption after replication by new Solr 1.4 Replication
Posted by bramsreddy <br...@gmail.com>.
Hi,
Is this already registered as bug.Is their any fix to this issue(i want use
EmbeddedSolrServer server only).
Regards,
Ram.
--
View this message in context: http://lucene.472066.n3.nabble.com/1-3-help-with-update-timeout-issue-tp505766p3310048.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: Index Courruption after replication by new Solr 1.4 Replication
Posted by Osborn Chan <oc...@shutterfly.com>.
Hi All,
I found out there is file corruption issue by using both "EmbeddedSolrServer" & "Solr 1.4 Java based replication" together in slave server.
In my slave server, I have 2 webapps in a tomcat instance.
1) "multicore" webapp with slave config
2) "my custom" webapp using EmbeddedSolrServer while queries Solr Index Data.
Both webapps were set up according to the instruction from Solr wiki.
However, I found out there are multi-threading issue which cause index file corruption.
The following is the root case:
EmbeddedSolrServer requires to have a CoreContainer object as parameter. However, during the creation of CoreContainer object, the process load the slave solr configuration which silently creates an Extra ReplcationHandler (SnapPuller) in background. However, there is a ReplcationHandler (SnapPuller) already created by multicore webapp because of the slave configuration.
As a result, there are 2 threads doing file replication as same time. It causes index corruption with different IOExceptions.
After I replaced the usage of EmbeddedSolrServer with CommonsHttpSolrServer (Stop creating CoreContainer object in slave server), Solr 1.4 Java based replication work perfectly without having any file corruption issue.
In other to use EmbeddedSolrServer in slave server, I think we need to have a way to create CoreContainer object with slave configuration without creating extra thread to replicate files.
Should I file a bug?
Thanks,
Osborn
-----Original Message-----
From: Osborn Chan [mailto:ochan@shutterfly.com]
Sent: Friday, January 15, 2010 12:35 PM
To: solr-user@lucene.apache.org
Subject: RE: Index Courruption after replication by new Solr 1.4 Replication
Hi Otis,
Thanks. There is no NFS anymore, and all index files are local. We migrated to new Solr 1.4 new Replication in order to avoid all the NSF Stale Exception.
Thanks,
Osborn
-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
Sent: Friday, January 15, 2010 12:31 PM
To: solr-user@lucene.apache.org
Subject: Re: Index Courruption after replication by new Solr 1.4 Replication
This is not a direct answer to your question, but can you avoid NFS? My first guess would be that NFS somehow causes this problem. If you check the ML archives for: NFS lock , you will see what I mean.
Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
----- Original Message ----
> From: Osborn Chan <oc...@shutterfly.com>
> To: "solr-user@lucene.apache.org" <so...@lucene.apache.org>
> Sent: Fri, January 15, 2010 3:23:21 PM
> Subject: Index Courruption after replication by new Solr 1.4 Replication
>
> Hi all,
>
> I have migrated new Solr 1.4 Replication feature with multicore support from
> Solr 1.2 with NFS mounting recently. The following exceptions are in
> catalina.log from time to time, and there are some EOF exceptions which I
> believe the slave index files are corrupted after replication from index server.
> I have following configuration with Solr 1.4, please correct me if it is
> configured incorrectly.
>
> (The index files are not corrupted in master servers, but it is corrupted in
> slave servers. Usually only one of the slave servers are corrupted with EOF
> exception, but not all.)
>
> 1 Master Server: (Index Server)
> - 8 indexes with multicore configuration.
> - All indexes are configured to "replicateAfter" optimize only.
> - The size of index data are vary. The smallest index only have 2.5 MB. The
> biggest index have ~ 100 MB.
> - There would be infrequent optimize calls to indexes. (a optimize call
> every ~30 mins to 6 hours depending on indexes).
> - There are many commit calls to all indexes. (But there is no concurrent
> commit and optimize for all indexes.)
> - Did not configure "commitReserveDuration" in ReplicationHandler - Using
> default values.
>
> 4 Slave Servers (Search Server)
> - 8 indexes with multicore configuration.
> - All indexes are configured to poll for every ~15 minutes.
> - All update handler configuration are removed in solrconfig-slave.xml
> (solrconfig.xml) in order to prevent add/commit/optimize calls.
> - (Search Slave Servers are only responsible for search operation.)
> - removed.
> -
> removed.
> -
> class="solr.BinaryUpdateRequestHandler" /> removed.
>
> A) FileNotFoundException
>
> INFO: Total time taken for download : 1 secs
> Jan 15, 2010 10:34:16 AM org.apache.solr.handler.ReplicationHandler doFetch
> SEVERE: SnapPull failed
> org.apache.solr.common.SolrException: Index fetch failed :
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
> at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> Caused by: java.io.FileNotFoundException: File does not exist
> /slaveIndexData/publicGalleryTagDef/index.20100115103415/_al.fdx
> at org.apache.solr.common.util.FileUtils.sync(FileUtils.java:55)
> at
> org.apache.solr.handler.SnapPuller$FileFetcher$1.run(SnapPuller.java:911)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
> at java.util.concurrent.FutureTask.run(FutureTask.java:123)
> ... 3 more
> Jan 15, 2010 10:34:17 AM org.apache.solr.core.SolrCore execute
> INFO: [publicGalleryPostMaster] webapp=/multicore path=/select
> params={wt=javabin&rows=10&start=0&sort=createTime_dt+desc&q=%2B(profileId_s:/community/sfly/publicprofile/0AcM27Nw3aNWLi4)+%2Bstate_s:A&version=1}
> hits=1 status=0 QTime=1
>
> B) LockReleaseFailedException
>
> SEVERE: SnapPull failed
> org.apache.solr.common.SolrException: Index fetch failed :
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
> at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> Caused by: org.apache.lucene.store.LockReleaseFailedException: failed to delete
> /slaveIndexData/publicGalleryTagDefAggregate/index/lucene-fb30bdbbdc6927666873dd616884ba29-write.lock
> at
> org.apache.lucene.store.NativeFSLock.release(NativeFSLockFactory.java:298)
> at
> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:2225)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2153)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2117)
> at
> org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:229)
> at
> org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:181)
> at
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:409)
> at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:467)
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
> ... 11 more
> Jan 15, 2010 12:21:18 AM org.apache.solr.handler.SnapPuller fetchLatestIndex
> INFO: Slave in sync with master.
>
> C) EOF Exception
> INFO: [publicGalleryPostMaster] webapp=/multicore path=/select
> params={wt=javabin&rows=1&start=0&sort=createTime_dt+desc&q=%2B(profileId_s:/community/sfly/publicprofile/0AbOWLNszaOWTiw)+%2B(lastBookmarked_dt:[2010-01-08T08:49:38.271Z+TO+2010-01-15T08:49:38.271Z]+lastCommented_dt:[2010-01-08T08:49:38.271Z+TO+2010-01-15T08:49:38.271Z])+%2Bstate_s:A&version=1}
> hits=0 status=0 QTime=2
> Jan 15, 2010 12:49:42 AM org.apache.solr.common.SolrException log
> SEVERE: java.io.IOException: read past EOF
> at
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
> at
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
> at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:80)
> at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:64)
> at
> org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:129)
> at
> org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:160)
> at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:232)
> at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:179)
> at org.apache.lucene.index.SegmentReader.docFreq(SegmentReader.java:975)
> at
> org.apache.lucene.index.DirectoryReader.docFreq(DirectoryReader.java:627)
> at
> org.apache.solr.search.SolrIndexReader.docFreq(SolrIndexReader.java:308)
> at
> org.apache.lucene.search.IndexSearcher.docFreq(IndexSearcher.java:147)
> at org.apache.lucene.search.Similarity.idfExplain(Similarity.java:833)
> at
> org.apache.lucene.search.PhraseQuery$PhraseWeight.(PhraseQuery.java:122)
> at
> org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:250)
> at
> org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:184)
> at
> org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:415)
> at org.apache.lucene.search.Query.weight(Query.java:99)
> at org.apache.lucene.search.Searcher.createWeight(Searcher.java:230)
> at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
> at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
> at java.lang.Thread.run(Thread.java:595)
>
> Thanks a lot!
>
> Osborn
RE: Index Courruption after replication by new Solr 1.4 Replication
Posted by Osborn Chan <oc...@shutterfly.com>.
Hi Otis,
Thanks. There is no NFS anymore, and all index files are local. We migrated to new Solr 1.4 new Replication in order to avoid all the NSF Stale Exception.
Thanks,
Osborn
-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
Sent: Friday, January 15, 2010 12:31 PM
To: solr-user@lucene.apache.org
Subject: Re: Index Courruption after replication by new Solr 1.4 Replication
This is not a direct answer to your question, but can you avoid NFS? My first guess would be that NFS somehow causes this problem. If you check the ML archives for: NFS lock , you will see what I mean.
Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
----- Original Message ----
> From: Osborn Chan <oc...@shutterfly.com>
> To: "solr-user@lucene.apache.org" <so...@lucene.apache.org>
> Sent: Fri, January 15, 2010 3:23:21 PM
> Subject: Index Courruption after replication by new Solr 1.4 Replication
>
> Hi all,
>
> I have migrated new Solr 1.4 Replication feature with multicore support from
> Solr 1.2 with NFS mounting recently. The following exceptions are in
> catalina.log from time to time, and there are some EOF exceptions which I
> believe the slave index files are corrupted after replication from index server.
> I have following configuration with Solr 1.4, please correct me if it is
> configured incorrectly.
>
> (The index files are not corrupted in master servers, but it is corrupted in
> slave servers. Usually only one of the slave servers are corrupted with EOF
> exception, but not all.)
>
> 1 Master Server: (Index Server)
> - 8 indexes with multicore configuration.
> - All indexes are configured to "replicateAfter" optimize only.
> - The size of index data are vary. The smallest index only have 2.5 MB. The
> biggest index have ~ 100 MB.
> - There would be infrequent optimize calls to indexes. (a optimize call
> every ~30 mins to 6 hours depending on indexes).
> - There are many commit calls to all indexes. (But there is no concurrent
> commit and optimize for all indexes.)
> - Did not configure "commitReserveDuration" in ReplicationHandler - Using
> default values.
>
> 4 Slave Servers (Search Server)
> - 8 indexes with multicore configuration.
> - All indexes are configured to poll for every ~15 minutes.
> - All update handler configuration are removed in solrconfig-slave.xml
> (solrconfig.xml) in order to prevent add/commit/optimize calls.
> - (Search Slave Servers are only responsible for search operation.)
> - removed.
> -
> removed.
> -
> class="solr.BinaryUpdateRequestHandler" /> removed.
>
> A) FileNotFoundException
>
> INFO: Total time taken for download : 1 secs
> Jan 15, 2010 10:34:16 AM org.apache.solr.handler.ReplicationHandler doFetch
> SEVERE: SnapPull failed
> org.apache.solr.common.SolrException: Index fetch failed :
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
> at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> Caused by: java.io.FileNotFoundException: File does not exist
> /slaveIndexData/publicGalleryTagDef/index.20100115103415/_al.fdx
> at org.apache.solr.common.util.FileUtils.sync(FileUtils.java:55)
> at
> org.apache.solr.handler.SnapPuller$FileFetcher$1.run(SnapPuller.java:911)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
> at java.util.concurrent.FutureTask.run(FutureTask.java:123)
> ... 3 more
> Jan 15, 2010 10:34:17 AM org.apache.solr.core.SolrCore execute
> INFO: [publicGalleryPostMaster] webapp=/multicore path=/select
> params={wt=javabin&rows=10&start=0&sort=createTime_dt+desc&q=%2B(profileId_s:/community/sfly/publicprofile/0AcM27Nw3aNWLi4)+%2Bstate_s:A&version=1}
> hits=1 status=0 QTime=1
>
> B) LockReleaseFailedException
>
> SEVERE: SnapPull failed
> org.apache.solr.common.SolrException: Index fetch failed :
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
> at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> Caused by: org.apache.lucene.store.LockReleaseFailedException: failed to delete
> /slaveIndexData/publicGalleryTagDefAggregate/index/lucene-fb30bdbbdc6927666873dd616884ba29-write.lock
> at
> org.apache.lucene.store.NativeFSLock.release(NativeFSLockFactory.java:298)
> at
> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:2225)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2153)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2117)
> at
> org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:229)
> at
> org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:181)
> at
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:409)
> at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:467)
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
> ... 11 more
> Jan 15, 2010 12:21:18 AM org.apache.solr.handler.SnapPuller fetchLatestIndex
> INFO: Slave in sync with master.
>
> C) EOF Exception
> INFO: [publicGalleryPostMaster] webapp=/multicore path=/select
> params={wt=javabin&rows=1&start=0&sort=createTime_dt+desc&q=%2B(profileId_s:/community/sfly/publicprofile/0AbOWLNszaOWTiw)+%2B(lastBookmarked_dt:[2010-01-08T08:49:38.271Z+TO+2010-01-15T08:49:38.271Z]+lastCommented_dt:[2010-01-08T08:49:38.271Z+TO+2010-01-15T08:49:38.271Z])+%2Bstate_s:A&version=1}
> hits=0 status=0 QTime=2
> Jan 15, 2010 12:49:42 AM org.apache.solr.common.SolrException log
> SEVERE: java.io.IOException: read past EOF
> at
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
> at
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
> at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:80)
> at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:64)
> at
> org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:129)
> at
> org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:160)
> at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:232)
> at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:179)
> at org.apache.lucene.index.SegmentReader.docFreq(SegmentReader.java:975)
> at
> org.apache.lucene.index.DirectoryReader.docFreq(DirectoryReader.java:627)
> at
> org.apache.solr.search.SolrIndexReader.docFreq(SolrIndexReader.java:308)
> at
> org.apache.lucene.search.IndexSearcher.docFreq(IndexSearcher.java:147)
> at org.apache.lucene.search.Similarity.idfExplain(Similarity.java:833)
> at
> org.apache.lucene.search.PhraseQuery$PhraseWeight.(PhraseQuery.java:122)
> at
> org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:250)
> at
> org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:184)
> at
> org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:415)
> at org.apache.lucene.search.Query.weight(Query.java:99)
> at org.apache.lucene.search.Searcher.createWeight(Searcher.java:230)
> at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
> at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
> at java.lang.Thread.run(Thread.java:595)
>
> Thanks a lot!
>
> Osborn
Re: Index Courruption after replication by new Solr 1.4 Replication
Posted by Otis Gospodnetic <ot...@yahoo.com>.
This is not a direct answer to your question, but can you avoid NFS? My first guess would be that NFS somehow causes this problem. If you check the ML archives for: NFS lock , you will see what I mean.
Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
----- Original Message ----
> From: Osborn Chan <oc...@shutterfly.com>
> To: "solr-user@lucene.apache.org" <so...@lucene.apache.org>
> Sent: Fri, January 15, 2010 3:23:21 PM
> Subject: Index Courruption after replication by new Solr 1.4 Replication
>
> Hi all,
>
> I have migrated new Solr 1.4 Replication feature with multicore support from
> Solr 1.2 with NFS mounting recently. The following exceptions are in
> catalina.log from time to time, and there are some EOF exceptions which I
> believe the slave index files are corrupted after replication from index server.
> I have following configuration with Solr 1.4, please correct me if it is
> configured incorrectly.
>
> (The index files are not corrupted in master servers, but it is corrupted in
> slave servers. Usually only one of the slave servers are corrupted with EOF
> exception, but not all.)
>
> 1 Master Server: (Index Server)
> - 8 indexes with multicore configuration.
> - All indexes are configured to "replicateAfter" optimize only.
> - The size of index data are vary. The smallest index only have 2.5 MB. The
> biggest index have ~ 100 MB.
> - There would be infrequent optimize calls to indexes. (a optimize call
> every ~30 mins to 6 hours depending on indexes).
> - There are many commit calls to all indexes. (But there is no concurrent
> commit and optimize for all indexes.)
> - Did not configure "commitReserveDuration" in ReplicationHandler - Using
> default values.
>
> 4 Slave Servers (Search Server)
> - 8 indexes with multicore configuration.
> - All indexes are configured to poll for every ~15 minutes.
> - All update handler configuration are removed in solrconfig-slave.xml
> (solrconfig.xml) in order to prevent add/commit/optimize calls.
> - (Search Slave Servers are only responsible for search operation.)
> - removed.
> -
> removed.
> -
> class="solr.BinaryUpdateRequestHandler" /> removed.
>
> A) FileNotFoundException
>
> INFO: Total time taken for download : 1 secs
> Jan 15, 2010 10:34:16 AM org.apache.solr.handler.ReplicationHandler doFetch
> SEVERE: SnapPull failed
> org.apache.solr.common.SolrException: Index fetch failed :
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
> at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> Caused by: java.io.FileNotFoundException: File does not exist
> /slaveIndexData/publicGalleryTagDef/index.20100115103415/_al.fdx
> at org.apache.solr.common.util.FileUtils.sync(FileUtils.java:55)
> at
> org.apache.solr.handler.SnapPuller$FileFetcher$1.run(SnapPuller.java:911)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
> at java.util.concurrent.FutureTask.run(FutureTask.java:123)
> ... 3 more
> Jan 15, 2010 10:34:17 AM org.apache.solr.core.SolrCore execute
> INFO: [publicGalleryPostMaster] webapp=/multicore path=/select
> params={wt=javabin&rows=10&start=0&sort=createTime_dt+desc&q=%2B(profileId_s:/community/sfly/publicprofile/0AcM27Nw3aNWLi4)+%2Bstate_s:A&version=1}
> hits=1 status=0 QTime=1
>
> B) LockReleaseFailedException
>
> SEVERE: SnapPull failed
> org.apache.solr.common.SolrException: Index fetch failed :
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
> at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> Caused by: org.apache.lucene.store.LockReleaseFailedException: failed to delete
> /slaveIndexData/publicGalleryTagDefAggregate/index/lucene-fb30bdbbdc6927666873dd616884ba29-write.lock
> at
> org.apache.lucene.store.NativeFSLock.release(NativeFSLockFactory.java:298)
> at
> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:2225)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2153)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2117)
> at
> org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:229)
> at
> org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:181)
> at
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:409)
> at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:467)
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
> ... 11 more
> Jan 15, 2010 12:21:18 AM org.apache.solr.handler.SnapPuller fetchLatestIndex
> INFO: Slave in sync with master.
>
> C) EOF Exception
> INFO: [publicGalleryPostMaster] webapp=/multicore path=/select
> params={wt=javabin&rows=1&start=0&sort=createTime_dt+desc&q=%2B(profileId_s:/community/sfly/publicprofile/0AbOWLNszaOWTiw)+%2B(lastBookmarked_dt:[2010-01-08T08:49:38.271Z+TO+2010-01-15T08:49:38.271Z]+lastCommented_dt:[2010-01-08T08:49:38.271Z+TO+2010-01-15T08:49:38.271Z])+%2Bstate_s:A&version=1}
> hits=0 status=0 QTime=2
> Jan 15, 2010 12:49:42 AM org.apache.solr.common.SolrException log
> SEVERE: java.io.IOException: read past EOF
> at
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
> at
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
> at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:80)
> at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:64)
> at
> org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:129)
> at
> org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:160)
> at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:232)
> at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:179)
> at org.apache.lucene.index.SegmentReader.docFreq(SegmentReader.java:975)
> at
> org.apache.lucene.index.DirectoryReader.docFreq(DirectoryReader.java:627)
> at
> org.apache.solr.search.SolrIndexReader.docFreq(SolrIndexReader.java:308)
> at
> org.apache.lucene.search.IndexSearcher.docFreq(IndexSearcher.java:147)
> at org.apache.lucene.search.Similarity.idfExplain(Similarity.java:833)
> at
> org.apache.lucene.search.PhraseQuery$PhraseWeight.(PhraseQuery.java:122)
> at
> org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:250)
> at
> org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:184)
> at
> org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:415)
> at org.apache.lucene.search.Query.weight(Query.java:99)
> at org.apache.lucene.search.Searcher.createWeight(Searcher.java:230)
> at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
> at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
> at java.lang.Thread.run(Thread.java:595)
>
> Thanks a lot!
>
> Osborn
Index Courruption after replication by new Solr 1.4 Replication
Posted by Osborn Chan <oc...@shutterfly.com>.
Hi all,
I have migrated new Solr 1.4 Replication feature with multicore support from Solr 1.2 with NFS mounting recently. The following exceptions are in catalina.log from time to time, and there are some EOF exceptions which I believe the slave index files are corrupted after replication from index server. I have following configuration with Solr 1.4, please correct me if it is configured incorrectly.
(The index files are not corrupted in master servers, but it is corrupted in slave servers. Usually only one of the slave servers are corrupted with EOF exception, but not all.)
1 Master Server: (Index Server)
- 8 indexes with multicore configuration.
- All indexes are configured to "replicateAfter" optimize only.
- The size of index data are vary. The smallest index only have 2.5 MB. The biggest index have ~ 100 MB.
- There would be infrequent optimize calls to indexes. (a optimize call every ~30 mins to 6 hours depending on indexes).
- There are many commit calls to all indexes. (But there is no concurrent commit and optimize for all indexes.)
- Did not configure "commitReserveDuration" in ReplicationHandler - Using default values.
4 Slave Servers (Search Server)
- 8 indexes with multicore configuration.
- All indexes are configured to poll for every ~15 minutes.
- All update handler configuration are removed in solrconfig-slave.xml (solrconfig.xml) in order to prevent add/commit/optimize calls.
- (Search Slave Servers are only responsible for search operation.)
- <updateHandler class="solr.DirectUpdateHandler2"> removed.
- <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" /> removed.
- <requestHandler name="/update/javabin" class="solr.BinaryUpdateRequestHandler" /> removed.
A) FileNotFoundException
INFO: Total time taken for download : 1 secs
Jan 15, 2010 10:34:16 AM org.apache.solr.handler.ReplicationHandler doFetch
SEVERE: SnapPull failed
org.apache.solr.common.SolrException: Index fetch failed :
at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.FileNotFoundException: File does not exist /slaveIndexData/publicGalleryTagDef/index.20100115103415/_al.fdx
at org.apache.solr.common.util.FileUtils.sync(FileUtils.java:55)
at org.apache.solr.handler.SnapPuller$FileFetcher$1.run(SnapPuller.java:911)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
... 3 more
Jan 15, 2010 10:34:17 AM org.apache.solr.core.SolrCore execute
INFO: [publicGalleryPostMaster] webapp=/multicore path=/select params={wt=javabin&rows=10&start=0&sort=createTime_dt+desc&q=%2B(profileId_s:/community/sfly/publicprofile/0AcM27Nw3aNWLi4)+%2Bstate_s:A&version=1} hits=1 status=0 QTime=1
B) LockReleaseFailedException
SEVERE: SnapPull failed
org.apache.solr.common.SolrException: Index fetch failed :
at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: org.apache.lucene.store.LockReleaseFailedException: failed to delete /slaveIndexData/publicGalleryTagDefAggregate/index/lucene-fb30bdbbdc6927666873dd616884ba29-write.lock
at org.apache.lucene.store.NativeFSLock.release(NativeFSLockFactory.java:298)
at org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:2225)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2153)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2117)
at org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:229)
at org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:181)
at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:409)
at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:467)
at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
... 11 more
Jan 15, 2010 12:21:18 AM org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Slave in sync with master.
C) EOF Exception
INFO: [publicGalleryPostMaster] webapp=/multicore path=/select params={wt=javabin&rows=1&start=0&sort=createTime_dt+desc&q=%2B(profileId_s:/community/sfly/publicprofile/0AbOWLNszaOWTiw)+%2B(lastBookmarked_dt:[2010-01-08T08:49:38.271Z+TO+2010-01-15T08:49:38.271Z]+lastCommented_dt:[2010-01-08T08:49:38.271Z+TO+2010-01-15T08:49:38.271Z])+%2Bstate_s:A&version=1} hits=0 status=0 QTime=2
Jan 15, 2010 12:49:42 AM org.apache.solr.common.SolrException log
SEVERE: java.io.IOException: read past EOF
at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:80)
at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:64)
at org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:129)
at org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:160)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:232)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:179)
at org.apache.lucene.index.SegmentReader.docFreq(SegmentReader.java:975)
at org.apache.lucene.index.DirectoryReader.docFreq(DirectoryReader.java:627)
at org.apache.solr.search.SolrIndexReader.docFreq(SolrIndexReader.java:308)
at org.apache.lucene.search.IndexSearcher.docFreq(IndexSearcher.java:147)
at org.apache.lucene.search.Similarity.idfExplain(Similarity.java:833)
at org.apache.lucene.search.PhraseQuery$PhraseWeight.<init>(PhraseQuery.java:122)
at org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:250)
at org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:184)
at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:415)
at org.apache.lucene.search.Query.weight(Query.java:99)
at org.apache.lucene.search.Searcher.createWeight(Searcher.java:230)
at org.apache.lucene.search.Searcher.search(Searcher.java:171)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
Thanks a lot!
Osborn
RE: Stripping Punctuation in a fieldType
Posted by David Seltzer <ds...@TVEyes.com>.
Does anyone out there know how to use PatternReplaceCharFilterFactory?
The closest think to an example I see is in the default schema.xml:
<!--
The PatternReplaceFilter gives you the flexibility to use
Java Regular expression to replace any sequence of
characters
matching a pattern with an arbitrary replacement string,
which may include back references to portions of the
original
string matched by the pattern.
See the Java Regular Expression documentation for more
information on pattern and replacement string syntax.
http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/package-summary.
html
-->
<filter class="solr.PatternReplaceFilterFactory" pattern="([^a-z])"
replacement="" replace="all"/>
I'm not sure how the PatternReplaceCharFilterFactory differs from the
PatternReplaceFilterFactory. Can anyone give me an example of how to
strip all commas for example using this technique?
Thanks!
-Dave
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Friday, January 15, 2010 2:32 PM
To: solr-user@lucene.apache.org
Subject: Re: Stripping Punctuation in a fieldType
Ah, ok, your approach makes sense. Mostly I was trying
to insure that you weren't flying blind.
Perhaps you would find some joy with
PatternReplaceCharFilterFactory, replacing
all non-alphanum with empty string?
HTH
Erick
On Fri, Jan 15, 2010 at 2:07 PM, David Seltzer <ds...@tveyes.com>
wrote:
> Hi Erik,
>
> Thanks for your thoughtful reply!
>
> > It's actually quite rare for simple tokenizers like these to be
> satisfactory
> > unless it's a field you can guarantee is indexed/searched in a very
> > controlled manner, say part numbers or words from a list. In your
> > example above, none of the three variants would get a hit if the
> > user searched for "nation". Is that what you want?
>
> Yes, this is what I want. The reason for this behavior is that the
> output of SOLR needs to closely match the search results provided by a
> different legacy system. Our user have rigidly defined queries. A user
> who was interested in "nation's" is required either to search for
> "nations" or "nation*".
>
> > But no, Standard* don't have any stemming built in. And
> > what do you mean by "language specific functionality"?
> > They do NOT fold accents for instance if that's what
> > you're getting at.
>
> I asked that because I'm not super comfortable I know what's going on
> under the hood inside these tokenizers. Do they work the same on
> RightToLeft languages (such as Arabic) as they do in LeftToRight
> languages? (My assumption regarding the WhiteSpaceTokenizer is that it
> would be very language/direction neutral)
>
> > Could you explain a bit about *why* you want this behavior?
> In short we have to support multiple languages and match the behavior
of
> an existing non-solr system.
>
> -Dave
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Friday, January 15, 2010 1:42 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Stripping Punctuation in a fieldType
>
> If you haven't seen it, this page is invaluable for this kind of
> question:
>
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LetterT
> okenizerFactory
>
<http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.Letter
> TokenizerFactory>
>
> LetterTokenizerFactory may well be your friend here, followed by
> LowerCaserFilterFactory. There is a problem that it would
> split "nation's" up into "nation" and "s", so searching on "nations"
> wouldn't get a hit.
>
> But you have equally ugly stuff with WhiteSpaceTokenizerFactory
> as you're finding out.
>
> It's actually quite rare for simple tokenizers like these to be
> satisfactory
> unless it's a field you can guarantee is indexed/searched in a very
> controlled manner, say part numbers or words from a list. In your
> example above, none of the three variants would get a hit if the
> user searched for "nation". Is that what you want?
>
> But no, Standard* don't have any stemming built in. And
> what do you mean by "language specific functionality"?
> They do NOT fold accents for instance if that's what
> you're getting at.
>
> Could you explain a bit about *why* you want this behavior?
>
> HTH
> Erick
>
> On Fri, Jan 15, 2010 at 1:17 PM, David Seltzer <ds...@tveyes.com>
> wrote:
>
> > I'm hesitant to change Tokenizers at the moment because what we have
> is
> > working so nicely - or so I thought.
> >
> > What I'm looking for is case-insensitive search for words and
numbers
> > without any of the stemming features turned on. The new requirement
is
> > that we take punctuation out of the mix.
> >
> > Right now when I search for "Obama" I'm not getting any hits on
> "Obama."
> >
> > So I'm basically looking to strip punctuation. The consequence would
> be
> > that "nation's", "nations" and "nations," would all be represented
the
> > same way.
> >
> > Would the StandardTokenizerFactory accomplish this?
> > Does it have any language specific functionality?
> > Does it do anything with stemming?
> >
> > Thanks for everyone's input!
> >
> > -Dave
> >
> >
> >
> > -----Original Message-----
> > From: Ahmet Arslan [mailto:iorixxx@yahoo.com]
> > Sent: Friday, January 15, 2010 12:42 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Stripping Punctuation in a fieldType
> >
> > > I'm trying to find the best way to set up a fieldType that
> > > strips punctuation.
> >
> > Use solr.StandardTokenizerFactory that strips punctuations.
> >
> > Or if you do not care about alphanumeric or numeric queries use
> > solr.LowerCaseTokenizerFactory that uses LetterTokenizer.
> >
> > I think the right way to do this is using a
> > > CharacterFilter
> > > of some type, but I can't seem to find any examples of how
> > > to set this
> > > up in a schema.xml file.
> >
> > If you want to use solr.MappingCharFilterFactory you need to write
all
> > punctiation characters to a text file manually. e.g. "," => ""
> >
> >
> >
> >
>
Re: Stripping Punctuation in a fieldType
Posted by Erick Erickson <er...@gmail.com>.
Ah, ok, your approach makes sense. Mostly I was trying
to insure that you weren't flying blind.
Perhaps you would find some joy with
PatternReplaceCharFilterFactory, replacing
all non-alphanum with empty string?
HTH
Erick
On Fri, Jan 15, 2010 at 2:07 PM, David Seltzer <ds...@tveyes.com> wrote:
> Hi Erik,
>
> Thanks for your thoughtful reply!
>
> > It's actually quite rare for simple tokenizers like these to be
> satisfactory
> > unless it's a field you can guarantee is indexed/searched in a very
> > controlled manner, say part numbers or words from a list. In your
> > example above, none of the three variants would get a hit if the
> > user searched for "nation". Is that what you want?
>
> Yes, this is what I want. The reason for this behavior is that the
> output of SOLR needs to closely match the search results provided by a
> different legacy system. Our user have rigidly defined queries. A user
> who was interested in "nation's" is required either to search for
> "nations" or "nation*".
>
> > But no, Standard* don't have any stemming built in. And
> > what do you mean by "language specific functionality"?
> > They do NOT fold accents for instance if that's what
> > you're getting at.
>
> I asked that because I'm not super comfortable I know what's going on
> under the hood inside these tokenizers. Do they work the same on
> RightToLeft languages (such as Arabic) as they do in LeftToRight
> languages? (My assumption regarding the WhiteSpaceTokenizer is that it
> would be very language/direction neutral)
>
> > Could you explain a bit about *why* you want this behavior?
> In short we have to support multiple languages and match the behavior of
> an existing non-solr system.
>
> -Dave
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Friday, January 15, 2010 1:42 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Stripping Punctuation in a fieldType
>
> If you haven't seen it, this page is invaluable for this kind of
> question:
> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LetterT
> okenizerFactory
> <http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.Letter
> TokenizerFactory>
>
> LetterTokenizerFactory may well be your friend here, followed by
> LowerCaserFilterFactory. There is a problem that it would
> split "nation's" up into "nation" and "s", so searching on "nations"
> wouldn't get a hit.
>
> But you have equally ugly stuff with WhiteSpaceTokenizerFactory
> as you're finding out.
>
> It's actually quite rare for simple tokenizers like these to be
> satisfactory
> unless it's a field you can guarantee is indexed/searched in a very
> controlled manner, say part numbers or words from a list. In your
> example above, none of the three variants would get a hit if the
> user searched for "nation". Is that what you want?
>
> But no, Standard* don't have any stemming built in. And
> what do you mean by "language specific functionality"?
> They do NOT fold accents for instance if that's what
> you're getting at.
>
> Could you explain a bit about *why* you want this behavior?
>
> HTH
> Erick
>
> On Fri, Jan 15, 2010 at 1:17 PM, David Seltzer <ds...@tveyes.com>
> wrote:
>
> > I'm hesitant to change Tokenizers at the moment because what we have
> is
> > working so nicely - or so I thought.
> >
> > What I'm looking for is case-insensitive search for words and numbers
> > without any of the stemming features turned on. The new requirement is
> > that we take punctuation out of the mix.
> >
> > Right now when I search for "Obama" I'm not getting any hits on
> "Obama."
> >
> > So I'm basically looking to strip punctuation. The consequence would
> be
> > that "nation's", "nations" and "nations," would all be represented the
> > same way.
> >
> > Would the StandardTokenizerFactory accomplish this?
> > Does it have any language specific functionality?
> > Does it do anything with stemming?
> >
> > Thanks for everyone's input!
> >
> > -Dave
> >
> >
> >
> > -----Original Message-----
> > From: Ahmet Arslan [mailto:iorixxx@yahoo.com]
> > Sent: Friday, January 15, 2010 12:42 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Stripping Punctuation in a fieldType
> >
> > > I'm trying to find the best way to set up a fieldType that
> > > strips punctuation.
> >
> > Use solr.StandardTokenizerFactory that strips punctuations.
> >
> > Or if you do not care about alphanumeric or numeric queries use
> > solr.LowerCaseTokenizerFactory that uses LetterTokenizer.
> >
> > I think the right way to do this is using a
> > > CharacterFilter
> > > of some type, but I can't seem to find any examples of how
> > > to set this
> > > up in a schema.xml file.
> >
> > If you want to use solr.MappingCharFilterFactory you need to write all
> > punctiation characters to a text file manually. e.g. "," => ""
> >
> >
> >
> >
>
RE: Stripping Punctuation in a fieldType
Posted by David Seltzer <ds...@TVEyes.com>.
Hi Erik,
Thanks for your thoughtful reply!
> It's actually quite rare for simple tokenizers like these to be
satisfactory
> unless it's a field you can guarantee is indexed/searched in a very
> controlled manner, say part numbers or words from a list. In your
> example above, none of the three variants would get a hit if the
> user searched for "nation". Is that what you want?
Yes, this is what I want. The reason for this behavior is that the
output of SOLR needs to closely match the search results provided by a
different legacy system. Our user have rigidly defined queries. A user
who was interested in "nation's" is required either to search for
"nations" or "nation*".
> But no, Standard* don't have any stemming built in. And
> what do you mean by "language specific functionality"?
> They do NOT fold accents for instance if that's what
> you're getting at.
I asked that because I'm not super comfortable I know what's going on
under the hood inside these tokenizers. Do they work the same on
RightToLeft languages (such as Arabic) as they do in LeftToRight
languages? (My assumption regarding the WhiteSpaceTokenizer is that it
would be very language/direction neutral)
> Could you explain a bit about *why* you want this behavior?
In short we have to support multiple languages and match the behavior of
an existing non-solr system.
-Dave
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Friday, January 15, 2010 1:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Stripping Punctuation in a fieldType
If you haven't seen it, this page is invaluable for this kind of
question:
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LetterT
okenizerFactory
<http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.Letter
TokenizerFactory>
LetterTokenizerFactory may well be your friend here, followed by
LowerCaserFilterFactory. There is a problem that it would
split "nation's" up into "nation" and "s", so searching on "nations"
wouldn't get a hit.
But you have equally ugly stuff with WhiteSpaceTokenizerFactory
as you're finding out.
It's actually quite rare for simple tokenizers like these to be
satisfactory
unless it's a field you can guarantee is indexed/searched in a very
controlled manner, say part numbers or words from a list. In your
example above, none of the three variants would get a hit if the
user searched for "nation". Is that what you want?
But no, Standard* don't have any stemming built in. And
what do you mean by "language specific functionality"?
They do NOT fold accents for instance if that's what
you're getting at.
Could you explain a bit about *why* you want this behavior?
HTH
Erick
On Fri, Jan 15, 2010 at 1:17 PM, David Seltzer <ds...@tveyes.com>
wrote:
> I'm hesitant to change Tokenizers at the moment because what we have
is
> working so nicely - or so I thought.
>
> What I'm looking for is case-insensitive search for words and numbers
> without any of the stemming features turned on. The new requirement is
> that we take punctuation out of the mix.
>
> Right now when I search for "Obama" I'm not getting any hits on
"Obama."
>
> So I'm basically looking to strip punctuation. The consequence would
be
> that "nation's", "nations" and "nations," would all be represented the
> same way.
>
> Would the StandardTokenizerFactory accomplish this?
> Does it have any language specific functionality?
> Does it do anything with stemming?
>
> Thanks for everyone's input!
>
> -Dave
>
>
>
> -----Original Message-----
> From: Ahmet Arslan [mailto:iorixxx@yahoo.com]
> Sent: Friday, January 15, 2010 12:42 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Stripping Punctuation in a fieldType
>
> > I'm trying to find the best way to set up a fieldType that
> > strips punctuation.
>
> Use solr.StandardTokenizerFactory that strips punctuations.
>
> Or if you do not care about alphanumeric or numeric queries use
> solr.LowerCaseTokenizerFactory that uses LetterTokenizer.
>
> I think the right way to do this is using a
> > CharacterFilter
> > of some type, but I can't seem to find any examples of how
> > to set this
> > up in a schema.xml file.
>
> If you want to use solr.MappingCharFilterFactory you need to write all
> punctiation characters to a text file manually. e.g. "," => ""
>
>
>
>
Re: Stripping Punctuation in a fieldType
Posted by Erick Erickson <er...@gmail.com>.
If you haven't seen it, this page is invaluable for this kind of question:
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LetterTokenizerFactory
<http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LetterTokenizerFactory>
LetterTokenizerFactory may well be your friend here, followed by
LowerCaserFilterFactory. There is a problem that it would
split "nation's" up into "nation" and "s", so searching on "nations"
wouldn't get a hit.
But you have equally ugly stuff with WhiteSpaceTokenizerFactory
as you're finding out.
It's actually quite rare for simple tokenizers like these to be
satisfactory
unless it's a field you can guarantee is indexed/searched in a very
controlled manner, say part numbers or words from a list. In your
example above, none of the three variants would get a hit if the
user searched for "nation". Is that what you want?
But no, Standard* don't have any stemming built in. And
what do you mean by "language specific functionality"?
They do NOT fold accents for instance if that's what
you're getting at.
Could you explain a bit about *why* you want this behavior?
HTH
Erick
On Fri, Jan 15, 2010 at 1:17 PM, David Seltzer <ds...@tveyes.com> wrote:
> I'm hesitant to change Tokenizers at the moment because what we have is
> working so nicely - or so I thought.
>
> What I'm looking for is case-insensitive search for words and numbers
> without any of the stemming features turned on. The new requirement is
> that we take punctuation out of the mix.
>
> Right now when I search for "Obama" I'm not getting any hits on "Obama."
>
> So I'm basically looking to strip punctuation. The consequence would be
> that "nation's", "nations" and "nations," would all be represented the
> same way.
>
> Would the StandardTokenizerFactory accomplish this?
> Does it have any language specific functionality?
> Does it do anything with stemming?
>
> Thanks for everyone's input!
>
> -Dave
>
>
>
> -----Original Message-----
> From: Ahmet Arslan [mailto:iorixxx@yahoo.com]
> Sent: Friday, January 15, 2010 12:42 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Stripping Punctuation in a fieldType
>
> > I'm trying to find the best way to set up a fieldType that
> > strips punctuation.
>
> Use solr.StandardTokenizerFactory that strips punctuations.
>
> Or if you do not care about alphanumeric or numeric queries use
> solr.LowerCaseTokenizerFactory that uses LetterTokenizer.
>
> I think the right way to do this is using a
> > CharacterFilter
> > of some type, but I can't seem to find any examples of how
> > to set this
> > up in a schema.xml file.
>
> If you want to use solr.MappingCharFilterFactory you need to write all
> punctiation characters to a text file manually. e.g. "," => ""
>
>
>
>
RE: Stripping Punctuation in a fieldType
Posted by David Seltzer <ds...@TVEyes.com>.
I'm hesitant to change Tokenizers at the moment because what we have is
working so nicely - or so I thought.
What I'm looking for is case-insensitive search for words and numbers
without any of the stemming features turned on. The new requirement is
that we take punctuation out of the mix.
Right now when I search for "Obama" I'm not getting any hits on "Obama."
So I'm basically looking to strip punctuation. The consequence would be
that "nation's", "nations" and "nations," would all be represented the
same way.
Would the StandardTokenizerFactory accomplish this?
Does it have any language specific functionality?
Does it do anything with stemming?
Thanks for everyone's input!
-Dave
-----Original Message-----
From: Ahmet Arslan [mailto:iorixxx@yahoo.com]
Sent: Friday, January 15, 2010 12:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Stripping Punctuation in a fieldType
> I'm trying to find the best way to set up a fieldType that
> strips punctuation.
Use solr.StandardTokenizerFactory that strips punctuations.
Or if you do not care about alphanumeric or numeric queries use
solr.LowerCaseTokenizerFactory that uses LetterTokenizer.
I think the right way to do this is using a
> CharacterFilter
> of some type, but I can't seem to find any examples of how
> to set this
> up in a schema.xml file.
If you want to use solr.MappingCharFilterFactory you need to write all
punctiation characters to a text file manually. e.g. "," => ""
Re: Stripping Punctuation in a fieldType
Posted by Chris Hostetter <ho...@fucit.org>.
: Subject: Stripping Punctuation in a fieldType
: In-Reply-To: <27...@talk.nabble.com>
: References:
: <OF...@us.ibm.com>
: <27...@talk.nabble.com> <27...@talk.nabble.com>
http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which thread
you replied to and your question is "hidden" in that thread and gets less
attention. It makes following discussions in the mailing list archives
particularly difficult.
See Also: http://en.wikipedia.org/wiki/User:DonDiego/Thread_hijacking
-Hoss
Re: Stripping Punctuation in a fieldType
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi David,
(this is the David I know, right?)
The simplest may be to use one of the existing TokenFilters that removes punctuation. StandardFilterFactory should do it.
Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
----- Original Message ----
> From: David Seltzer <ds...@TVEyes.com>
> To: solr-user@lucene.apache.org
> Sent: Fri, January 15, 2010 12:31:41 PM
> Subject: Stripping Punctuation in a fieldType
>
> Hello All,
>
> I'm trying to find the best way to set up a fieldType that strips
> punctuation. I think the right way to do this is using a CharacterFilter
> of some type, but I can't seem to find any examples of how to set this
> up in a schema.xml file.
>
> Can anyone point me in the right direction?
>
> Right now my fieldType looks like this:
>
>
> positionIncrementGap="100">
>
>
>
>
>
>
> Thanks!
>
> -Dave Seltzer
Re: Stripping Punctuation in a fieldType
Posted by Ahmet Arslan <io...@yahoo.com>.
> I'm trying to find the best way to set up a fieldType that
> strips punctuation.
Use solr.StandardTokenizerFactory that strips punctuations.
Or if you do not care about alphanumeric or numeric queries use
solr.LowerCaseTokenizerFactory that uses LetterTokenizer.
I think the right way to do this is using a
> CharacterFilter
> of some type, but I can't seem to find any examples of how
> to set this
> up in a schema.xml file.
If you want to use solr.MappingCharFilterFactory you need to write all punctiation characters to a text file manually. e.g. "," => ""
Stripping Punctuation in a fieldType
Posted by David Seltzer <ds...@TVEyes.com>.
Hello All,
I'm trying to find the best way to set up a fieldType that strips
punctuation. I think the right way to do this is using a CharacterFilter
of some type, but I can't seem to find any examples of how to set this
up in a schema.xml file.
Can anyone point me in the right direction?
Right now my fieldType looks like this:
<fieldType name="text_ws_lc" class="solr.TextField"
positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
Thanks!
-Dave Seltzer
Re: [1.3] help with update timeout issue?
Posted by MitchK <mi...@web.de>.
The current topic "Need deployment strategy" may give you another answer
quite similar to mine one. It sounds much cleaner.
--
View this message in context: http://old.nabble.com/-1.3--help-with-update-timeout-issue--tp27171798p27179780.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: [1.3] help with update timeout issue?
Posted by MitchK <mi...@web.de>.
If, and only if you need to fix your problem as fast as you can, I would
think about virtualization.
You need to replicate your Solr and his index-files.
The idea is quiete easy: while one Solr-server does its optimization, the
other one is available for searching documents without any downtime. After
the first Solr-server has finished, add every document that has been added
to the second Solr's index and then optimize the second one.
Afterwards you can choose one of these servers as your default-server and
minimize system's ressources for the other one.
It's only an idea. I have never done something like that, but maybe it
helps.
For more information consider the wiki for "distributed search".
Kind regards
Mitch
--
View this message in context: http://old.nabble.com/-1.3--help-with-update-timeout-issue--tp27171798p27178423.html
Sent from the Solr - User mailing list archive at Nabble.com.