Index corruption with replication

[Santosh Sidnal <si...@gmail.com>]

Hi all,

I am facing issues of index corruption at regular intervals of the time on live server where i pull index data from one master server.

Can anyone please give us some ppinters why we are facing issue on regular interval of time? 
I am aware of how can we correct corrupted index but i am looking some pointers how can i stop or reduce this occurrence. 

Thanks in advance.


Sent from my iPhone

Re: Index corruption with replication

[santosh sidnal <si...@gmail.com>]

Hi Erik/David,

Schema is same on both live and stage servers. We are using the same schema
files on the stage and live files.


   - Schema files are included in replication but these are not being
   changed whenever we are observing schema corruption issue.
   - My guess is that because of replication the core is getting corrupted.
   - SOLR version used is 4.7.0
   -



Exception which I see in log is

org.apache.solr.common.SolrException log
org.apache.lucene.index.CorruptIndexException: Corrupted: docID=8195,
docBase=7, chunkDocs=249, numDocs=10596
(resource=MMapIndexInput(path="/app/IBM/WebSphere/CommerceServer70/instances/RBUATLV/search/solr/home/MC_10001/fr_FR/CatalogEntry/data/index/_5a.fdt"))
at
org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:236)
at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:276)
at
org.apache.lucene.index.BaseCompositeReader.document(BaseCompositeReader.java:110)
at org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:661)
at
org.apache.solr.util.SolrPluginUtils.optimizePreFetchDocs(SolrPluginUtils.java:213)
at
org.apache.solr.handler.component.QueryComponent.doPrefetch(QueryComponent.java:568)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:475)
at

On 15 March 2017 at 22:42, Erick Erickson <er...@gmail.com> wrote:

> You can specify your replication to include config files, but if the
> schema has changed you'll have to restart your Solr afterwards.
>
> How is it corrupt? what is the symptom? Any error messages in the solr
> log on the slave? What version of Solr? Details matter.
>
> Best,
> Erick
>
> On Wed, Mar 15, 2017 at 9:12 AM, David Hastings
> <ha...@gmail.com> wrote:
> > are you certain the schema is the same on both master and slave?  I find
> > that the schema file doesnt always go with the replication and if a field
> > is different on the slave it will cause problems
> >
> > On Wed, Mar 15, 2017 at 12:08 PM, Santosh Sidnal <
> sidnal.santosh@gmail.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> I am facing issues of index corruption at regular intervals of the time
> on
> >> live server where i pull index data from one master server.
> >>
> >> Can anyone please give us some ppinters why we are facing issue on
> regular
> >> interval of time?
> >> I am aware of how can we correct corrupted index but i am looking some
> >> pointers how can i stop or reduce this occurrence.
> >>
> >> Thanks in advance.
> >>
> >>
> >> Sent from my iPhone
>



-- 
Regards,
Santosh Sidnal

Re: Index corruption with replication

[Erick Erickson <er...@gmail.com>]

You can specify your replication to include config files, but if the
schema has changed you'll have to restart your Solr afterwards.

How is it corrupt? what is the symptom? Any error messages in the solr
log on the slave? What version of Solr? Details matter.

Best,
Erick

On Wed, Mar 15, 2017 at 9:12 AM, David Hastings
<ha...@gmail.com> wrote:
> are you certain the schema is the same on both master and slave?  I find
> that the schema file doesnt always go with the replication and if a field
> is different on the slave it will cause problems
>
> On Wed, Mar 15, 2017 at 12:08 PM, Santosh Sidnal <si...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> I am facing issues of index corruption at regular intervals of the time on
>> live server where i pull index data from one master server.
>>
>> Can anyone please give us some ppinters why we are facing issue on regular
>> interval of time?
>> I am aware of how can we correct corrupted index but i am looking some
>> pointers how can i stop or reduce this occurrence.
>>
>> Thanks in advance.
>>
>>
>> Sent from my iPhone

Re: Index corruption with replication

[David Hastings <ha...@gmail.com>]

are you certain the schema is the same on both master and slave?  I find
that the schema file doesnt always go with the replication and if a field
is different on the slave it will cause problems

On Wed, Mar 15, 2017 at 12:08 PM, Santosh Sidnal <si...@gmail.com>
wrote:

> Hi all,
>
> I am facing issues of index corruption at regular intervals of the time on
> live server where i pull index data from one master server.
>
> Can anyone please give us some ppinters why we are facing issue on regular
> interval of time?
> I am aware of how can we correct corrupted index but i am looking some
> pointers how can i stop or reduce this occurrence.
>
> Thanks in advance.
>
>
> Sent from my iPhone