You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Parshant Kumar <pa...@indiamart.com.INVALID> on 2020/10/17 09:08:02 UTC
Index Replication Failure
Hi all,
We are having solr architecture as below.
*We are facing the frequent replication failure between master to repeater
server as well as between repeater to slave servers.*
On checking logs found every time one of the below exceptions occurred
whenever the replication have failed.
1)
[image: image.png]
2)
[image: image.png]
3)
[image: image.png]
The replication configuration of master,repeater,slave's is given below:
[image: image.png]
Commit Configuration master,repeater,slave's is given below :
[image: image.png]
Replication between master and repeater occurs every 10 mins.
Replication between repeater and slave servers occurs every 15 mins between
4-7 am and after that in every 3 hours.
Thanks,
Parshant Kumar
--
Re: Index Replication Failure
Posted by Parshant Kumar <pa...@indiamart.com.INVALID>.
Hi all, please check the details
On Sat, Oct 17, 2020 at 5:52 PM Parshant Kumar <pa...@indiamart.com>
wrote:
>
>
> *Architecture is master->repeater->slave servers in hierarchy.*
>
> *One of the Below exceptions are occuring whenever replication fails.*
>
> 1)WARN : Error in fetching file: _4rnu_t.liv (downloaded 0 of 11505507
> bytes)
> java.io.EOFException: Unexpected end of ZLIB input stream
> at
> java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
> at
> java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
> at
> org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:79)
> at
> org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:88)
> at
> org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:139)
> at
> org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:166)
> at
> org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:160)
> at
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetchPackets(IndexFetcher.java:1443)
> at
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1409)
>
> 2)
> WARN : Error getting file length for [segments_568]
> java.nio.file.NoSuchFileException:
> /data/solr/search/application/core-conf/im-search/data/index.20200711012319226/segments_568
> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
> at
> sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
> at
> sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
> at java.nio.file.Files.readAttributes(Files.java:1737)
> at java.nio.file.Files.size(Files.java:2332)
> at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
> at
> org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:615)
> at
> org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:588)
> at
> org.apache.solr.handler.admin.CoreAdminOperation.getCoreStatus(CoreAdminOperation.java:335)
>
>
> 3)
> WARN : Error in fetching file: _4nji.nvd (downloaded 507510784 of
> 555377795 bytes)
> org.apache.http.MalformedChunkCodingException: CRLF expected at end of
> chunk
> at
> org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:255)
> at
> org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:227)
> at
> org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:186)
> at
> org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:137)
> at
> java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238)
> at
> java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
> at
> org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:79)
> at
> org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:128)
> at
> org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:166)
> at
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetchPackets(IndexFetcher.java:1458)
> at
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1409)
> at
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1390)
> at
> org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:872)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:438)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254)
>
>
> *Replication configuration of master,repeater,slave's is given below:*
>
> <requestHandler name="/replication" class="solr.ReplicationHandler">
> <lst name="master">
> <str name="enable">${enable.master:false}</str>
> <str name="replicateAfter">commit</str>
> <str name="replicateAfter">startup</str>
> <str name="commitReserveDuration">00:00:10</str>
> </lst>
>
>
> *Commit Configuration master,repeater,slave's is given below :*
>
> <autoCommit><maxDocs>100000</maxDocs><openSearcher>false</openSearcher></autoCommit>
>
>
>
>
>
>
> On Sat, Oct 17, 2020 at 5:12 PM Erick Erickson <er...@gmail.com>
> wrote:
>
>> None of your images made it through the mail server. You’ll
>> have to put them somewhere and provide a link.
>>
>> > On Oct 17, 2020, at 5:17 AM, Parshant Kumar <
>> parshant.kumar@indiamart.com.INVALID> wrote:
>> >
>> > Architecture image: If not visible in previous mail
>> >
>> >
>> >
>> >
>> > On Sat, Oct 17, 2020 at 2:38 PM Parshant Kumar <
>> parshant.kumar@indiamart.com> wrote:
>> > Hi all,
>> >
>> > We are having solr architecture as below.
>> >
>> >
>> >
>> > We are facing the frequent replication failure between master to
>> repeater server as well as between repeater to slave servers.
>> > On checking logs found every time one of the below exceptions occurred
>> whenever the replication have failed.
>> >
>> > 1)
>> >
>> > 2)
>> >
>> >
>> > 3)
>> >
>> >
>> > The replication configuration of master,repeater,slave's is given below:
>> >
>> >
>> >
>> > Commit Configuration master,repeater,slave's is given below :
>> >
>> >
>> >
>> > Replication between master and repeater occurs every 10 mins.
>> > Replication between repeater and slave servers occurs every 15 mins
>> between 4-7 am and after that in every 3 hours.
>> >
>> > Thanks,
>> > Parshant Kumar
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
--
Re: Index Replication Failure
Posted by Parshant Kumar <pa...@indiamart.com.INVALID>.
*Architecture is master->repeater->slave servers in hierarchy.*
*One of the Below exceptions are occuring whenever replication fails.*
1)WARN : Error in fetching file: _4rnu_t.liv (downloaded 0 of 11505507
bytes)
java.io.EOFException: Unexpected end of ZLIB input stream
at
java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
at
java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at
org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:79)
at
org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:88)
at
org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:139)
at
org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:166)
at
org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:160)
at
org.apache.solr.handler.IndexFetcher$FileFetcher.fetchPackets(IndexFetcher.java:1443)
at
org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1409)
2)
WARN : Error getting file length for [segments_568]
java.nio.file.NoSuchFileException:
/data/solr/search/application/core-conf/im-search/data/index.20200711012319226/segments_568
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at
sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at
sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at
sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.Files.size(Files.java:2332)
at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
at
org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:615)
at
org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:588)
at
org.apache.solr.handler.admin.CoreAdminOperation.getCoreStatus(CoreAdminOperation.java:335)
3)
WARN : Error in fetching file: _4nji.nvd (downloaded 507510784 of 555377795
bytes)
org.apache.http.MalformedChunkCodingException: CRLF expected at end of chunk
at
org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:255)
at
org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:227)
at
org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:186)
at
org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:137)
at
java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238)
at
java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at
org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:79)
at
org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:128)
at
org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:166)
at
org.apache.solr.handler.IndexFetcher$FileFetcher.fetchPackets(IndexFetcher.java:1458)
at
org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1409)
at
org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1390)
at
org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:872)
at
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:438)
at
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254)
*Replication configuration of master,repeater,slave's is given below:*
<requestHandler name="/replication" class="solr.ReplicationHandler">
<lst name="master">
<str name="enable">${enable.master:false}</str>
<str name="replicateAfter">commit</str>
<str name="replicateAfter">startup</str>
<str name="commitReserveDuration">00:00:10</str>
</lst>
*Commit Configuration master,repeater,slave's is given below :*
<autoCommit><maxDocs>100000</maxDocs><openSearcher>false</openSearcher></autoCommit>
On Sat, Oct 17, 2020 at 5:12 PM Erick Erickson <er...@gmail.com>
wrote:
> None of your images made it through the mail server. You’ll
> have to put them somewhere and provide a link.
>
> > On Oct 17, 2020, at 5:17 AM, Parshant Kumar <
> parshant.kumar@indiamart.com.INVALID> wrote:
> >
> > Architecture image: If not visible in previous mail
> >
> >
> >
> >
> > On Sat, Oct 17, 2020 at 2:38 PM Parshant Kumar <
> parshant.kumar@indiamart.com> wrote:
> > Hi all,
> >
> > We are having solr architecture as below.
> >
> >
> >
> > We are facing the frequent replication failure between master to
> repeater server as well as between repeater to slave servers.
> > On checking logs found every time one of the below exceptions occurred
> whenever the replication have failed.
> >
> > 1)
> >
> > 2)
> >
> >
> > 3)
> >
> >
> > The replication configuration of master,repeater,slave's is given below:
> >
> >
> >
> > Commit Configuration master,repeater,slave's is given below :
> >
> >
> >
> > Replication between master and repeater occurs every 10 mins.
> > Replication between repeater and slave servers occurs every 15 mins
> between 4-7 am and after that in every 3 hours.
> >
> > Thanks,
> > Parshant Kumar
> >
> >
> >
> >
> >
> >
> >
>
>
--
Re: Index Replication Failure
Posted by Erick Erickson <er...@gmail.com>.
None of your images made it through the mail server. You’ll
have to put them somewhere and provide a link.
> On Oct 17, 2020, at 5:17 AM, Parshant Kumar <pa...@indiamart.com.INVALID> wrote:
>
> Architecture image: If not visible in previous mail
>
>
>
>
> On Sat, Oct 17, 2020 at 2:38 PM Parshant Kumar <pa...@indiamart.com> wrote:
> Hi all,
>
> We are having solr architecture as below.
>
>
>
> We are facing the frequent replication failure between master to repeater server as well as between repeater to slave servers.
> On checking logs found every time one of the below exceptions occurred whenever the replication have failed.
>
> 1)
>
> 2)
>
>
> 3)
>
>
> The replication configuration of master,repeater,slave's is given below:
>
>
>
> Commit Configuration master,repeater,slave's is given below :
>
>
>
> Replication between master and repeater occurs every 10 mins.
> Replication between repeater and slave servers occurs every 15 mins between 4-7 am and after that in every 3 hours.
>
> Thanks,
> Parshant Kumar
>
>
>
>
>
>
>
Re: Index Replication Failure
Posted by Parshant Kumar <pa...@indiamart.com.INVALID>.
Architecture image: If not visible in previous mail
[image: image.png]
On Sat, Oct 17, 2020 at 2:38 PM Parshant Kumar <pa...@indiamart.com>
wrote:
> Hi all,
>
> We are having solr architecture as below.
>
>
>
> *We are facing the frequent replication failure between master to repeater
> server as well as between repeater to slave servers.*
> On checking logs found every time one of the below exceptions occurred
> whenever the replication have failed.
>
> 1)
> [image: image.png]
> 2)
> [image: image.png]
>
> 3)
> [image: image.png]
>
> The replication configuration of master,repeater,slave's is given below:
>
> [image: image.png]
>
> Commit Configuration master,repeater,slave's is given below :
>
> [image: image.png]
>
> Replication between master and repeater occurs every 10 mins.
> Replication between repeater and slave servers occurs every 15 mins
> between 4-7 am and after that in every 3 hours.
>
> Thanks,
> Parshant Kumar
>
>
>
>
>
>
--