You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Akreeti Agarwal <Ak...@hcl.com> on 2019/08/28 04:57:35 UTC
Index fetch failed
Hello Everyone,
I am getting this error continuously on Solr slave, can anyone tell me the solution for this:
642141666 ERROR (indexFetcher-72-thread-1) [ x:sitecore_web_index] o.a.s.h.ReplicationHandler Index fetch failed :org.apache.solr.common.SolrException: Unable to download _12i7v_f.liv completely. Downloaded 0!=123
at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1434)
at org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1314)
at org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:812)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:427)
Thanks & Regards,
Akreeti Agarwal
(M) +91-8318686601
::DISCLAIMER::
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Re: Index fetch failed
Posted by Erick Erickson <er...@gmail.com>.
Shankar:
Two things:
1> please do not hijack threads
2> Follow the instructions here: http://lucene.apache.org/solr/community.html#mailing-lists-irc. You must use the _exact_ same e-mail as you used to subscribe.
If the initial try doesn't work and following the suggestions at the "problems" link doesn't work for you, let us know. But note you need to show us the _entire_ return header to allow anyone to diagnose the problem.
Best,
Erick
> On Sep 3, 2019, at 5:55 AM, Shankar Ramalingam <sh...@gmail.com> wrote:
>
> Please remove my email id from this list.
>
> On Tue, 3 Sep, 2019, 11:06 AM Akreeti Agarwal, <Ak...@hcl.com> wrote:
>
>> Hello,
>>
>> Please help me with the solution for below error.
>>
>> Memory details of slave server:
>> total used free shared buffers cached
>> Mem: 15947 15460 487 62 144 6007
>> -/+ buffers/cache: 9308 6639
>> Swap: 0 0 0
>>
>>
>> Thanks & Regards,
>> Akreeti Agarwal
>>
>> -----Original Message-----
>> From: Akreeti Agarwal <Ak...@hcl.com>
>> Sent: Wednesday, August 28, 2019 2:45 PM
>> To: solr-user@lucene.apache.org
>> Subject: RE: Index fetch failed
>>
>> Yes I am using solr-5.5.5.
>> This error is intermittent. I don't think there must be any issue with
>> master connection limits. This error is accompanied by this on master side:
>>
>> ERROR (qtp1450821318-60072) [ x:sitecore_web_index]
>> o.a.s.h.ReplicationHandler Unable to get file names for indexCommit
>> generation: 1558637
>> java.nio.file.NoSuchFileException:
>> /solrm-efs/solr-m/server/solr/sitecore_web_index/data/index/_12i9p_1.liv
>> at
>> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>> at
>> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>> at
>> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>> at
>> sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
>> at
>> sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
>> at
>> sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
>> at java.nio.file.Files.readAttributes(Files.java:1737)
>> at java.nio.file.Files.size(Files.java:2332)
>> at
>> org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:210)
>> at
>> org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:124)
>> at
>> org.apache.solr.handler.ReplicationHandler.getFileList(ReplicationHandler.java:563)
>> at
>> org.apache.solr.handler.ReplicationHandler.handleRequestBody(ReplicationHandler.java:253)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2102)
>> at
>> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
>> at
>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
>> at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>> at
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>> at
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>> at
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>> at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>> at org.eclipse.jetty.server.Server.handle(Server.java:499)
>> at
>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>> at
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>> at org.eclipse.jetty.io
>> .AbstractConnection$2.run(AbstractConnection.java:540)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>> at java.lang.Thread.run(Thread.java:748)
>>
>> Thanks & Regards,
>> Akreeti Agarwal
>>
>> -----Original Message-----
>> From: Atita Arora <at...@gmail.com>
>> Sent: Wednesday, August 28, 2019 2:23 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Index fetch failed
>>
>> This looks like ample memory to get the index chunk.
>> Also, I looked at the IndexFetcher code, I remember you were using Solr
>> 5.5.5 and the only reason in my view, this would happen is when the index
>> chunk is not downloaded as can also be seen in the error (Downloaded
>> 0!=123) which clearly states that the index generations are not in sync
>> and this is not user aborted action too.
>>
>> Is this error intermittent? could there be a possibility that your master
>> has connection limits? or maybe some network hiccup?
>>
>>
>>
>> On Wed, Aug 28, 2019 at 10:40 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
>>
>>> Hi,
>>>
>>> Memory details for slave1:
>>>
>>> Filesystem Size Used Avail Use% Mounted on
>>> /dev/xvda1 99G 40G 55G 43% /
>>> tmpfs 7.8G 0 7.8G 0% /dev/shm
>>>
>>> Memory details for slave2:
>>>
>>> Filesystem Size Used Avail Use% Mounted on
>>> /dev/xvda1 99G 45G 49G 48% /
>>> tmpfs 7.8G 0 7.8G 0% /dev/shm
>>>
>>> Thanks & Regards,
>>> Akreeti Agarwal
>>>
>>> -----Original Message-----
>>> From: Atita Arora <at...@gmail.com>
>>> Sent: Wednesday, August 28, 2019 11:15 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Index fetch failed
>>>
>>> Hii,
>>>
>>> Do you have enough memory free for the index chunk to be
>>> fetched/Downloaded on the slave node?
>>>
>>>
>>> On Wed, Aug 28, 2019 at 6:57 AM Akreeti Agarwal <Ak...@hcl.com>
>> wrote:
>>>
>>>> Hello Everyone,
>>>>
>>>> I am getting this error continuously on Solr slave, can anyone tell
>>>> me the solution for this:
>>>>
>>>> 642141666 ERROR (indexFetcher-72-thread-1) [ x:sitecore_web_index]
>>>> o.a.s.h.ReplicationHandler Index fetch failed
>>>> :org.apache.solr.common.SolrException: Unable to download
>>>> _12i7v_f.liv completely. Downloaded 0!=123
>>>> at
>>>>
>>> org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.
>>> java:1434)
>>>> at
>>>>
>>> org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetche
>>> r.java:1314)
>>>> at
>>>>
>>> org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.j
>>> ava:812)
>>>> at
>>>> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.j
>>>> av
>>>> a:427)
>>>>
>>>>
>>>> Thanks & Regards,
>>>> Akreeti Agarwal
>>>> (M) +91-8318686601
>>>>
>>>> ::DISCLAIMER::
>>>>
>>>> --------------------------------------------------------------------
>>>> --
>>>> --------------------------------------------------------------------
>>>> --
>>>> --------------------------------------------------------------------
>>>> --
>>>> --------------------------------------------------------------------
>>>> The contents of this e-mail and any attachment(s) are confidential
>>>> and intended for the named recipient(s) only. E-mail transmission is
>>>> not guaranteed to be secure or error-free as information could be
>>>> intercepted, corrupted, lost, destroyed, arrive late or incomplete,
>>>> or may contain viruses in transmission. The e mail and its contents
>>>> (with or without referred errors) shall therefore not attach any
>>>> liability on the originator or HCL or its affiliates. Views or
>>>> opinions, if any, presented in this email are solely those of the
>>>> author and may not necessarily reflect the views or opinions of HCL or
>> its affiliates.
>>>> Any form of reproduction, dissemination, copying, disclosure,
>>>> modification, distribution and / or publication of this message
>>>> without the prior written consent of authorized representative of
>>>> HCL is strictly prohibited. If you have received this email in error
>>>> please delete it and notify the sender immediately. Before opening
>>>> any email and/or attachments, please check them for viruses and
>>>> other
>>> defects.
>>>>
>>>> --------------------------------------------------------------------
>>>> --
>>>>
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> --------------------------------------------------------------------
>>>>
>>>
>>
Re: Index fetch failed
Posted by Shankar Ramalingam <sh...@gmail.com>.
Please remove my email id from this list.
On Tue, 3 Sep, 2019, 11:06 AM Akreeti Agarwal, <Ak...@hcl.com> wrote:
> Hello,
>
> Please help me with the solution for below error.
>
> Memory details of slave server:
> total used free shared buffers cached
> Mem: 15947 15460 487 62 144 6007
> -/+ buffers/cache: 9308 6639
> Swap: 0 0 0
>
>
> Thanks & Regards,
> Akreeti Agarwal
>
> -----Original Message-----
> From: Akreeti Agarwal <Ak...@hcl.com>
> Sent: Wednesday, August 28, 2019 2:45 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Index fetch failed
>
> Yes I am using solr-5.5.5.
> This error is intermittent. I don't think there must be any issue with
> master connection limits. This error is accompanied by this on master side:
>
> ERROR (qtp1450821318-60072) [ x:sitecore_web_index]
> o.a.s.h.ReplicationHandler Unable to get file names for indexCommit
> generation: 1558637
> java.nio.file.NoSuchFileException:
> /solrm-efs/solr-m/server/solr/sitecore_web_index/data/index/_12i9p_1.liv
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
> at
> sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
> at
> sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
> at java.nio.file.Files.readAttributes(Files.java:1737)
> at java.nio.file.Files.size(Files.java:2332)
> at
> org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:210)
> at
> org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:124)
> at
> org.apache.solr.handler.ReplicationHandler.getFileList(ReplicationHandler.java:563)
> at
> org.apache.solr.handler.ReplicationHandler.handleRequestBody(ReplicationHandler.java:253)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2102)
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
> at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at org.eclipse.jetty.io
> .AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:748)
>
> Thanks & Regards,
> Akreeti Agarwal
>
> -----Original Message-----
> From: Atita Arora <at...@gmail.com>
> Sent: Wednesday, August 28, 2019 2:23 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Index fetch failed
>
> This looks like ample memory to get the index chunk.
> Also, I looked at the IndexFetcher code, I remember you were using Solr
> 5.5.5 and the only reason in my view, this would happen is when the index
> chunk is not downloaded as can also be seen in the error (Downloaded
> 0!=123) which clearly states that the index generations are not in sync
> and this is not user aborted action too.
>
> Is this error intermittent? could there be a possibility that your master
> has connection limits? or maybe some network hiccup?
>
>
>
> On Wed, Aug 28, 2019 at 10:40 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
>
> > Hi,
> >
> > Memory details for slave1:
> >
> > Filesystem Size Used Avail Use% Mounted on
> > /dev/xvda1 99G 40G 55G 43% /
> > tmpfs 7.8G 0 7.8G 0% /dev/shm
> >
> > Memory details for slave2:
> >
> > Filesystem Size Used Avail Use% Mounted on
> > /dev/xvda1 99G 45G 49G 48% /
> > tmpfs 7.8G 0 7.8G 0% /dev/shm
> >
> > Thanks & Regards,
> > Akreeti Agarwal
> >
> > -----Original Message-----
> > From: Atita Arora <at...@gmail.com>
> > Sent: Wednesday, August 28, 2019 11:15 AM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Index fetch failed
> >
> > Hii,
> >
> > Do you have enough memory free for the index chunk to be
> > fetched/Downloaded on the slave node?
> >
> >
> > On Wed, Aug 28, 2019 at 6:57 AM Akreeti Agarwal <Ak...@hcl.com>
> wrote:
> >
> > > Hello Everyone,
> > >
> > > I am getting this error continuously on Solr slave, can anyone tell
> > > me the solution for this:
> > >
> > > 642141666 ERROR (indexFetcher-72-thread-1) [ x:sitecore_web_index]
> > > o.a.s.h.ReplicationHandler Index fetch failed
> > > :org.apache.solr.common.SolrException: Unable to download
> > > _12i7v_f.liv completely. Downloaded 0!=123
> > > at
> > >
> > org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.
> > java:1434)
> > > at
> > >
> > org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetche
> > r.java:1314)
> > > at
> > >
> > org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.j
> > ava:812)
> > > at
> > > org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.j
> > > av
> > > a:427)
> > >
> > >
> > > Thanks & Regards,
> > > Akreeti Agarwal
> > > (M) +91-8318686601
> > >
> > > ::DISCLAIMER::
> > >
> > > --------------------------------------------------------------------
> > > --
> > > --------------------------------------------------------------------
> > > --
> > > --------------------------------------------------------------------
> > > --
> > > --------------------------------------------------------------------
> > > The contents of this e-mail and any attachment(s) are confidential
> > > and intended for the named recipient(s) only. E-mail transmission is
> > > not guaranteed to be secure or error-free as information could be
> > > intercepted, corrupted, lost, destroyed, arrive late or incomplete,
> > > or may contain viruses in transmission. The e mail and its contents
> > > (with or without referred errors) shall therefore not attach any
> > > liability on the originator or HCL or its affiliates. Views or
> > > opinions, if any, presented in this email are solely those of the
> > > author and may not necessarily reflect the views or opinions of HCL or
> its affiliates.
> > > Any form of reproduction, dissemination, copying, disclosure,
> > > modification, distribution and / or publication of this message
> > > without the prior written consent of authorized representative of
> > > HCL is strictly prohibited. If you have received this email in error
> > > please delete it and notify the sender immediately. Before opening
> > > any email and/or attachments, please check them for viruses and
> > > other
> > defects.
> > >
> > > --------------------------------------------------------------------
> > > --
> > >
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > --------------------------------------------------------------------
> > >
> >
>
RE: Index fetch failed
Posted by Akreeti Agarwal <Ak...@hcl.com>.
Hello,
Please help me with the solution for below error.
Memory details of slave server:
total used free shared buffers cached
Mem: 15947 15460 487 62 144 6007
-/+ buffers/cache: 9308 6639
Swap: 0 0 0
Thanks & Regards,
Akreeti Agarwal
-----Original Message-----
From: Akreeti Agarwal <Ak...@hcl.com>
Sent: Wednesday, August 28, 2019 2:45 PM
To: solr-user@lucene.apache.org
Subject: RE: Index fetch failed
Yes I am using solr-5.5.5.
This error is intermittent. I don't think there must be any issue with master connection limits. This error is accompanied by this on master side:
ERROR (qtp1450821318-60072) [ x:sitecore_web_index] o.a.s.h.ReplicationHandler Unable to get file names for indexCommit generation: 1558637
java.nio.file.NoSuchFileException: /solrm-efs/solr-m/server/solr/sitecore_web_index/data/index/_12i9p_1.liv
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.Files.size(Files.java:2332)
at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:210)
at org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:124)
at org.apache.solr.handler.ReplicationHandler.getFileList(ReplicationHandler.java:563)
at org.apache.solr.handler.ReplicationHandler.handleRequestBody(ReplicationHandler.java:253)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2102)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:748)
Thanks & Regards,
Akreeti Agarwal
-----Original Message-----
From: Atita Arora <at...@gmail.com>
Sent: Wednesday, August 28, 2019 2:23 PM
To: solr-user@lucene.apache.org
Subject: Re: Index fetch failed
This looks like ample memory to get the index chunk.
Also, I looked at the IndexFetcher code, I remember you were using Solr
5.5.5 and the only reason in my view, this would happen is when the index chunk is not downloaded as can also be seen in the error (Downloaded
0!=123) which clearly states that the index generations are not in sync and this is not user aborted action too.
Is this error intermittent? could there be a possibility that your master has connection limits? or maybe some network hiccup?
On Wed, Aug 28, 2019 at 10:40 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
> Hi,
>
> Memory details for slave1:
>
> Filesystem Size Used Avail Use% Mounted on
> /dev/xvda1 99G 40G 55G 43% /
> tmpfs 7.8G 0 7.8G 0% /dev/shm
>
> Memory details for slave2:
>
> Filesystem Size Used Avail Use% Mounted on
> /dev/xvda1 99G 45G 49G 48% /
> tmpfs 7.8G 0 7.8G 0% /dev/shm
>
> Thanks & Regards,
> Akreeti Agarwal
>
> -----Original Message-----
> From: Atita Arora <at...@gmail.com>
> Sent: Wednesday, August 28, 2019 11:15 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Index fetch failed
>
> Hii,
>
> Do you have enough memory free for the index chunk to be
> fetched/Downloaded on the slave node?
>
>
> On Wed, Aug 28, 2019 at 6:57 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
>
> > Hello Everyone,
> >
> > I am getting this error continuously on Solr slave, can anyone tell
> > me the solution for this:
> >
> > 642141666 ERROR (indexFetcher-72-thread-1) [ x:sitecore_web_index]
> > o.a.s.h.ReplicationHandler Index fetch failed
> > :org.apache.solr.common.SolrException: Unable to download
> > _12i7v_f.liv completely. Downloaded 0!=123
> > at
> >
> org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.
> java:1434)
> > at
> >
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetche
> r.java:1314)
> > at
> >
> org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.j
> ava:812)
> > at
> > org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.j
> > av
> > a:427)
> >
> >
> > Thanks & Regards,
> > Akreeti Agarwal
> > (M) +91-8318686601
> >
> > ::DISCLAIMER::
> >
> > --------------------------------------------------------------------
> > --
> > --------------------------------------------------------------------
> > --
> > --------------------------------------------------------------------
> > --
> > --------------------------------------------------------------------
> > The contents of this e-mail and any attachment(s) are confidential
> > and intended for the named recipient(s) only. E-mail transmission is
> > not guaranteed to be secure or error-free as information could be
> > intercepted, corrupted, lost, destroyed, arrive late or incomplete,
> > or may contain viruses in transmission. The e mail and its contents
> > (with or without referred errors) shall therefore not attach any
> > liability on the originator or HCL or its affiliates. Views or
> > opinions, if any, presented in this email are solely those of the
> > author and may not necessarily reflect the views or opinions of HCL or its affiliates.
> > Any form of reproduction, dissemination, copying, disclosure,
> > modification, distribution and / or publication of this message
> > without the prior written consent of authorized representative of
> > HCL is strictly prohibited. If you have received this email in error
> > please delete it and notify the sender immediately. Before opening
> > any email and/or attachments, please check them for viruses and
> > other
> defects.
> >
> > --------------------------------------------------------------------
> > --
> >
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> --------------------------------------------------------------------
> >
>
RE: Index fetch failed
Posted by Akreeti Agarwal <Ak...@hcl.com>.
Yes I am using solr-5.5.5.
This error is intermittent. I don't think there must be any issue with master connection limits. This error is accompanied by this on master side:
ERROR (qtp1450821318-60072) [ x:sitecore_web_index] o.a.s.h.ReplicationHandler Unable to get file names for indexCommit generation: 1558637
java.nio.file.NoSuchFileException: /solrm-efs/solr-m/server/solr/sitecore_web_index/data/index/_12i9p_1.liv
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.Files.size(Files.java:2332)
at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:210)
at org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:124)
at org.apache.solr.handler.ReplicationHandler.getFileList(ReplicationHandler.java:563)
at org.apache.solr.handler.ReplicationHandler.handleRequestBody(ReplicationHandler.java:253)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2102)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:748)
Thanks & Regards,
Akreeti Agarwal
-----Original Message-----
From: Atita Arora <at...@gmail.com>
Sent: Wednesday, August 28, 2019 2:23 PM
To: solr-user@lucene.apache.org
Subject: Re: Index fetch failed
This looks like ample memory to get the index chunk.
Also, I looked at the IndexFetcher code, I remember you were using Solr
5.5.5 and the only reason in my view, this would happen is when the index chunk is not downloaded as can also be seen in the error (Downloaded
0!=123) which clearly states that the index generations are not in sync and this is not user aborted action too.
Is this error intermittent? could there be a possibility that your master has connection limits? or maybe some network hiccup?
On Wed, Aug 28, 2019 at 10:40 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
> Hi,
>
> Memory details for slave1:
>
> Filesystem Size Used Avail Use% Mounted on
> /dev/xvda1 99G 40G 55G 43% /
> tmpfs 7.8G 0 7.8G 0% /dev/shm
>
> Memory details for slave2:
>
> Filesystem Size Used Avail Use% Mounted on
> /dev/xvda1 99G 45G 49G 48% /
> tmpfs 7.8G 0 7.8G 0% /dev/shm
>
> Thanks & Regards,
> Akreeti Agarwal
>
> -----Original Message-----
> From: Atita Arora <at...@gmail.com>
> Sent: Wednesday, August 28, 2019 11:15 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Index fetch failed
>
> Hii,
>
> Do you have enough memory free for the index chunk to be
> fetched/Downloaded on the slave node?
>
>
> On Wed, Aug 28, 2019 at 6:57 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
>
> > Hello Everyone,
> >
> > I am getting this error continuously on Solr slave, can anyone tell
> > me the solution for this:
> >
> > 642141666 ERROR (indexFetcher-72-thread-1) [ x:sitecore_web_index]
> > o.a.s.h.ReplicationHandler Index fetch failed
> > :org.apache.solr.common.SolrException: Unable to download
> > _12i7v_f.liv completely. Downloaded 0!=123
> > at
> >
> org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.
> java:1434)
> > at
> >
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetche
> r.java:1314)
> > at
> >
> org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.j
> ava:812)
> > at
> > org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.j
> > av
> > a:427)
> >
> >
> > Thanks & Regards,
> > Akreeti Agarwal
> > (M) +91-8318686601
> >
> > ::DISCLAIMER::
> >
> > --------------------------------------------------------------------
> > --
> > --------------------------------------------------------------------
> > --
> > --------------------------------------------------------------------
> > --
> > --------------------------------------------------------------------
> > The contents of this e-mail and any attachment(s) are confidential
> > and intended for the named recipient(s) only. E-mail transmission is
> > not guaranteed to be secure or error-free as information could be
> > intercepted, corrupted, lost, destroyed, arrive late or incomplete,
> > or may contain viruses in transmission. The e mail and its contents
> > (with or without referred errors) shall therefore not attach any
> > liability on the originator or HCL or its affiliates. Views or
> > opinions, if any, presented in this email are solely those of the
> > author and may not necessarily reflect the views or opinions of HCL or its affiliates.
> > Any form of reproduction, dissemination, copying, disclosure,
> > modification, distribution and / or publication of this message
> > without the prior written consent of authorized representative of
> > HCL is strictly prohibited. If you have received this email in error
> > please delete it and notify the sender immediately. Before opening
> > any email and/or attachments, please check them for viruses and
> > other
> defects.
> >
> > --------------------------------------------------------------------
> > --
> >
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> --------------------------------------------------------------------
> >
>
Re: Index fetch failed
Posted by Atita Arora <at...@gmail.com>.
This looks like ample memory to get the index chunk.
Also, I looked at the IndexFetcher code, I remember you were using Solr
5.5.5 and the only reason in my view, this would happen is when the index
chunk is not downloaded as can also be seen in the error (Downloaded
0!=123) which clearly states that the index generations are not in sync and
this is not user aborted action too.
Is this error intermittent? could there be a possibility that your master
has connection limits? or maybe some network hiccup?
On Wed, Aug 28, 2019 at 10:40 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
> Hi,
>
> Memory details for slave1:
>
> Filesystem Size Used Avail Use% Mounted on
> /dev/xvda1 99G 40G 55G 43% /
> tmpfs 7.8G 0 7.8G 0% /dev/shm
>
> Memory details for slave2:
>
> Filesystem Size Used Avail Use% Mounted on
> /dev/xvda1 99G 45G 49G 48% /
> tmpfs 7.8G 0 7.8G 0% /dev/shm
>
> Thanks & Regards,
> Akreeti Agarwal
>
> -----Original Message-----
> From: Atita Arora <at...@gmail.com>
> Sent: Wednesday, August 28, 2019 11:15 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Index fetch failed
>
> Hii,
>
> Do you have enough memory free for the index chunk to be
> fetched/Downloaded on the slave node?
>
>
> On Wed, Aug 28, 2019 at 6:57 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
>
> > Hello Everyone,
> >
> > I am getting this error continuously on Solr slave, can anyone tell me
> > the solution for this:
> >
> > 642141666 ERROR (indexFetcher-72-thread-1) [ x:sitecore_web_index]
> > o.a.s.h.ReplicationHandler Index fetch failed
> > :org.apache.solr.common.SolrException: Unable to download _12i7v_f.liv
> > completely. Downloaded 0!=123
> > at
> >
> org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1434)
> > at
> >
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1314)
> > at
> >
> org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:812)
> > at
> > org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.jav
> > a:427)
> >
> >
> > Thanks & Regards,
> > Akreeti Agarwal
> > (M) +91-8318686601
> >
> > ::DISCLAIMER::
> >
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > --------------------------------------------------------------------
> > The contents of this e-mail and any attachment(s) are confidential and
> > intended for the named recipient(s) only. E-mail transmission is not
> > guaranteed to be secure or error-free as information could be
> > intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
> > may contain viruses in transmission. The e mail and its contents (with
> > or without referred errors) shall therefore not attach any liability
> > on the originator or HCL or its affiliates. Views or opinions, if any,
> > presented in this email are solely those of the author and may not
> > necessarily reflect the views or opinions of HCL or its affiliates.
> > Any form of reproduction, dissemination, copying, disclosure,
> > modification, distribution and / or publication of this message
> > without the prior written consent of authorized representative of HCL
> > is strictly prohibited. If you have received this email in error
> > please delete it and notify the sender immediately. Before opening any
> > email and/or attachments, please check them for viruses and other
> defects.
> >
> > ----------------------------------------------------------------------
> >
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
>
RE: Index fetch failed
Posted by Akreeti Agarwal <Ak...@hcl.com>.
Hi,
Memory details for slave1:
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 99G 40G 55G 43% /
tmpfs 7.8G 0 7.8G 0% /dev/shm
Memory details for slave2:
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 99G 45G 49G 48% /
tmpfs 7.8G 0 7.8G 0% /dev/shm
Thanks & Regards,
Akreeti Agarwal
-----Original Message-----
From: Atita Arora <at...@gmail.com>
Sent: Wednesday, August 28, 2019 11:15 AM
To: solr-user@lucene.apache.org
Subject: Re: Index fetch failed
Hii,
Do you have enough memory free for the index chunk to be fetched/Downloaded on the slave node?
On Wed, Aug 28, 2019 at 6:57 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
> Hello Everyone,
>
> I am getting this error continuously on Solr slave, can anyone tell me
> the solution for this:
>
> 642141666 ERROR (indexFetcher-72-thread-1) [ x:sitecore_web_index]
> o.a.s.h.ReplicationHandler Index fetch failed
> :org.apache.solr.common.SolrException: Unable to download _12i7v_f.liv
> completely. Downloaded 0!=123
> at
> org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1434)
> at
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1314)
> at
> org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:812)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.jav
> a:427)
>
>
> Thanks & Regards,
> Akreeti Agarwal
> (M) +91-8318686601
>
> ::DISCLAIMER::
>
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> --------------------------------------------------------------------
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be
> intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
> may contain viruses in transmission. The e mail and its contents (with
> or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates. Views or opinions, if any,
> presented in this email are solely those of the author and may not
> necessarily reflect the views or opinions of HCL or its affiliates.
> Any form of reproduction, dissemination, copying, disclosure,
> modification, distribution and / or publication of this message
> without the prior written consent of authorized representative of HCL
> is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately. Before opening any
> email and/or attachments, please check them for viruses and other defects.
>
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
Re: Index fetch failed
Posted by Atita Arora <at...@gmail.com>.
Hii,
Do you have enough memory free for the index chunk to be fetched/Downloaded
on the slave node?
On Wed, Aug 28, 2019 at 6:57 AM Akreeti Agarwal <Ak...@hcl.com> wrote:
> Hello Everyone,
>
> I am getting this error continuously on Solr slave, can anyone tell me the
> solution for this:
>
> 642141666 ERROR (indexFetcher-72-thread-1) [ x:sitecore_web_index]
> o.a.s.h.ReplicationHandler Index fetch failed
> :org.apache.solr.common.SolrException: Unable to download _12i7v_f.liv
> completely. Downloaded 0!=123
> at
> org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1434)
> at
> org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1314)
> at
> org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:812)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:427)
>
>
> Thanks & Regards,
> Akreeti Agarwal
> (M) +91-8318686601
>
> ::DISCLAIMER::
>
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be intercepted,
> corrupted, lost, destroyed, arrive late or incomplete, or may contain
> viruses in transmission. The e mail and its contents (with or without
> referred errors) shall therefore not attach any liability on the originator
> or HCL or its affiliates. Views or opinions, if any, presented in this
> email are solely those of the author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification, distribution and / or
> publication of this message without the prior written consent of authorized
> representative of HCL is strictly prohibited. If you have received this
> email in error please delete it and notify the sender immediately. Before
> opening any email and/or attachments, please check them for viruses and
> other defects.
>
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>