You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Susheel Kumar <su...@gmail.com> on 2017/05/16 16:41:47 UTC
Shard marked as down while its operational & SOLR-9120
Hello,
One of the shard (out of 6 six shards and 6 replica's) we have, is being
shown as down in Solr Admin Cloud Panel (orange color) while the Solr
process is running and even can connect to it Solr Admin Cloud panel. Each
shard size is around 18GB and have around 40+ million docs.
Also I do notice lots of below warnings as described by SOLR-9120.
Any idea what is going on and if it is due to SOLR-9120.
Thanks,
Susheel
LukeRequestHandler Error getting file length for [segments_1p6l]
java.nio.file.NoSuchFileException:
..._shard3_replica1/data/index.20170128034221957/segments_1p6l
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
Re: Shard marked as down while its operational & SOLR-9120
Posted by Susheel Kumar <su...@gmail.com>.
Do not see any similar issue fixed part of later releases.
Anyone experienced similar issue / suggestions to look into?
Thanks,
Susheel
On Tue, May 16, 2017 at 2:43 PM, Walter Underwood <wu...@wunderwood.org>
wrote:
> Look at all the bugs fixed or reported after 6.0.0. This might have been
> reported and their might be a workaround.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
>
> > On May 16, 2017, at 11:41 AM, Susheel Kumar <su...@gmail.com>
> wrote:
> >
> > Hi Walter - will upgrade to 6.5.1 but this will take time to go from all
> > the environments to Prod.
> >
> > Looking for something short term while we upgrade.
> >
> > Though we can connect to Solr Admin panel on the shard which is down but
> if
> > I query shards.info, then that shard is not being shown/queried. So
> its
> > partially down
> >
> > "*shards.info <http://shards.info>*":{
> >
> > "*http://server1:8080/solr/COL_shard3_replica2/
> > <http://server1:8080/solr/COL_shard3_replica2/>*":{
> >
> > "*numFound*":38013303,
> >
> > "*maxScore*":1.0,
> >
> > "*shardAddress*":"http://server1:8080/solr/COL_shard3_replica2/",
> >
> > "*time*":2},
> >
> > "*http://server2:8080/solr/COL_shard4_replica2/|http://
> server3:8080/solr/COL_shard4_replica1/
> > <http://server2:8080/solr/COL_shard4_replica2/|http://
> server3:8080/solr/COL_shard4_replica1/>*
> > ":{
> >
> > "*numFound*":43816942,
> >
> > "*maxScore*":1.0,
> >
> > "*shardAddress*":"http://server3:8080/solr/COL_shard4_replica1",
> >
> > "*time*":2},
> >
> > …….....
> >
> > ………..
> >
> >
> >
> > On Tue, May 16, 2017 at 1:55 PM, Walter Underwood <wunder@wunderwood.org
> >
> > wrote:
> >
> >> I would upgrade to 6.5.1 before doing anything else. 6.0.0 is more than
> a
> >> year old.
> >>
> >> wunder
> >> Walter Underwood
> >> wunder@wunderwood.org
> >> http://observer.wunderwood.org/ (my blog)
> >>
> >>
> >>> On May 16, 2017, at 10:27 AM, Susheel Kumar <su...@gmail.com>
> >> wrote:
> >>>
> >>> Also this is for Solr-6.0.0.
> >>>
> >>> On Tue, May 16, 2017 at 12:41 PM, Susheel Kumar <susheel2777@gmail.com
> >
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> One of the shard (out of 6 six shards and 6 replica's) we have, is
> being
> >>>> shown as down in Solr Admin Cloud Panel (orange color) while the Solr
> >>>> process is running and even can connect to it Solr Admin Cloud panel.
> >> Each
> >>>> shard size is around 18GB and have around 40+ million docs.
> >>>>
> >>>> Also I do notice lots of below warnings as described by SOLR-9120.
> >>>>
> >>>> Any idea what is going on and if it is due to SOLR-9120.
> >>>>
> >>>> Thanks,
> >>>> Susheel
> >>>>
> >>>> LukeRequestHandler Error getting file length for [segments_1p6l]
> >>>>
> >>>> java.nio.file.NoSuchFileException: ..._shard3_replica1/data/
> >> index.20170128034221957/segments_1p6l
> >>>> at sun.nio.fs.UnixException.translateToIOException(
> >> UnixException.java:86)
> >>>> at sun.nio.fs.UnixException.rethrowAsIOException(
> >> UnixException.java:102)
> >>>> at sun.nio.fs.UnixException.rethrowAsIOException(
> >> UnixException.java:107)
> >>>> at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(
> >> UnixFileAttributeViews.java:55)
> >>>>
> >>>>
> >>
> >>
>
>
Re: Shard marked as down while its operational & SOLR-9120
Posted by Walter Underwood <wu...@wunderwood.org>.
Look at all the bugs fixed or reported after 6.0.0. This might have been reported and their might be a workaround.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On May 16, 2017, at 11:41 AM, Susheel Kumar <su...@gmail.com> wrote:
>
> Hi Walter - will upgrade to 6.5.1 but this will take time to go from all
> the environments to Prod.
>
> Looking for something short term while we upgrade.
>
> Though we can connect to Solr Admin panel on the shard which is down but if
> I query shards.info, then that shard is not being shown/queried. So its
> partially down
>
> "*shards.info <http://shards.info>*":{
>
> "*http://server1:8080/solr/COL_shard3_replica2/
> <http://server1:8080/solr/COL_shard3_replica2/>*":{
>
> "*numFound*":38013303,
>
> "*maxScore*":1.0,
>
> "*shardAddress*":"http://server1:8080/solr/COL_shard3_replica2/",
>
> "*time*":2},
>
> "*http://server2:8080/solr/COL_shard4_replica2/|http://server3:8080/solr/COL_shard4_replica1/
> <http://server2:8080/solr/COL_shard4_replica2/|http://server3:8080/solr/COL_shard4_replica1/>*
> ":{
>
> "*numFound*":43816942,
>
> "*maxScore*":1.0,
>
> "*shardAddress*":"http://server3:8080/solr/COL_shard4_replica1",
>
> "*time*":2},
>
> …….....
>
> ………..
>
>
>
> On Tue, May 16, 2017 at 1:55 PM, Walter Underwood <wu...@wunderwood.org>
> wrote:
>
>> I would upgrade to 6.5.1 before doing anything else. 6.0.0 is more than a
>> year old.
>>
>> wunder
>> Walter Underwood
>> wunder@wunderwood.org
>> http://observer.wunderwood.org/ (my blog)
>>
>>
>>> On May 16, 2017, at 10:27 AM, Susheel Kumar <su...@gmail.com>
>> wrote:
>>>
>>> Also this is for Solr-6.0.0.
>>>
>>> On Tue, May 16, 2017 at 12:41 PM, Susheel Kumar <su...@gmail.com>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> One of the shard (out of 6 six shards and 6 replica's) we have, is being
>>>> shown as down in Solr Admin Cloud Panel (orange color) while the Solr
>>>> process is running and even can connect to it Solr Admin Cloud panel.
>> Each
>>>> shard size is around 18GB and have around 40+ million docs.
>>>>
>>>> Also I do notice lots of below warnings as described by SOLR-9120.
>>>>
>>>> Any idea what is going on and if it is due to SOLR-9120.
>>>>
>>>> Thanks,
>>>> Susheel
>>>>
>>>> LukeRequestHandler Error getting file length for [segments_1p6l]
>>>>
>>>> java.nio.file.NoSuchFileException: ..._shard3_replica1/data/
>> index.20170128034221957/segments_1p6l
>>>> at sun.nio.fs.UnixException.translateToIOException(
>> UnixException.java:86)
>>>> at sun.nio.fs.UnixException.rethrowAsIOException(
>> UnixException.java:102)
>>>> at sun.nio.fs.UnixException.rethrowAsIOException(
>> UnixException.java:107)
>>>> at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(
>> UnixFileAttributeViews.java:55)
>>>>
>>>>
>>
>>
Re: Shard marked as down while its operational & SOLR-9120
Posted by Susheel Kumar <su...@gmail.com>.
Hi Walter - will upgrade to 6.5.1 but this will take time to go from all
the environments to Prod.
Looking for something short term while we upgrade.
Though we can connect to Solr Admin panel on the shard which is down but if
I query shards.info, then that shard is not being shown/queried. So its
partially down
"*shards.info <http://shards.info>*":{
"*http://server1:8080/solr/COL_shard3_replica2/
<http://server1:8080/solr/COL_shard3_replica2/>*":{
"*numFound*":38013303,
"*maxScore*":1.0,
"*shardAddress*":"http://server1:8080/solr/COL_shard3_replica2/",
"*time*":2},
"*http://server2:8080/solr/COL_shard4_replica2/|http://server3:8080/solr/COL_shard4_replica1/
<http://server2:8080/solr/COL_shard4_replica2/|http://server3:8080/solr/COL_shard4_replica1/>*
":{
"*numFound*":43816942,
"*maxScore*":1.0,
"*shardAddress*":"http://server3:8080/solr/COL_shard4_replica1",
"*time*":2},
…….....
………..
On Tue, May 16, 2017 at 1:55 PM, Walter Underwood <wu...@wunderwood.org>
wrote:
> I would upgrade to 6.5.1 before doing anything else. 6.0.0 is more than a
> year old.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
>
> > On May 16, 2017, at 10:27 AM, Susheel Kumar <su...@gmail.com>
> wrote:
> >
> > Also this is for Solr-6.0.0.
> >
> > On Tue, May 16, 2017 at 12:41 PM, Susheel Kumar <su...@gmail.com>
> > wrote:
> >
> >> Hello,
> >>
> >> One of the shard (out of 6 six shards and 6 replica's) we have, is being
> >> shown as down in Solr Admin Cloud Panel (orange color) while the Solr
> >> process is running and even can connect to it Solr Admin Cloud panel.
> Each
> >> shard size is around 18GB and have around 40+ million docs.
> >>
> >> Also I do notice lots of below warnings as described by SOLR-9120.
> >>
> >> Any idea what is going on and if it is due to SOLR-9120.
> >>
> >> Thanks,
> >> Susheel
> >>
> >> LukeRequestHandler Error getting file length for [segments_1p6l]
> >>
> >> java.nio.file.NoSuchFileException: ..._shard3_replica1/data/
> index.20170128034221957/segments_1p6l
> >> at sun.nio.fs.UnixException.translateToIOException(
> UnixException.java:86)
> >> at sun.nio.fs.UnixException.rethrowAsIOException(
> UnixException.java:102)
> >> at sun.nio.fs.UnixException.rethrowAsIOException(
> UnixException.java:107)
> >> at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(
> UnixFileAttributeViews.java:55)
> >>
> >>
>
>
Re: Shard marked as down while its operational & SOLR-9120
Posted by Walter Underwood <wu...@wunderwood.org>.
I would upgrade to 6.5.1 before doing anything else. 6.0.0 is more than a year old.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On May 16, 2017, at 10:27 AM, Susheel Kumar <su...@gmail.com> wrote:
>
> Also this is for Solr-6.0.0.
>
> On Tue, May 16, 2017 at 12:41 PM, Susheel Kumar <su...@gmail.com>
> wrote:
>
>> Hello,
>>
>> One of the shard (out of 6 six shards and 6 replica's) we have, is being
>> shown as down in Solr Admin Cloud Panel (orange color) while the Solr
>> process is running and even can connect to it Solr Admin Cloud panel. Each
>> shard size is around 18GB and have around 40+ million docs.
>>
>> Also I do notice lots of below warnings as described by SOLR-9120.
>>
>> Any idea what is going on and if it is due to SOLR-9120.
>>
>> Thanks,
>> Susheel
>>
>> LukeRequestHandler Error getting file length for [segments_1p6l]
>>
>> java.nio.file.NoSuchFileException: ..._shard3_replica1/data/index.20170128034221957/segments_1p6l
>> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>> at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
>>
>>
Re: Shard marked as down while its operational & SOLR-9120
Posted by Susheel Kumar <su...@gmail.com>.
Also this is for Solr-6.0.0.
On Tue, May 16, 2017 at 12:41 PM, Susheel Kumar <su...@gmail.com>
wrote:
> Hello,
>
> One of the shard (out of 6 six shards and 6 replica's) we have, is being
> shown as down in Solr Admin Cloud Panel (orange color) while the Solr
> process is running and even can connect to it Solr Admin Cloud panel. Each
> shard size is around 18GB and have around 40+ million docs.
>
> Also I do notice lots of below warnings as described by SOLR-9120.
>
> Any idea what is going on and if it is due to SOLR-9120.
>
> Thanks,
> Susheel
>
> LukeRequestHandler Error getting file length for [segments_1p6l]
>
> java.nio.file.NoSuchFileException: ..._shard3_replica1/data/index.20170128034221957/segments_1p6l
> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
>
>