You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ahmed Adel <ah...@badrit.com> on 2015/04/09 12:07:05 UTC
clusterstate.json is sometimes out-of-sync
Hi All,
On Solr 5.0 and ZK 3.4.6 sometimes clusterstate.json does not reflect the
aggregation of states of collections, the latter is always correct. I could
verify this from the admin panel (under Tree view) and from ZKCli. Is there
something I'm missing that could generate this issue?
--
A. Adel
clusterstate.json is sometimes out-of-sync
Posted by Ahmed Adel <ah...@badrit.com>.
What I really meant is trying to get cluster status directly through ZK
API. Your approach a bit different from what I meant but it's a nice one as
it seems will work across versions 4 and 5.
On Thursday, April 9, 2015, Shalin Shekhar Mangar <shalinmangar@gmail.com
<javascript:_e(%7B%7D,'cvml','shalinmangar@gmail.com');>> wrote:
> I don't quite follow. Are you saying that you intend to use the ZK REST API
> to fetch live_nodes and then send the 'clusterstatus' API call to one of
> the live nodes?
>
> On Thu, Apr 9, 2015 at 7:13 PM, Ahmed Adel <ah...@badrit.com> wrote:
>
> > In fact, the advantage I see of using ZK is that we don't have to iterate
> > over nodes in case the first node receiving that request is down,
> whereas,
> > by using ZK REST API, we can do that in a single request as I assume we
> can
> > check live_nodes (in case this approach is guaranteed when using Solr
> 4.x)
> > and send the request directly to a live node. Let me know if this makes
> > sense.
> >
> > On Thu, Apr 9, 2015 at 2:31 PM, Shalin Shekhar Mangar <
> > shalinmangar@gmail.com> wrote:
> >
> > > Yes, you can use the 'clusterstatus' API which will return an
> aggregation
> > > of all states. See
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api18
> > >
> > >
> > > On Thu, Apr 9, 2015 at 5:52 PM, Ahmed Adel <ah...@badrit.com>
> > wrote:
> > >
> > > > Hi Shalin,
> > > >
> > > > Thanks for your response. I'm actually looking inside ZooKeeper in
> > order
> > > to
> > > > obtain highest availability. What I expected is that
> clusterstate.json
> > > > contains the aggregation of all state.json children nodes of each
> > > > collection. But your second paragraph explains the behavior I see in
> > Solr
> > > > 5.0 while others using prior versions of Solr don't see.
> > > >
> > > > By the way, is there one method to retrieve state across 4.x and 5.x?
> > It
> > > > seems that there are different methods depending on Solr version.
> > > >
> > > > On Thu, Apr 9, 2015 at 12:23 PM, Shalin Shekhar Mangar <
> > > > shalinmangar@gmail.com> wrote:
> > > >
> > > > > Hi Ahmed,
> > > > >
> > > > > Can you give more details? What did you expect and what was the
> > actual?
> > > > > Also, are you looking directly at the clusterstate.json inside
> > > ZooKeeper
> > > > or
> > > > > are you using the 'clusterstatus' Collection API?
> > > > >
> > > > > You shouldn't look at the clusterstate.json directly because 1)
> > things
> > > > like
> > > > > live-ness is not stored in clusterstate.json and 2) collections
> > created
> > > > > with Solr 5.0 have their own individual state.json inside
> > > > > /collections/collection_name/state.json
> > > > >
> > > > > On Thu, Apr 9, 2015 at 3:37 PM, Ahmed Adel <ah...@badrit.com>
> > > > wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > On Solr 5.0 and ZK 3.4.6 sometimes clusterstate.json does not
> > reflect
> > > > the
> > > > > > aggregation of states of collections, the latter is always
> > correct. I
> > > > > could
> > > > > > verify this from the admin panel (under Tree view) and from
> ZKCli.
> > Is
> > > > > there
> > > > > > something I'm missing that could generate this issue?
> > > > > >
> > > > > > --
> > > > > >
> > > > > > A. Adel
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > > Shalin Shekhar Mangar.
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > A. Adel
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Shalin Shekhar Mangar.
> > >
> >
> >
> >
> > --
> > *Ahmed Adel*
> > www.badrit.com
> > <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2F>
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
--
Sent from my iPhone
Re: clusterstate.json is sometimes out-of-sync
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
I don't quite follow. Are you saying that you intend to use the ZK REST API
to fetch live_nodes and then send the 'clusterstatus' API call to one of
the live nodes?
On Thu, Apr 9, 2015 at 7:13 PM, Ahmed Adel <ah...@badrit.com> wrote:
> In fact, the advantage I see of using ZK is that we don't have to iterate
> over nodes in case the first node receiving that request is down, whereas,
> by using ZK REST API, we can do that in a single request as I assume we can
> check live_nodes (in case this approach is guaranteed when using Solr 4.x)
> and send the request directly to a live node. Let me know if this makes
> sense.
>
> On Thu, Apr 9, 2015 at 2:31 PM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
> > Yes, you can use the 'clusterstatus' API which will return an aggregation
> > of all states. See
> >
> >
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api18
> >
> >
> > On Thu, Apr 9, 2015 at 5:52 PM, Ahmed Adel <ah...@badrit.com>
> wrote:
> >
> > > Hi Shalin,
> > >
> > > Thanks for your response. I'm actually looking inside ZooKeeper in
> order
> > to
> > > obtain highest availability. What I expected is that clusterstate.json
> > > contains the aggregation of all state.json children nodes of each
> > > collection. But your second paragraph explains the behavior I see in
> Solr
> > > 5.0 while others using prior versions of Solr don't see.
> > >
> > > By the way, is there one method to retrieve state across 4.x and 5.x?
> It
> > > seems that there are different methods depending on Solr version.
> > >
> > > On Thu, Apr 9, 2015 at 12:23 PM, Shalin Shekhar Mangar <
> > > shalinmangar@gmail.com> wrote:
> > >
> > > > Hi Ahmed,
> > > >
> > > > Can you give more details? What did you expect and what was the
> actual?
> > > > Also, are you looking directly at the clusterstate.json inside
> > ZooKeeper
> > > or
> > > > are you using the 'clusterstatus' Collection API?
> > > >
> > > > You shouldn't look at the clusterstate.json directly because 1)
> things
> > > like
> > > > live-ness is not stored in clusterstate.json and 2) collections
> created
> > > > with Solr 5.0 have their own individual state.json inside
> > > > /collections/collection_name/state.json
> > > >
> > > > On Thu, Apr 9, 2015 at 3:37 PM, Ahmed Adel <ah...@badrit.com>
> > > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > On Solr 5.0 and ZK 3.4.6 sometimes clusterstate.json does not
> reflect
> > > the
> > > > > aggregation of states of collections, the latter is always
> correct. I
> > > > could
> > > > > verify this from the admin panel (under Tree view) and from ZKCli.
> Is
> > > > there
> > > > > something I'm missing that could generate this issue?
> > > > >
> > > > > --
> > > > >
> > > > > A. Adel
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Shalin Shekhar Mangar.
> > > >
> > >
> > >
> > >
> > > --
> > > A. Adel
> > >
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>
>
>
> --
> *Ahmed Adel*
> www.badrit.com
> <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2F>
>
--
Regards,
Shalin Shekhar Mangar.
Re: clusterstate.json is sometimes out-of-sync
Posted by Ahmed Adel <ah...@badrit.com>.
In fact, the advantage I see of using ZK is that we don't have to iterate
over nodes in case the first node receiving that request is down, whereas,
by using ZK REST API, we can do that in a single request as I assume we can
check live_nodes (in case this approach is guaranteed when using Solr 4.x)
and send the request directly to a live node. Let me know if this makes
sense.
On Thu, Apr 9, 2015 at 2:31 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> Yes, you can use the 'clusterstatus' API which will return an aggregation
> of all states. See
>
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api18
>
>
> On Thu, Apr 9, 2015 at 5:52 PM, Ahmed Adel <ah...@badrit.com> wrote:
>
> > Hi Shalin,
> >
> > Thanks for your response. I'm actually looking inside ZooKeeper in order
> to
> > obtain highest availability. What I expected is that clusterstate.json
> > contains the aggregation of all state.json children nodes of each
> > collection. But your second paragraph explains the behavior I see in Solr
> > 5.0 while others using prior versions of Solr don't see.
> >
> > By the way, is there one method to retrieve state across 4.x and 5.x? It
> > seems that there are different methods depending on Solr version.
> >
> > On Thu, Apr 9, 2015 at 12:23 PM, Shalin Shekhar Mangar <
> > shalinmangar@gmail.com> wrote:
> >
> > > Hi Ahmed,
> > >
> > > Can you give more details? What did you expect and what was the actual?
> > > Also, are you looking directly at the clusterstate.json inside
> ZooKeeper
> > or
> > > are you using the 'clusterstatus' Collection API?
> > >
> > > You shouldn't look at the clusterstate.json directly because 1) things
> > like
> > > live-ness is not stored in clusterstate.json and 2) collections created
> > > with Solr 5.0 have their own individual state.json inside
> > > /collections/collection_name/state.json
> > >
> > > On Thu, Apr 9, 2015 at 3:37 PM, Ahmed Adel <ah...@badrit.com>
> > wrote:
> > >
> > > > Hi All,
> > > >
> > > > On Solr 5.0 and ZK 3.4.6 sometimes clusterstate.json does not reflect
> > the
> > > > aggregation of states of collections, the latter is always correct. I
> > > could
> > > > verify this from the admin panel (under Tree view) and from ZKCli. Is
> > > there
> > > > something I'm missing that could generate this issue?
> > > >
> > > > --
> > > >
> > > > A. Adel
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Shalin Shekhar Mangar.
> > >
> >
> >
> >
> > --
> > A. Adel
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
--
*Ahmed Adel*
www.badrit.com
<http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2F>
Re: clusterstate.json is sometimes out-of-sync
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Yes, you can use the 'clusterstatus' API which will return an aggregation
of all states. See
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api18
On Thu, Apr 9, 2015 at 5:52 PM, Ahmed Adel <ah...@badrit.com> wrote:
> Hi Shalin,
>
> Thanks for your response. I'm actually looking inside ZooKeeper in order to
> obtain highest availability. What I expected is that clusterstate.json
> contains the aggregation of all state.json children nodes of each
> collection. But your second paragraph explains the behavior I see in Solr
> 5.0 while others using prior versions of Solr don't see.
>
> By the way, is there one method to retrieve state across 4.x and 5.x? It
> seems that there are different methods depending on Solr version.
>
> On Thu, Apr 9, 2015 at 12:23 PM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
> > Hi Ahmed,
> >
> > Can you give more details? What did you expect and what was the actual?
> > Also, are you looking directly at the clusterstate.json inside ZooKeeper
> or
> > are you using the 'clusterstatus' Collection API?
> >
> > You shouldn't look at the clusterstate.json directly because 1) things
> like
> > live-ness is not stored in clusterstate.json and 2) collections created
> > with Solr 5.0 have their own individual state.json inside
> > /collections/collection_name/state.json
> >
> > On Thu, Apr 9, 2015 at 3:37 PM, Ahmed Adel <ah...@badrit.com>
> wrote:
> >
> > > Hi All,
> > >
> > > On Solr 5.0 and ZK 3.4.6 sometimes clusterstate.json does not reflect
> the
> > > aggregation of states of collections, the latter is always correct. I
> > could
> > > verify this from the admin panel (under Tree view) and from ZKCli. Is
> > there
> > > something I'm missing that could generate this issue?
> > >
> > > --
> > >
> > > A. Adel
> > >
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>
>
>
> --
> A. Adel
>
--
Regards,
Shalin Shekhar Mangar.
Re: clusterstate.json is sometimes out-of-sync
Posted by Ahmed Adel <ah...@badrit.com>.
Hi Shalin,
Thanks for your response. I'm actually looking inside ZooKeeper in order to
obtain highest availability. What I expected is that clusterstate.json
contains the aggregation of all state.json children nodes of each
collection. But your second paragraph explains the behavior I see in Solr
5.0 while others using prior versions of Solr don't see.
By the way, is there one method to retrieve state across 4.x and 5.x? It
seems that there are different methods depending on Solr version.
On Thu, Apr 9, 2015 at 12:23 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> Hi Ahmed,
>
> Can you give more details? What did you expect and what was the actual?
> Also, are you looking directly at the clusterstate.json inside ZooKeeper or
> are you using the 'clusterstatus' Collection API?
>
> You shouldn't look at the clusterstate.json directly because 1) things like
> live-ness is not stored in clusterstate.json and 2) collections created
> with Solr 5.0 have their own individual state.json inside
> /collections/collection_name/state.json
>
> On Thu, Apr 9, 2015 at 3:37 PM, Ahmed Adel <ah...@badrit.com> wrote:
>
> > Hi All,
> >
> > On Solr 5.0 and ZK 3.4.6 sometimes clusterstate.json does not reflect the
> > aggregation of states of collections, the latter is always correct. I
> could
> > verify this from the admin panel (under Tree view) and from ZKCli. Is
> there
> > something I'm missing that could generate this issue?
> >
> > --
> >
> > A. Adel
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
--
A. Adel
Re: clusterstate.json is sometimes out-of-sync
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Hi Ahmed,
Can you give more details? What did you expect and what was the actual?
Also, are you looking directly at the clusterstate.json inside ZooKeeper or
are you using the 'clusterstatus' Collection API?
You shouldn't look at the clusterstate.json directly because 1) things like
live-ness is not stored in clusterstate.json and 2) collections created
with Solr 5.0 have their own individual state.json inside
/collections/collection_name/state.json
On Thu, Apr 9, 2015 at 3:37 PM, Ahmed Adel <ah...@badrit.com> wrote:
> Hi All,
>
> On Solr 5.0 and ZK 3.4.6 sometimes clusterstate.json does not reflect the
> aggregation of states of collections, the latter is always correct. I could
> verify this from the admin panel (under Tree view) and from ZKCli. Is there
> something I'm missing that could generate this issue?
>
> --
>
> A. Adel
>
--
Regards,
Shalin Shekhar Mangar.