You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Anil <an...@gmail.com> on 2018/06/11 12:41:53 UTC
clusterstate json check in Solrj
HI Team,
I was trying solrcloud cluster setup using solr 7.3.1 and it is up. Admin
console looks good and queries in console are working fine. But solrj
connection failing with following exception
org.apache.solr.common.SolrException: Cannot connect to cluster at
127.0.0.1:2181/solr : cluster not found/not ready
at
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:376)
~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:132)
~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:631)
~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1084)
~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at
com.apple.ist.cds.sseanalytics.repository.AnalyticsRepositoryImpl.getMetrics(AnalyticsRepositoryImpl.java:77)
[classes/:?]
at
com.apple.ist.cds.sseanalytics.repository.AnalyticsRepositoryImpl$$FastClassBySpringCGLIB$$1a4d1d78.invoke(<generated>)
[classes/:?]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
[spring-core-5.0.6.RELEASE.jar:5.0.6.RELEASE]
code : ZkStateReader.java
// Sanity check ZK structure.
if (!zkClient.exists(CLUSTER_STATE, true)) {
throw new SolrException(ErrorCode.SERVICE_UNAVAILABLE,
"Cannot connect to cluster at " +
zkClient.getZkServerAddress() + ": cluster not found/not ready");
}
but solr is not updating the cluster status in clusterstate.json and it is
empty and created state.json under each collection.
https://issues.apache.org/jira/browse/SOLR-5473
Can you please point out the issue here ? Thanks.
Regards,
Anil
Re: clusterstate json check in Solrj
Posted by Shawn Heisey <ap...@elyograg.org>.
On 6/11/2018 8:35 AM, Anil wrote:
> for failure could be #1. but not sure why client started with different
> zkHost. i used 127.0.01:2181/solr itself. zk nodes started with
> 127.0.0.1:2181, 2182, 2183
> is there anyway i can figure this out and correct it ? Thanks.
I suspect that the difference there is the /solr chroot. I bet what
you've got happening that the chroot is in the client's connection
information, but NOT in the server's connection information. For the
server to use a chroot with the information you have supplied, this is
what you zkHost string would look like:
127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183/solr
The client should be using exactly the same information -- all the
servers and the chroot (if present). If you only use one ZK server on
the client and that ZK server goes down, the client will stop working
correctly. If it has all the servers and one of them goes down, then
everything keeps working.
FYI -- because all your ZK servers are on one system, you do not
actually have redundancy. If that machine fails, ALL of your ZK servers
are gone.
Thanks,
Shawn
Re: clusterstate json check in Solrj
Posted by Anil <an...@gmail.com>.
HI Erick,
Yes. clusterstate.json is available in /solr path (i.e
/solr/clusterstate.json)
As Shawn said, client might be starting on different zknode. :(
Thanks,
Anil
On 11 June 2018 at 20:29, Erick Erickson <er...@gmail.com> wrote:
> _where_ is clusterstate.json? The connection string adds a solr
> prefix, so when you look at Zookeeper it should be in
> /solr/clusterstate.json rather than /clusterstate.json.
>
> When you use a different Zookeeper root, you need to use the exact
> same root on all your operations.
>
> As Shawn says, clusterstate.json should be empty and the individual
> collection's state.json should be where the state information is kept
> for each collection.
>
> Best,
> Erick
>
> On Mon, Jun 11, 2018 at 7:35 AM, Anil <an...@gmail.com> wrote:
> > HI Shawn,
> >
> > Thanks for response. please find comments in the context.
> >
> > Thanks,
> > Anil
> >
> >
> > On 11 June 2018 at 19:06, Shawn Heisey <ap...@elyograg.org> wrote:
> >
> >> On 6/11/2018 6:41 AM, Anil wrote:
> >>
> >>> I was trying solrcloud cluster setup using solr 7.3.1 and it is up.
> Admin
> >>> console looks good and queries in console are working fine. But solrj
> >>> connection failing with following exception
> >>>
> >>> org.apache.solr.common.SolrException: Cannot connect to cluster at
> >>> 127.0.0.1:2181/solr : cluster not found/not ready
> >>> at
> >>> org.apache.solr.common.cloud.ZkStateReader.createClusterStat
> >>> eWatchersAndUpdate(ZkStateReader.java:376)
> >>> ~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b
> -
> >>> sarowe - 2018-03-02 15:09:35]
> >>>
> >>
> >> If you're running Solr 7.3.1, then you should be using SolrJ 7.3.x, not
> >> 6.6.3. SolrCloud is evolving so rapidly that differences in version
> >> between these two components, especially when SolrJ is older, are not
> >> likely to work. This doesn't appear to be the cause of the specific
> error
> >> you are seeing, but assuming you can get past this error, it might lead
> to
> >> other problems.
> >>
> >
> > [Anil] : I tried with Solr 6.6 and solrj 6.6.3 as well. i see same
> > exception.
> >
> >>
> >> solr is not updating the cluster status in clusterstate.json and it is
> >>> empty and created state.json under each collection.
> >>>
> >>> https://issues.apache.org/jira/browse/SOLR-5473
> >>>
> >>> Can you please point out the issue here ? Thanks.
> >>>
> >>
> >> Recent versions of SolrCloud may not put any data in clusterstate.json,
> >> but this file is still used for cluster coordination, through ZK
> watches on
> >> the clusterstate.json znode. I do not know the details of how this
> works.
> >>
> >> Looking into the code, it appears that this error is saying that
> >> /clusterstate.json does not exist in zookeeper. This znode must exist.
> >> Since Solr creates that znode when it starts, I think there are two
> >> possible reasons for this error. 1) The Solr client is being started
> with
> >> a different zkHost value than the servers, so it is not finding the
> >> information written by the servers. 2) You are intentionally deleting
> the
> >> /clusterstate.json znode from zookeeper.
> >>
> >> [Anil] - i can see clusterstate.json in zookeeper. looks like the reason
> > for failure could be #1. but not sure why client started with different
> > zkHost. i used 127.0.01:2181/solr itself. zk nodes started with
> > 127.0.0.1:2181, 2182, 2183
> > is there anyway i can figure this out and correct it ? Thanks.
> >
> > Thanks,
> >> Shawn
> >>
> >>
>
Re: clusterstate json check in Solrj
Posted by Erick Erickson <er...@gmail.com>.
_where_ is clusterstate.json? The connection string adds a solr
prefix, so when you look at Zookeeper it should be in
/solr/clusterstate.json rather than /clusterstate.json.
When you use a different Zookeeper root, you need to use the exact
same root on all your operations.
As Shawn says, clusterstate.json should be empty and the individual
collection's state.json should be where the state information is kept
for each collection.
Best,
Erick
On Mon, Jun 11, 2018 at 7:35 AM, Anil <an...@gmail.com> wrote:
> HI Shawn,
>
> Thanks for response. please find comments in the context.
>
> Thanks,
> Anil
>
>
> On 11 June 2018 at 19:06, Shawn Heisey <ap...@elyograg.org> wrote:
>
>> On 6/11/2018 6:41 AM, Anil wrote:
>>
>>> I was trying solrcloud cluster setup using solr 7.3.1 and it is up. Admin
>>> console looks good and queries in console are working fine. But solrj
>>> connection failing with following exception
>>>
>>> org.apache.solr.common.SolrException: Cannot connect to cluster at
>>> 127.0.0.1:2181/solr : cluster not found/not ready
>>> at
>>> org.apache.solr.common.cloud.ZkStateReader.createClusterStat
>>> eWatchersAndUpdate(ZkStateReader.java:376)
>>> ~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
>>> sarowe - 2018-03-02 15:09:35]
>>>
>>
>> If you're running Solr 7.3.1, then you should be using SolrJ 7.3.x, not
>> 6.6.3. SolrCloud is evolving so rapidly that differences in version
>> between these two components, especially when SolrJ is older, are not
>> likely to work. This doesn't appear to be the cause of the specific error
>> you are seeing, but assuming you can get past this error, it might lead to
>> other problems.
>>
>
> [Anil] : I tried with Solr 6.6 and solrj 6.6.3 as well. i see same
> exception.
>
>>
>> solr is not updating the cluster status in clusterstate.json and it is
>>> empty and created state.json under each collection.
>>>
>>> https://issues.apache.org/jira/browse/SOLR-5473
>>>
>>> Can you please point out the issue here ? Thanks.
>>>
>>
>> Recent versions of SolrCloud may not put any data in clusterstate.json,
>> but this file is still used for cluster coordination, through ZK watches on
>> the clusterstate.json znode. I do not know the details of how this works.
>>
>> Looking into the code, it appears that this error is saying that
>> /clusterstate.json does not exist in zookeeper. This znode must exist.
>> Since Solr creates that znode when it starts, I think there are two
>> possible reasons for this error. 1) The Solr client is being started with
>> a different zkHost value than the servers, so it is not finding the
>> information written by the servers. 2) You are intentionally deleting the
>> /clusterstate.json znode from zookeeper.
>>
>> [Anil] - i can see clusterstate.json in zookeeper. looks like the reason
> for failure could be #1. but not sure why client started with different
> zkHost. i used 127.0.01:2181/solr itself. zk nodes started with
> 127.0.0.1:2181, 2182, 2183
> is there anyway i can figure this out and correct it ? Thanks.
>
> Thanks,
>> Shawn
>>
>>
Re: clusterstate json check in Solrj
Posted by Anil <an...@gmail.com>.
HI Shawn,
Thanks for response. please find comments in the context.
Thanks,
Anil
On 11 June 2018 at 19:06, Shawn Heisey <ap...@elyograg.org> wrote:
> On 6/11/2018 6:41 AM, Anil wrote:
>
>> I was trying solrcloud cluster setup using solr 7.3.1 and it is up. Admin
>> console looks good and queries in console are working fine. But solrj
>> connection failing with following exception
>>
>> org.apache.solr.common.SolrException: Cannot connect to cluster at
>> 127.0.0.1:2181/solr : cluster not found/not ready
>> at
>> org.apache.solr.common.cloud.ZkStateReader.createClusterStat
>> eWatchersAndUpdate(ZkStateReader.java:376)
>> ~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
>> sarowe - 2018-03-02 15:09:35]
>>
>
> If you're running Solr 7.3.1, then you should be using SolrJ 7.3.x, not
> 6.6.3. SolrCloud is evolving so rapidly that differences in version
> between these two components, especially when SolrJ is older, are not
> likely to work. This doesn't appear to be the cause of the specific error
> you are seeing, but assuming you can get past this error, it might lead to
> other problems.
>
[Anil] : I tried with Solr 6.6 and solrj 6.6.3 as well. i see same
exception.
>
> solr is not updating the cluster status in clusterstate.json and it is
>> empty and created state.json under each collection.
>>
>> https://issues.apache.org/jira/browse/SOLR-5473
>>
>> Can you please point out the issue here ? Thanks.
>>
>
> Recent versions of SolrCloud may not put any data in clusterstate.json,
> but this file is still used for cluster coordination, through ZK watches on
> the clusterstate.json znode. I do not know the details of how this works.
>
> Looking into the code, it appears that this error is saying that
> /clusterstate.json does not exist in zookeeper. This znode must exist.
> Since Solr creates that znode when it starts, I think there are two
> possible reasons for this error. 1) The Solr client is being started with
> a different zkHost value than the servers, so it is not finding the
> information written by the servers. 2) You are intentionally deleting the
> /clusterstate.json znode from zookeeper.
>
> [Anil] - i can see clusterstate.json in zookeeper. looks like the reason
for failure could be #1. but not sure why client started with different
zkHost. i used 127.0.01:2181/solr itself. zk nodes started with
127.0.0.1:2181, 2182, 2183
is there anyway i can figure this out and correct it ? Thanks.
Thanks,
> Shawn
>
>
Re: clusterstate json check in Solrj
Posted by Shawn Heisey <ap...@elyograg.org>.
On 6/11/2018 6:41 AM, Anil wrote:
> I was trying solrcloud cluster setup using solr 7.3.1 and it is up. Admin
> console looks good and queries in console are working fine. But solrj
> connection failing with following exception
>
> org.apache.solr.common.SolrException: Cannot connect to cluster at
> 127.0.0.1:2181/solr : cluster not found/not ready
> at
> org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:376)
> ~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
> sarowe - 2018-03-02 15:09:35]
If you're running Solr 7.3.1, then you should be using SolrJ 7.3.x, not
6.6.3. SolrCloud is evolving so rapidly that differences in version
between these two components, especially when SolrJ is older, are not
likely to work. This doesn't appear to be the cause of the specific
error you are seeing, but assuming you can get past this error, it might
lead to other problems.
> solr is not updating the cluster status in clusterstate.json and it is
> empty and created state.json under each collection.
>
> https://issues.apache.org/jira/browse/SOLR-5473
>
> Can you please point out the issue here ? Thanks.
Recent versions of SolrCloud may not put any data in clusterstate.json,
but this file is still used for cluster coordination, through ZK watches
on the clusterstate.json znode. I do not know the details of how this works.
Looking into the code, it appears that this error is saying that
/clusterstate.json does not exist in zookeeper. This znode must exist.
Since Solr creates that znode when it starts, I think there are two
possible reasons for this error. 1) The Solr client is being started
with a different zkHost value than the servers, so it is not finding the
information written by the servers. 2) You are intentionally deleting
the /clusterstate.json znode from zookeeper.
Thanks,
Shawn