You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by mtn search <se...@gmail.com> on 2021/05/17 21:35:21 UTC
Solr 8.8.2 Error on Distributed Index Query
Hello,
Has anyone observed a Distributed Index Query against a Solr 8 server fail
when a shard outside the cluster (and from a different Solr version) is
specified in the shards list? Note the shard white list is disabled. This
type of query works in both Solr 6.4.2 and 7.7.3 combinations, but not when
the query is made against a Solr 8.8.2 server.
Not sure if this is a bug that needs to be written up. Appreciate any
feedback. Understand that this is likely an unusual use case. This type
of query is being considered as part of a solution for a migration effort.
*Failing Distributed Index Query against Solr 8 server with Solr 8 and Solr
7 shards:*
http://localhost:8828/solr/techproduct_882/select?q=*%3A*&shards=http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1
{
"responseHeader":{
"zkConnected":true,
"status":500,
"QTime":108,
"params":{
"q":"*:*",
"shards":"http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1"}},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","java.io.IOException"],
"msg":"org.apache.solr.client.solrj.SolrServerException:
IOException occured when talking to server at: null",
"trace":"org.apache.solr.common.SolrException:
org.apache.solr.client.solrj.SolrServerException: *IOException occured
when talking to server at: null*\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:477)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:216)\n\tat
*Prior to the above call I am reminded to update or disable the shard
whitelist:*
http://localhost:8828/solr/techproduct_882/select?q=*%3A*&shards=http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1
{
"responseHeader":{
"status":403,
"QTime":5,
"params":{
"q":"*:*",
"shards":"http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1"}},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.common.SolrException"],
"msg":"The 'shards' parameter value
'http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1'
contained value(s) not on the shards whitelist.
shardUrl:http://localhost:8828/solr/techproduct_882_shard1_replica_n1.*
set -Dsolr.disable.shardsWhitelist=true *to disable shards whitelist
checks",
"code":403}}
*Starting Solr 8 server with Whitelist disabled (with this flag
enabled is when the first error listed above is observed):*
bin/solr start -c -p 8828 -Dsolr.disable.shardsWhitelist=true -z
localhost:2181/solr8828
Thanks,
Mattt
Re: Solr 8.8.2 Error on Distributed Index Query
Posted by mtn search <se...@gmail.com>.
Thanks Koji!
This is helpful information. I believe in the back of my mind I knew I
would need to consider compatible field types on this type of query across
versions. Your example makes a clear point of it.
As I mentioned as a part of my experimentation I was able to make the query
work (with a simple example Tech Product schema and now sorting) across
Solr 6/7 and forgot to mention I was able to make it work for Solr 6 and 8
if I sent the query to the Solr 6 server. What I could not understand is
why it failed when making the query against a Solr 8 server. Your
statement regarding 8 where HttpShardHandlerFactory uses http2 seems to
explain the failure I am seeing.
Matt
On Mon, May 17, 2021 at 8:06 PM Koji Sekiguchi <ko...@rondhuit.com>
wrote:
> Hi Mattt,
>
> It sounds interesting use case, but does Solr officially support
> distributed search among different
> versions? I don't think so, because I can think of some cases that don't
> work correctly.
>
> - From Solr 8(?), HttpShardHandlerFactory uses http2 that cannot talk to
> older Solr servers.
> - Some field types are incompatible with different Solr versions in Lucene
> level and it can be
> problem when comparing field values for sorting. As for date field type
> for example, old Lucene used
> java.util.Date (I'm sorry if it's wrong, but this is just an example)
> while if the latest Lucene
> uses long, you'll get runtime exception when you try to sort on that field.
>
> Koji
>
> On 2021/05/18 6:35, mtn search wrote:
> > Hello,
> >
> > Has anyone observed a Distributed Index Query against a Solr 8 server
> fail
> > when a shard outside the cluster (and from a different Solr version) is
> > specified in the shards list? Note the shard white list is disabled.
> This
> > type of query works in both Solr 6.4.2 and 7.7.3 combinations, but not
> when
> > the query is made against a Solr 8.8.2 server.
> >
> > Not sure if this is a bug that needs to be written up. Appreciate any
> > feedback. Understand that this is likely an unusual use case. This type
> > of query is being considered as part of a solution for a migration
> effort.
> >
> > *Failing Distributed Index Query against Solr 8 server with Solr 8 and
> Solr
> > 7 shards:*
> >
> >
> http://localhost:8828/solr/techproduct_882/select?q=*%3A*&shards=http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1
> >
> > {
> > "responseHeader":{
> > "zkConnected":true,
> > "status":500,
> > "QTime":108,
> > "params":{
> > "q":"*:*",
> > "shards":"
> http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1
> "}},
> > "error":{
> > "metadata":[
> > "error-class","org.apache.solr.common.SolrException",
> > "root-error-class","java.io.IOException"],
> > "msg":"org.apache.solr.client.solrj.SolrServerException:
> > IOException occured when talking to server at: null",
> > "trace":"org.apache.solr.common.SolrException:
> > org.apache.solr.client.solrj.SolrServerException: *IOException occured
> > when talking to server at: null*\n\tat
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:477)\n\tat
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:216)\n\tat
> >
> >
> > *Prior to the above call I am reminded to update or disable the shard
> > whitelist:*
> >
> >
> http://localhost:8828/solr/techproduct_882/select?q=*%3A*&shards=http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1
> >
> > {
> >
> > "responseHeader":{
> >
> > "status":403,
> >
> > "QTime":5,
> >
> > "params":{
> >
> > "q":"*:*",
> >
> > "shards":"
> http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1
> "}},
> >
> > "error":{
> >
> > "metadata":[
> >
> > "error-class","org.apache.solr.common.SolrException",
> >
> > "root-error-class","org.apache.solr.common.SolrException"],
> >
> > "msg":"The 'shards' parameter value
> > '
> http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1
> '
> > contained value(s) not on the shards whitelist.
> > shardUrl:http://localhost:8828/solr/techproduct_882_shard1_replica_n1.*
> > set -Dsolr.disable.shardsWhitelist=true *to disable shards whitelist
> > checks",
> >
> > "code":403}}
> >
> >
> > *Starting Solr 8 server with Whitelist disabled (with this flag
> > enabled is when the first error listed above is observed):*
> >
> > bin/solr start -c -p 8828 -Dsolr.disable.shardsWhitelist=true -z
> > localhost:2181/solr8828
> >
> > Thanks,
> > Mattt
> >
>
Re: Solr 8.8.2 Error on Distributed Index Query
Posted by Koji Sekiguchi <ko...@rondhuit.com>.
Hi Mattt,
It sounds interesting use case, but does Solr officially support distributed search among different
versions? I don't think so, because I can think of some cases that don't work correctly.
- From Solr 8(?), HttpShardHandlerFactory uses http2 that cannot talk to older Solr servers.
- Some field types are incompatible with different Solr versions in Lucene level and it can be
problem when comparing field values for sorting. As for date field type for example, old Lucene used
java.util.Date (I'm sorry if it's wrong, but this is just an example) while if the latest Lucene
uses long, you'll get runtime exception when you try to sort on that field.
Koji
On 2021/05/18 6:35, mtn search wrote:
> Hello,
>
> Has anyone observed a Distributed Index Query against a Solr 8 server fail
> when a shard outside the cluster (and from a different Solr version) is
> specified in the shards list? Note the shard white list is disabled. This
> type of query works in both Solr 6.4.2 and 7.7.3 combinations, but not when
> the query is made against a Solr 8.8.2 server.
>
> Not sure if this is a bug that needs to be written up. Appreciate any
> feedback. Understand that this is likely an unusual use case. This type
> of query is being considered as part of a solution for a migration effort.
>
> *Failing Distributed Index Query against Solr 8 server with Solr 8 and Solr
> 7 shards:*
>
> http://localhost:8828/solr/techproduct_882/select?q=*%3A*&shards=http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1
>
> {
> "responseHeader":{
> "zkConnected":true,
> "status":500,
> "QTime":108,
> "params":{
> "q":"*:*",
> "shards":"http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1"}},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","java.io.IOException"],
> "msg":"org.apache.solr.client.solrj.SolrServerException:
> IOException occured when talking to server at: null",
> "trace":"org.apache.solr.common.SolrException:
> org.apache.solr.client.solrj.SolrServerException: *IOException occured
> when talking to server at: null*\n\tat
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:477)\n\tat
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:216)\n\tat
>
>
> *Prior to the above call I am reminded to update or disable the shard
> whitelist:*
>
> http://localhost:8828/solr/techproduct_882/select?q=*%3A*&shards=http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1
>
> {
>
> "responseHeader":{
>
> "status":403,
>
> "QTime":5,
>
> "params":{
>
> "q":"*:*",
>
> "shards":"http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1"}},
>
> "error":{
>
> "metadata":[
>
> "error-class","org.apache.solr.common.SolrException",
>
> "root-error-class","org.apache.solr.common.SolrException"],
>
> "msg":"The 'shards' parameter value
> 'http://localhost:8828/solr/techproduct_882_shard1_replica_n1,http://localhost:8828/solr/techproduct_882_shard2_replica_n2,http://localhost:7738/solr/techproduct_773_shard1_replica_n1'
> contained value(s) not on the shards whitelist.
> shardUrl:http://localhost:8828/solr/techproduct_882_shard1_replica_n1.*
> set -Dsolr.disable.shardsWhitelist=true *to disable shards whitelist
> checks",
>
> "code":403}}
>
>
> *Starting Solr 8 server with Whitelist disabled (with this flag
> enabled is when the first error listed above is observed):*
>
> bin/solr start -c -p 8828 -Dsolr.disable.shardsWhitelist=true -z
> localhost:2181/solr8828
>
> Thanks,
> Mattt
>