You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by 王立生 <wa...@gmail.com> on 2019/08/20 01:39:53 UTC
question about solrCloud joining
Hello,
I have a question about solrCloud joining. i knew solrCloud joining can do
join only when index is not splited to shards, but when i test it, i found
a problem which make me confused.
i tested it on version 8.2
assuming i have 2 collections like sample about "joining" on solr offcial
website,
one collection called "movies", another called "movieDirectors".
movies's fields: id, title, director_id
movieDirectors's fields: id, name, has_oscar
the information of shards and replicas as below, i started two nodes on my
laptop:
[image: image.png]
moviesDirectors have 2 docs:
[image: image.png]
movies also have 2 docs:
[image: image.png]
everything is ok when i run query with "{!join from=id
fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983 and
8984, i can got expacted result:
[image: image.png]
but when i run "{!join from=director_id fromIndex=movies
to=id}title:"Dunkirk"" on 8983
got 1 doc and if i filter by "title:Get Out", i got nothing. i understood
"Get Out" is not exist in 8983.
[image: image.png]
[image: image.png]
but question is coming, when i run "{!join from=director_id
fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join:
multiple shards not yet supported movies"
no matter what filter value is.
i found following code:
[image: image.png]
when i run joining from movies on 8983, slice length is 2 as movies have 2
shards. "fromReplica " was assigned in second cycle, because zkController
name is 8983 and replica name is 8984 in first cycle.
but when run on 8984, "fromReplica" was assigned in first cycle, because
zkController name isand replica name both are 8984 in first cycle, so throw
"SolrCloud join: multiple shards not yet supported" in second cycle.
Thanks for your patience, it's too long. i'm confused about why use this
way to judge "multiple shards", because the result is also wrong running on
8983 even if didnt throw exception. why dont use slice length>1 to judge
"multiple shards" ? or maybe have other better way?
Please advise.
Thanks in advance!
Re: question about solrCloud joining
Posted by Mikhail Khludnev <mk...@apache.org>.
I'm not sure, but it might be an issue. It make sense to add negative test
and assert the exception at
https://github.com/apache/lucene-solr/blob/master/solr/core/src/test/org/apache/solr/cloud/DistribJoinFromCollectionTest.java
On Wed, Aug 21, 2019 at 10:37 AM Lisheng Wang <wa...@gmail.com>
wrote:
> Hi Mikhail,
>
> okay.
>
> below is 2 requests:
>
> both are select from "movieDirectors" collection join "movies" collection
> which has 2 shards.
>
>
> http://localhost:8983/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*
>
> http://localhost:8984/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*
>
> first request can get result without Exception, response is following
>
> { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3, "params":{
> "
> q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> to=id}title:\"Dunkirk\"", "_":"1566261450613"}}, "response":{"numFound":1,"
> start":0,"docs":[ { "id":"1", "name":"Christopher Nolan", "has_oscar":true,
> "_version_":1642343436642156544}] }}
>
> second request will get Exception
> { "responseHeader":{ "zkConnected":true, "status":400, "QTime":29,
> "params":{
> "q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> to=id}title:\"Dunkirk\"", "_":"1566261620152"}}, "error":{ "metadata":[
> "error-class","org.apache.solr.common.SolrException", "root-error-class",
> "org.apache.solr.common.SolrException"], "msg":"SolrCloud join: multiple
> shards not yet supported movies", "code":400}}
>
> i don't know why get 2 different result when you request from different
> node, i think both should get Exception with "SolrCloud join: multiple
> shards not yet supported movies".
>
> Best,
> Lisheng
>
>
> Mikhail Khludnev <mk...@apache.org> 于2019年8月21日周三 下午3:19写道:
>
> > Ok. Still hard to follow. Can you clarify which collection you run these
> > queries on?
> > Collection name (url segment before /select) is more significant than any
> > port (jvm) identity.
> >
> > On Wed, Aug 21, 2019 at 5:14 AM Lisheng Wang <wa...@gmail.com>
> > wrote:
> >
> > > Hi Mikhail
> > >
> > > Thanks for your response, but question is not related to "title:Get
> > Out",
> > > maybe i did not describe clearly.
> > >
> > > I knew solrCloud joining is not working in index which is splited to
> > > multiple shards.
> > >
> > > but why i run "*{!join from=director_id fromIndex=movies
> > > to=id}title:"Dunkirk"*" on 8984 (fromIndex=movies, movies has 2 shards)
> > i
> > > got exception "SolrCloud join: multiple shards not yet supported
> movies"
> > >
> > > but when run on 8983, i got result but it is incorrect without above
> > > exception. i think should get same exception no matter run joining on
> > 8983
> > > or 8984.
> > >
> > > Not sure my explanation is clear?
> > >
> > > Please kindly let me know if you have any question.
> > >
> > > Thanks!
> > >
> > > Lisheng
> > >
> > >
> > >
> > > Mikhail Khludnev <mk...@apache.org> 于2019年8月21日周三 上午4:41写道:
> > >
> > > > Hello, Lisheng.
> > > > I barely follow, but couldn't the space symbol in "title:Get Out"
> > > > cause the problem
> > > > ?
> > > > Check debugQuery and nested query in local param.
> > > >
> > > >
> > > > On Tue, Aug 20, 2019 at 6:35 PM Lisheng Wang <
> wanglisheng81@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Erick
> > > > >
> > > > > Thanks for your quick response and remaining me about attachment
> > issue.
> > > > >
> > > > > Yes, i run on 2 different jvms that not related to if they are on
> > same
> > > > > machine or not.
> > > > >
> > > > > let me describe my scenario, i have two collection:
> > > > >
> > > > > i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and
> > > 8984.
> > > > >
> > > > > 1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is
> on
> > > > 8983
> > > > > 2. movies: 2 shard, 1 replica/shard shard1 is on 8983, shard2 is
> > on
> > > > > 8984.
> > > > >
> > > > > collection movieDirectors has 2 docs:
> > > > > {
> > > > > "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > > 1642343781358370816
> > > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > > 1642343828930166784
> > > > > }
> > > > > collection movies has 2 docs too:
> > > > > { "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > > 1642343781358370816
> > > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > > 1642343828930166784
> > > > > }
> > > > > everything is ok when i run query with "{!join from=id
> > > > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both
> 8983
> > > and
> > > > > 8984, i can got expected result:
> > > > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":79,
> > > > "params":{
> > > > > "q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
> > > > > to=director_id}has_oscar:true", "_":"1566313944099"}},
> > > > > "response":{"numFound
> > > > > ":2,"start":0,"maxScore":1.0,"docs":[ { "id":"1",
> "title":"Dunkirk",
> > "
> > > > > director_id":"1", "_version_":1642343781358370816}, { "id":"2",
> > > > > "title":"Get
> > > > > Out", "director_id":"2", "_version_":1642343828930166784}] }}
> > > > > but when i run "{!join from=director_id fromIndex=movies
> > > > > to=id}title:"Dunkirk"" on 8983 got 1 doc,
> > > > > if i filter by "title:Get Out", i got nothing. i understood "Get
> > Out"
> > > > is
> > > > > not exist in 8983.
> > > > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3,
> > > > "params":{
> > > > > "
> > > > > q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> > > > > to=id}title:\"Dunkirk\"", "_":"1566261450613"}},
> > > > "response":{"numFound":1,"
> > > > > start":0,"docs":[ { "id":"1", "name":"Christopher Nolan",
> > > > "has_oscar":true,
> > > > > "_version_":1642343436642156544}] }}
> > > > >
> > > > > but question is coming, when i run "{!join from=director_id
> > > > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud
> > join:
> > > > > multiple shards not yet supported movies"
> > > > > no matter what filter value is.
> > > > >
> > > > > i found following code:
> > > > >
> > > > > private static String findLocalReplicaForFromIndex(ZkController
> > > > > zkController, String fromIndex) {
> > > > > String fromReplica = null;
> > > > >
> > > > > String nodeName = zkController.getNodeName();
> > > > > for (Slice slice :
> > > > >
> > > > >
> > > >
> > >
> >
> zkController.getClusterState().getCollection(fromIndex).getActiveSlicesArr())
> > > > > {
> > > > > if (fromReplica != null)
> > > > > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > > > "SolrCloud join: multiple shards not yet supported " +
> > > > > fromIndex);
> > > > > for (Replica replica : slice.getReplicas()) {
> > > > > if (replica.getNodeName().equals(nodeName)) {
> > > > > fromReplica = replica.getStr(ZkStateReader.CORE_NAME_PROP);
> > > > > // found local replica, but is it Active?
> > > > > if (replica.getState() != Replica.State.ACTIVE)
> > > > > throw new
> > SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > > > "SolrCloud join: "+fromIndex+" has a local replica
> > > > > ("+fromReplica+
> > > > > ") on "+nodeName+", but it is
> > "+replica.getState());
> > > > >
> > > > > break;
> > > > > }
> > > > > }
> > > > > }
> > > > >
> > > > > if (fromReplica == null)
> > > > > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > > > "SolrCloud join: No active replicas for "+fromIndex+
> > > > > " found in node " + nodeName);
> > > > >
> > > > > return fromReplica;
> > > > > }
> > > > >
> > > > >
> > > > > when i run joining from movies on 8983, slice length is 2 as movies
> > > have
> > > > 2
> > > > > shards. "fromReplica " was assigned in second cycle, because
> > > > zkController
> > > > > name is 8983 and replica name is 8984 in first cycle.
> > > > >
> > > > > but when run on 8984, "fromReplica" was assigned in first cycle,
> > > because
> > > > > zkController name isand replica name both are 8984 in first cycle,
> so
> > > > throw
> > > > > "SolrCloud join: multiple shards not yet supported" in second
> cycle.
> > > > >
> > > > > Thanks for your patience, it's too long. i'm confused about why use
> > > this
> > > > > way to judge "multiple shards", because the result is also wrong
> > > running
> > > > on
> > > > > 8983 even if didnt throw exception. why dont use slice length>1 to
> > > judge
> > > > > "multiple shards" ? or maybe have other better way?
> > > > >
> > > > > Please advise.
> > > > >
> > > > > Thanks in advance!
> > > > >
> > > > > Erick Erickson <er...@gmail.com> 于2019年8月20日周二 下午7:39写道:
> > > > >
> > > > > > None of your images came through, the mail server aggressively
> > strips
> > > > > > attachments. You’ll have to put them somewhere and provide a
> link.
> > > > > >
> > > > > > Given that, I’m guessing without much data so this may be totally
> > > > > > misguided. You mention ports 8984 and 8984. Assuming those are
> two
> > > > > > different Solr JVMs, the fact that they’re running on the same
> > > machine
> > > > is
> > > > > > irrelevant; As far as SolrCloud is concerned, they are two
> separate
> > > > > > machines. Your directors collection must be completely resident
> on
> > > both
> > > > > > Solr instances for cross-collection join to work.
> > > > > >
> > > > > > Best,
> > > > > > Erick
> > > > > >
> > > > > > > On Aug 19, 2019, at 9:39 PM, 王立生 <wa...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I have a question about solrCloud joining. i knew solrCloud
> > joining
> > > > can
> > > > > > do join only when index is not splited to shards, but when i
> test
> > > it,
> > > > i
> > > > > > found a problem which make me confused.
> > > > > > >
> > > > > > > i tested it on version 8.2
> > > > > > >
> > > > > > > assuming i have 2 collections like sample about "joining" on
> solr
> > > > > > offcial website,
> > > > > > >
> > > > > > > one collection called "movies", another called
> "movieDirectors".
> > > > > > >
> > > > > > > movies's fields: id, title, director_id
> > > > > > > movieDirectors's fields: id, name, has_oscar
> > > > > > >
> > > > > > > the information of shards and replicas as below, i started two
> > > nodes
> > > > on
> > > > > > my laptop:
> > > > > > >
> > > > > > > moviesDirectors have 2 docs:
> > > > > > >
> > > > > > > movies also have 2 docs:
> > > > > > >
> > > > > > > everything is ok when i run query with "{!join from=id
> > > > > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both
> > 8983
> > > > and
> > > > > > 8984, i can got expacted result:
> > > > > > >
> > > > > > > but when i run "{!join from=director_id fromIndex=movies
> > > > > > to=id}title:"Dunkirk"" on 8983
> > > > > > > got 1 doc and if i filter by "title:Get Out", i got nothing. i
> > > > > > understood "Get Out" is not exist in 8983.
> > > > > > >
> > > > > > >
> > > > > > > but question is coming, when i run "{!join from=director_id
> > > > > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud
> > > join:
> > > > > > multiple shards not yet supported movies"
> > > > > > > no matter what filter value is.
> > > > > > >
> > > > > > > i found following code:
> > > > > > >
> > > > > > >
> > > > > > > when i run joining from movies on 8983, slice length is 2 as
> > movies
> > > > > have
> > > > > > 2 shards. "fromReplica " was assigned in second cycle, because
> > > > > > zkController name is 8983 and replica name is 8984 in first
> cycle.
> > > > > > >
> > > > > > > but when run on 8984, "fromReplica" was assigned in first
> cycle,
> > > > > because
> > > > > > zkController name isand replica name both are 8984 in first
> cycle,
> > so
> > > > > throw
> > > > > > "SolrCloud join: multiple shards not yet supported" in second
> > cycle.
> > > > > > >
> > > > > > > Thanks for your patience, it's too long. i'm confused about why
> > use
> > > > > this
> > > > > > way to judge "multiple shards", because the result is also wrong
> > > > running
> > > > > on
> > > > > > 8983 even if didnt throw exception. why dont use slice length>1
> to
> > > > judge
> > > > > > "multiple shards" ? or maybe have other better way?
> > > > > > >
> > > > > > > Please advise.
> > > > > > >
> > > > > > > Thanks in advance!
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Sincerely yours
> > > > Mikhail Khludnev
> > > >
> > >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>
--
Sincerely yours
Mikhail Khludnev
Re: question about solrCloud joining
Posted by Mikhail Khludnev <mk...@apache.org>.
Raised https://issues.apache.org/jira/browse/SOLR-13716
On Wed, Aug 21, 2019 at 10:37 AM Lisheng Wang <wa...@gmail.com>
wrote:
> Hi Mikhail,
>
> okay.
>
> below is 2 requests:
>
> both are select from "movieDirectors" collection join "movies" collection
> which has 2 shards.
>
>
> http://localhost:8983/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*
>
> http://localhost:8984/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*
>
> first request can get result without Exception, response is following
>
> { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3, "params":{
> "
> q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> to=id}title:\"Dunkirk\"", "_":"1566261450613"}}, "response":{"numFound":1,"
> start":0,"docs":[ { "id":"1", "name":"Christopher Nolan", "has_oscar":true,
> "_version_":1642343436642156544}] }}
>
> second request will get Exception
> { "responseHeader":{ "zkConnected":true, "status":400, "QTime":29,
> "params":{
> "q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> to=id}title:\"Dunkirk\"", "_":"1566261620152"}}, "error":{ "metadata":[
> "error-class","org.apache.solr.common.SolrException", "root-error-class",
> "org.apache.solr.common.SolrException"], "msg":"SolrCloud join: multiple
> shards not yet supported movies", "code":400}}
>
> i don't know why get 2 different result when you request from different
> node, i think both should get Exception with "SolrCloud join: multiple
> shards not yet supported movies".
>
> Best,
> Lisheng
>
>
> Mikhail Khludnev <mk...@apache.org> 于2019年8月21日周三 下午3:19写道:
>
> > Ok. Still hard to follow. Can you clarify which collection you run these
> > queries on?
> > Collection name (url segment before /select) is more significant than any
> > port (jvm) identity.
> >
> > On Wed, Aug 21, 2019 at 5:14 AM Lisheng Wang <wa...@gmail.com>
> > wrote:
> >
> > > Hi Mikhail
> > >
> > > Thanks for your response, but question is not related to "title:Get
> > Out",
> > > maybe i did not describe clearly.
> > >
> > > I knew solrCloud joining is not working in index which is splited to
> > > multiple shards.
> > >
> > > but why i run "*{!join from=director_id fromIndex=movies
> > > to=id}title:"Dunkirk"*" on 8984 (fromIndex=movies, movies has 2 shards)
> > i
> > > got exception "SolrCloud join: multiple shards not yet supported
> movies"
> > >
> > > but when run on 8983, i got result but it is incorrect without above
> > > exception. i think should get same exception no matter run joining on
> > 8983
> > > or 8984.
> > >
> > > Not sure my explanation is clear?
> > >
> > > Please kindly let me know if you have any question.
> > >
> > > Thanks!
> > >
> > > Lisheng
> > >
> > >
> > >
> > > Mikhail Khludnev <mk...@apache.org> 于2019年8月21日周三 上午4:41写道:
> > >
> > > > Hello, Lisheng.
> > > > I barely follow, but couldn't the space symbol in "title:Get Out"
> > > > cause the problem
> > > > ?
> > > > Check debugQuery and nested query in local param.
> > > >
> > > >
> > > > On Tue, Aug 20, 2019 at 6:35 PM Lisheng Wang <
> wanglisheng81@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Erick
> > > > >
> > > > > Thanks for your quick response and remaining me about attachment
> > issue.
> > > > >
> > > > > Yes, i run on 2 different jvms that not related to if they are on
> > same
> > > > > machine or not.
> > > > >
> > > > > let me describe my scenario, i have two collection:
> > > > >
> > > > > i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and
> > > 8984.
> > > > >
> > > > > 1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is
> on
> > > > 8983
> > > > > 2. movies: 2 shard, 1 replica/shard shard1 is on 8983, shard2 is
> > on
> > > > > 8984.
> > > > >
> > > > > collection movieDirectors has 2 docs:
> > > > > {
> > > > > "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > > 1642343781358370816
> > > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > > 1642343828930166784
> > > > > }
> > > > > collection movies has 2 docs too:
> > > > > { "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > > 1642343781358370816
> > > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > > 1642343828930166784
> > > > > }
> > > > > everything is ok when i run query with "{!join from=id
> > > > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both
> 8983
> > > and
> > > > > 8984, i can got expected result:
> > > > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":79,
> > > > "params":{
> > > > > "q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
> > > > > to=director_id}has_oscar:true", "_":"1566313944099"}},
> > > > > "response":{"numFound
> > > > > ":2,"start":0,"maxScore":1.0,"docs":[ { "id":"1",
> "title":"Dunkirk",
> > "
> > > > > director_id":"1", "_version_":1642343781358370816}, { "id":"2",
> > > > > "title":"Get
> > > > > Out", "director_id":"2", "_version_":1642343828930166784}] }}
> > > > > but when i run "{!join from=director_id fromIndex=movies
> > > > > to=id}title:"Dunkirk"" on 8983 got 1 doc,
> > > > > if i filter by "title:Get Out", i got nothing. i understood "Get
> > Out"
> > > > is
> > > > > not exist in 8983.
> > > > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3,
> > > > "params":{
> > > > > "
> > > > > q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> > > > > to=id}title:\"Dunkirk\"", "_":"1566261450613"}},
> > > > "response":{"numFound":1,"
> > > > > start":0,"docs":[ { "id":"1", "name":"Christopher Nolan",
> > > > "has_oscar":true,
> > > > > "_version_":1642343436642156544}] }}
> > > > >
> > > > > but question is coming, when i run "{!join from=director_id
> > > > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud
> > join:
> > > > > multiple shards not yet supported movies"
> > > > > no matter what filter value is.
> > > > >
> > > > > i found following code:
> > > > >
> > > > > private static String findLocalReplicaForFromIndex(ZkController
> > > > > zkController, String fromIndex) {
> > > > > String fromReplica = null;
> > > > >
> > > > > String nodeName = zkController.getNodeName();
> > > > > for (Slice slice :
> > > > >
> > > > >
> > > >
> > >
> >
> zkController.getClusterState().getCollection(fromIndex).getActiveSlicesArr())
> > > > > {
> > > > > if (fromReplica != null)
> > > > > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > > > "SolrCloud join: multiple shards not yet supported " +
> > > > > fromIndex);
> > > > > for (Replica replica : slice.getReplicas()) {
> > > > > if (replica.getNodeName().equals(nodeName)) {
> > > > > fromReplica = replica.getStr(ZkStateReader.CORE_NAME_PROP);
> > > > > // found local replica, but is it Active?
> > > > > if (replica.getState() != Replica.State.ACTIVE)
> > > > > throw new
> > SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > > > "SolrCloud join: "+fromIndex+" has a local replica
> > > > > ("+fromReplica+
> > > > > ") on "+nodeName+", but it is
> > "+replica.getState());
> > > > >
> > > > > break;
> > > > > }
> > > > > }
> > > > > }
> > > > >
> > > > > if (fromReplica == null)
> > > > > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > > > "SolrCloud join: No active replicas for "+fromIndex+
> > > > > " found in node " + nodeName);
> > > > >
> > > > > return fromReplica;
> > > > > }
> > > > >
> > > > >
> > > > > when i run joining from movies on 8983, slice length is 2 as movies
> > > have
> > > > 2
> > > > > shards. "fromReplica " was assigned in second cycle, because
> > > > zkController
> > > > > name is 8983 and replica name is 8984 in first cycle.
> > > > >
> > > > > but when run on 8984, "fromReplica" was assigned in first cycle,
> > > because
> > > > > zkController name isand replica name both are 8984 in first cycle,
> so
> > > > throw
> > > > > "SolrCloud join: multiple shards not yet supported" in second
> cycle.
> > > > >
> > > > > Thanks for your patience, it's too long. i'm confused about why use
> > > this
> > > > > way to judge "multiple shards", because the result is also wrong
> > > running
> > > > on
> > > > > 8983 even if didnt throw exception. why dont use slice length>1 to
> > > judge
> > > > > "multiple shards" ? or maybe have other better way?
> > > > >
> > > > > Please advise.
> > > > >
> > > > > Thanks in advance!
> > > > >
> > > > > Erick Erickson <er...@gmail.com> 于2019年8月20日周二 下午7:39写道:
> > > > >
> > > > > > None of your images came through, the mail server aggressively
> > strips
> > > > > > attachments. You’ll have to put them somewhere and provide a
> link.
> > > > > >
> > > > > > Given that, I’m guessing without much data so this may be totally
> > > > > > misguided. You mention ports 8984 and 8984. Assuming those are
> two
> > > > > > different Solr JVMs, the fact that they’re running on the same
> > > machine
> > > > is
> > > > > > irrelevant; As far as SolrCloud is concerned, they are two
> separate
> > > > > > machines. Your directors collection must be completely resident
> on
> > > both
> > > > > > Solr instances for cross-collection join to work.
> > > > > >
> > > > > > Best,
> > > > > > Erick
> > > > > >
> > > > > > > On Aug 19, 2019, at 9:39 PM, 王立生 <wa...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I have a question about solrCloud joining. i knew solrCloud
> > joining
> > > > can
> > > > > > do join only when index is not splited to shards, but when i
> test
> > > it,
> > > > i
> > > > > > found a problem which make me confused.
> > > > > > >
> > > > > > > i tested it on version 8.2
> > > > > > >
> > > > > > > assuming i have 2 collections like sample about "joining" on
> solr
> > > > > > offcial website,
> > > > > > >
> > > > > > > one collection called "movies", another called
> "movieDirectors".
> > > > > > >
> > > > > > > movies's fields: id, title, director_id
> > > > > > > movieDirectors's fields: id, name, has_oscar
> > > > > > >
> > > > > > > the information of shards and replicas as below, i started two
> > > nodes
> > > > on
> > > > > > my laptop:
> > > > > > >
> > > > > > > moviesDirectors have 2 docs:
> > > > > > >
> > > > > > > movies also have 2 docs:
> > > > > > >
> > > > > > > everything is ok when i run query with "{!join from=id
> > > > > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both
> > 8983
> > > > and
> > > > > > 8984, i can got expacted result:
> > > > > > >
> > > > > > > but when i run "{!join from=director_id fromIndex=movies
> > > > > > to=id}title:"Dunkirk"" on 8983
> > > > > > > got 1 doc and if i filter by "title:Get Out", i got nothing. i
> > > > > > understood "Get Out" is not exist in 8983.
> > > > > > >
> > > > > > >
> > > > > > > but question is coming, when i run "{!join from=director_id
> > > > > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud
> > > join:
> > > > > > multiple shards not yet supported movies"
> > > > > > > no matter what filter value is.
> > > > > > >
> > > > > > > i found following code:
> > > > > > >
> > > > > > >
> > > > > > > when i run joining from movies on 8983, slice length is 2 as
> > movies
> > > > > have
> > > > > > 2 shards. "fromReplica " was assigned in second cycle, because
> > > > > > zkController name is 8983 and replica name is 8984 in first
> cycle.
> > > > > > >
> > > > > > > but when run on 8984, "fromReplica" was assigned in first
> cycle,
> > > > > because
> > > > > > zkController name isand replica name both are 8984 in first
> cycle,
> > so
> > > > > throw
> > > > > > "SolrCloud join: multiple shards not yet supported" in second
> > cycle.
> > > > > > >
> > > > > > > Thanks for your patience, it's too long. i'm confused about why
> > use
> > > > > this
> > > > > > way to judge "multiple shards", because the result is also wrong
> > > > running
> > > > > on
> > > > > > 8983 even if didnt throw exception. why dont use slice length>1
> to
> > > > judge
> > > > > > "multiple shards" ? or maybe have other better way?
> > > > > > >
> > > > > > > Please advise.
> > > > > > >
> > > > > > > Thanks in advance!
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Sincerely yours
> > > > Mikhail Khludnev
> > > >
> > >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>
--
Sincerely yours
Mikhail Khludnev
Re: question about solrCloud joining
Posted by Lisheng Wang <wa...@gmail.com>.
Hi Mikhail,
okay.
below is 2 requests:
both are select from "movieDirectors" collection join "movies" collection
which has 2 shards.
http://localhost:8983/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*
http://localhost:8984/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*
first request can get result without Exception, response is following
{ "responseHeader":{ "zkConnected":true, "status":0, "QTime":3, "params":{ "
q":"*:*", "fq":"{!join from=director_id fromIndex=movies
to=id}title:\"Dunkirk\"", "_":"1566261450613"}}, "response":{"numFound":1,"
start":0,"docs":[ { "id":"1", "name":"Christopher Nolan", "has_oscar":true,
"_version_":1642343436642156544}] }}
second request will get Exception
{ "responseHeader":{ "zkConnected":true, "status":400, "QTime":29, "params":{
"q":"*:*", "fq":"{!join from=director_id fromIndex=movies
to=id}title:\"Dunkirk\"", "_":"1566261620152"}}, "error":{ "metadata":[
"error-class","org.apache.solr.common.SolrException", "root-error-class",
"org.apache.solr.common.SolrException"], "msg":"SolrCloud join: multiple
shards not yet supported movies", "code":400}}
i don't know why get 2 different result when you request from different
node, i think both should get Exception with "SolrCloud join: multiple
shards not yet supported movies".
Best,
Lisheng
Mikhail Khludnev <mk...@apache.org> 于2019年8月21日周三 下午3:19写道:
> Ok. Still hard to follow. Can you clarify which collection you run these
> queries on?
> Collection name (url segment before /select) is more significant than any
> port (jvm) identity.
>
> On Wed, Aug 21, 2019 at 5:14 AM Lisheng Wang <wa...@gmail.com>
> wrote:
>
> > Hi Mikhail
> >
> > Thanks for your response, but question is not related to "title:Get
> Out",
> > maybe i did not describe clearly.
> >
> > I knew solrCloud joining is not working in index which is splited to
> > multiple shards.
> >
> > but why i run "*{!join from=director_id fromIndex=movies
> > to=id}title:"Dunkirk"*" on 8984 (fromIndex=movies, movies has 2 shards)
> i
> > got exception "SolrCloud join: multiple shards not yet supported movies"
> >
> > but when run on 8983, i got result but it is incorrect without above
> > exception. i think should get same exception no matter run joining on
> 8983
> > or 8984.
> >
> > Not sure my explanation is clear?
> >
> > Please kindly let me know if you have any question.
> >
> > Thanks!
> >
> > Lisheng
> >
> >
> >
> > Mikhail Khludnev <mk...@apache.org> 于2019年8月21日周三 上午4:41写道:
> >
> > > Hello, Lisheng.
> > > I barely follow, but couldn't the space symbol in "title:Get Out"
> > > cause the problem
> > > ?
> > > Check debugQuery and nested query in local param.
> > >
> > >
> > > On Tue, Aug 20, 2019 at 6:35 PM Lisheng Wang <wa...@gmail.com>
> > > wrote:
> > >
> > > > Hi Erick
> > > >
> > > > Thanks for your quick response and remaining me about attachment
> issue.
> > > >
> > > > Yes, i run on 2 different jvms that not related to if they are on
> same
> > > > machine or not.
> > > >
> > > > let me describe my scenario, i have two collection:
> > > >
> > > > i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and
> > 8984.
> > > >
> > > > 1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is on
> > > 8983
> > > > 2. movies: 2 shard, 1 replica/shard shard1 is on 8983, shard2 is
> on
> > > > 8984.
> > > >
> > > > collection movieDirectors has 2 docs:
> > > > {
> > > > "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > 1642343781358370816
> > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > 1642343828930166784
> > > > }
> > > > collection movies has 2 docs too:
> > > > { "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > 1642343781358370816
> > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > 1642343828930166784
> > > > }
> > > > everything is ok when i run query with "{!join from=id
> > > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983
> > and
> > > > 8984, i can got expected result:
> > > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":79,
> > > "params":{
> > > > "q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
> > > > to=director_id}has_oscar:true", "_":"1566313944099"}},
> > > > "response":{"numFound
> > > > ":2,"start":0,"maxScore":1.0,"docs":[ { "id":"1", "title":"Dunkirk",
> "
> > > > director_id":"1", "_version_":1642343781358370816}, { "id":"2",
> > > > "title":"Get
> > > > Out", "director_id":"2", "_version_":1642343828930166784}] }}
> > > > but when i run "{!join from=director_id fromIndex=movies
> > > > to=id}title:"Dunkirk"" on 8983 got 1 doc,
> > > > if i filter by "title:Get Out", i got nothing. i understood "Get
> Out"
> > > is
> > > > not exist in 8983.
> > > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3,
> > > "params":{
> > > > "
> > > > q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> > > > to=id}title:\"Dunkirk\"", "_":"1566261450613"}},
> > > "response":{"numFound":1,"
> > > > start":0,"docs":[ { "id":"1", "name":"Christopher Nolan",
> > > "has_oscar":true,
> > > > "_version_":1642343436642156544}] }}
> > > >
> > > > but question is coming, when i run "{!join from=director_id
> > > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud
> join:
> > > > multiple shards not yet supported movies"
> > > > no matter what filter value is.
> > > >
> > > > i found following code:
> > > >
> > > > private static String findLocalReplicaForFromIndex(ZkController
> > > > zkController, String fromIndex) {
> > > > String fromReplica = null;
> > > >
> > > > String nodeName = zkController.getNodeName();
> > > > for (Slice slice :
> > > >
> > > >
> > >
> >
> zkController.getClusterState().getCollection(fromIndex).getActiveSlicesArr())
> > > > {
> > > > if (fromReplica != null)
> > > > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > > "SolrCloud join: multiple shards not yet supported " +
> > > > fromIndex);
> > > > for (Replica replica : slice.getReplicas()) {
> > > > if (replica.getNodeName().equals(nodeName)) {
> > > > fromReplica = replica.getStr(ZkStateReader.CORE_NAME_PROP);
> > > > // found local replica, but is it Active?
> > > > if (replica.getState() != Replica.State.ACTIVE)
> > > > throw new
> SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > > "SolrCloud join: "+fromIndex+" has a local replica
> > > > ("+fromReplica+
> > > > ") on "+nodeName+", but it is
> "+replica.getState());
> > > >
> > > > break;
> > > > }
> > > > }
> > > > }
> > > >
> > > > if (fromReplica == null)
> > > > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > > "SolrCloud join: No active replicas for "+fromIndex+
> > > > " found in node " + nodeName);
> > > >
> > > > return fromReplica;
> > > > }
> > > >
> > > >
> > > > when i run joining from movies on 8983, slice length is 2 as movies
> > have
> > > 2
> > > > shards. "fromReplica " was assigned in second cycle, because
> > > zkController
> > > > name is 8983 and replica name is 8984 in first cycle.
> > > >
> > > > but when run on 8984, "fromReplica" was assigned in first cycle,
> > because
> > > > zkController name isand replica name both are 8984 in first cycle, so
> > > throw
> > > > "SolrCloud join: multiple shards not yet supported" in second cycle.
> > > >
> > > > Thanks for your patience, it's too long. i'm confused about why use
> > this
> > > > way to judge "multiple shards", because the result is also wrong
> > running
> > > on
> > > > 8983 even if didnt throw exception. why dont use slice length>1 to
> > judge
> > > > "multiple shards" ? or maybe have other better way?
> > > >
> > > > Please advise.
> > > >
> > > > Thanks in advance!
> > > >
> > > > Erick Erickson <er...@gmail.com> 于2019年8月20日周二 下午7:39写道:
> > > >
> > > > > None of your images came through, the mail server aggressively
> strips
> > > > > attachments. You’ll have to put them somewhere and provide a link.
> > > > >
> > > > > Given that, I’m guessing without much data so this may be totally
> > > > > misguided. You mention ports 8984 and 8984. Assuming those are two
> > > > > different Solr JVMs, the fact that they’re running on the same
> > machine
> > > is
> > > > > irrelevant; As far as SolrCloud is concerned, they are two separate
> > > > > machines. Your directors collection must be completely resident on
> > both
> > > > > Solr instances for cross-collection join to work.
> > > > >
> > > > > Best,
> > > > > Erick
> > > > >
> > > > > > On Aug 19, 2019, at 9:39 PM, 王立生 <wa...@gmail.com>
> wrote:
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I have a question about solrCloud joining. i knew solrCloud
> joining
> > > can
> > > > > do join only when index is not splited to shards, but when i test
> > it,
> > > i
> > > > > found a problem which make me confused.
> > > > > >
> > > > > > i tested it on version 8.2
> > > > > >
> > > > > > assuming i have 2 collections like sample about "joining" on solr
> > > > > offcial website,
> > > > > >
> > > > > > one collection called "movies", another called "movieDirectors".
> > > > > >
> > > > > > movies's fields: id, title, director_id
> > > > > > movieDirectors's fields: id, name, has_oscar
> > > > > >
> > > > > > the information of shards and replicas as below, i started two
> > nodes
> > > on
> > > > > my laptop:
> > > > > >
> > > > > > moviesDirectors have 2 docs:
> > > > > >
> > > > > > movies also have 2 docs:
> > > > > >
> > > > > > everything is ok when i run query with "{!join from=id
> > > > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both
> 8983
> > > and
> > > > > 8984, i can got expacted result:
> > > > > >
> > > > > > but when i run "{!join from=director_id fromIndex=movies
> > > > > to=id}title:"Dunkirk"" on 8983
> > > > > > got 1 doc and if i filter by "title:Get Out", i got nothing. i
> > > > > understood "Get Out" is not exist in 8983.
> > > > > >
> > > > > >
> > > > > > but question is coming, when i run "{!join from=director_id
> > > > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud
> > join:
> > > > > multiple shards not yet supported movies"
> > > > > > no matter what filter value is.
> > > > > >
> > > > > > i found following code:
> > > > > >
> > > > > >
> > > > > > when i run joining from movies on 8983, slice length is 2 as
> movies
> > > > have
> > > > > 2 shards. "fromReplica " was assigned in second cycle, because
> > > > > zkController name is 8983 and replica name is 8984 in first cycle.
> > > > > >
> > > > > > but when run on 8984, "fromReplica" was assigned in first cycle,
> > > > because
> > > > > zkController name isand replica name both are 8984 in first cycle,
> so
> > > > throw
> > > > > "SolrCloud join: multiple shards not yet supported" in second
> cycle.
> > > > > >
> > > > > > Thanks for your patience, it's too long. i'm confused about why
> use
> > > > this
> > > > > way to judge "multiple shards", because the result is also wrong
> > > running
> > > > on
> > > > > 8983 even if didnt throw exception. why dont use slice length>1 to
> > > judge
> > > > > "multiple shards" ? or maybe have other better way?
> > > > > >
> > > > > > Please advise.
> > > > > >
> > > > > > Thanks in advance!
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Sincerely yours
> > > Mikhail Khludnev
> > >
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>
Re: question about solrCloud joining
Posted by Mikhail Khludnev <mk...@apache.org>.
Ok. Still hard to follow. Can you clarify which collection you run these
queries on?
Collection name (url segment before /select) is more significant than any
port (jvm) identity.
On Wed, Aug 21, 2019 at 5:14 AM Lisheng Wang <wa...@gmail.com>
wrote:
> Hi Mikhail
>
> Thanks for your response, but question is not related to "title:Get Out",
> maybe i did not describe clearly.
>
> I knew solrCloud joining is not working in index which is splited to
> multiple shards.
>
> but why i run "*{!join from=director_id fromIndex=movies
> to=id}title:"Dunkirk"*" on 8984 (fromIndex=movies, movies has 2 shards) i
> got exception "SolrCloud join: multiple shards not yet supported movies"
>
> but when run on 8983, i got result but it is incorrect without above
> exception. i think should get same exception no matter run joining on 8983
> or 8984.
>
> Not sure my explanation is clear?
>
> Please kindly let me know if you have any question.
>
> Thanks!
>
> Lisheng
>
>
>
> Mikhail Khludnev <mk...@apache.org> 于2019年8月21日周三 上午4:41写道:
>
> > Hello, Lisheng.
> > I barely follow, but couldn't the space symbol in "title:Get Out"
> > cause the problem
> > ?
> > Check debugQuery and nested query in local param.
> >
> >
> > On Tue, Aug 20, 2019 at 6:35 PM Lisheng Wang <wa...@gmail.com>
> > wrote:
> >
> > > Hi Erick
> > >
> > > Thanks for your quick response and remaining me about attachment issue.
> > >
> > > Yes, i run on 2 different jvms that not related to if they are on same
> > > machine or not.
> > >
> > > let me describe my scenario, i have two collection:
> > >
> > > i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and
> 8984.
> > >
> > > 1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is on
> > 8983
> > > 2. movies: 2 shard, 1 replica/shard shard1 is on 8983, shard2 is on
> > > 8984.
> > >
> > > collection movieDirectors has 2 docs:
> > > {
> > > "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > 1642343781358370816
> > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > 1642343828930166784
> > > }
> > > collection movies has 2 docs too:
> > > { "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > 1642343781358370816
> > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > 1642343828930166784
> > > }
> > > everything is ok when i run query with "{!join from=id
> > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983
> and
> > > 8984, i can got expected result:
> > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":79,
> > "params":{
> > > "q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
> > > to=director_id}has_oscar:true", "_":"1566313944099"}},
> > > "response":{"numFound
> > > ":2,"start":0,"maxScore":1.0,"docs":[ { "id":"1", "title":"Dunkirk", "
> > > director_id":"1", "_version_":1642343781358370816}, { "id":"2",
> > > "title":"Get
> > > Out", "director_id":"2", "_version_":1642343828930166784}] }}
> > > but when i run "{!join from=director_id fromIndex=movies
> > > to=id}title:"Dunkirk"" on 8983 got 1 doc,
> > > if i filter by "title:Get Out", i got nothing. i understood "Get Out"
> > is
> > > not exist in 8983.
> > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3,
> > "params":{
> > > "
> > > q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> > > to=id}title:\"Dunkirk\"", "_":"1566261450613"}},
> > "response":{"numFound":1,"
> > > start":0,"docs":[ { "id":"1", "name":"Christopher Nolan",
> > "has_oscar":true,
> > > "_version_":1642343436642156544}] }}
> > >
> > > but question is coming, when i run "{!join from=director_id
> > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join:
> > > multiple shards not yet supported movies"
> > > no matter what filter value is.
> > >
> > > i found following code:
> > >
> > > private static String findLocalReplicaForFromIndex(ZkController
> > > zkController, String fromIndex) {
> > > String fromReplica = null;
> > >
> > > String nodeName = zkController.getNodeName();
> > > for (Slice slice :
> > >
> > >
> >
> zkController.getClusterState().getCollection(fromIndex).getActiveSlicesArr())
> > > {
> > > if (fromReplica != null)
> > > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > "SolrCloud join: multiple shards not yet supported " +
> > > fromIndex);
> > > for (Replica replica : slice.getReplicas()) {
> > > if (replica.getNodeName().equals(nodeName)) {
> > > fromReplica = replica.getStr(ZkStateReader.CORE_NAME_PROP);
> > > // found local replica, but is it Active?
> > > if (replica.getState() != Replica.State.ACTIVE)
> > > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > "SolrCloud join: "+fromIndex+" has a local replica
> > > ("+fromReplica+
> > > ") on "+nodeName+", but it is "+replica.getState());
> > >
> > > break;
> > > }
> > > }
> > > }
> > >
> > > if (fromReplica == null)
> > > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > > "SolrCloud join: No active replicas for "+fromIndex+
> > > " found in node " + nodeName);
> > >
> > > return fromReplica;
> > > }
> > >
> > >
> > > when i run joining from movies on 8983, slice length is 2 as movies
> have
> > 2
> > > shards. "fromReplica " was assigned in second cycle, because
> > zkController
> > > name is 8983 and replica name is 8984 in first cycle.
> > >
> > > but when run on 8984, "fromReplica" was assigned in first cycle,
> because
> > > zkController name isand replica name both are 8984 in first cycle, so
> > throw
> > > "SolrCloud join: multiple shards not yet supported" in second cycle.
> > >
> > > Thanks for your patience, it's too long. i'm confused about why use
> this
> > > way to judge "multiple shards", because the result is also wrong
> running
> > on
> > > 8983 even if didnt throw exception. why dont use slice length>1 to
> judge
> > > "multiple shards" ? or maybe have other better way?
> > >
> > > Please advise.
> > >
> > > Thanks in advance!
> > >
> > > Erick Erickson <er...@gmail.com> 于2019年8月20日周二 下午7:39写道:
> > >
> > > > None of your images came through, the mail server aggressively strips
> > > > attachments. You’ll have to put them somewhere and provide a link.
> > > >
> > > > Given that, I’m guessing without much data so this may be totally
> > > > misguided. You mention ports 8984 and 8984. Assuming those are two
> > > > different Solr JVMs, the fact that they’re running on the same
> machine
> > is
> > > > irrelevant; As far as SolrCloud is concerned, they are two separate
> > > > machines. Your directors collection must be completely resident on
> both
> > > > Solr instances for cross-collection join to work.
> > > >
> > > > Best,
> > > > Erick
> > > >
> > > > > On Aug 19, 2019, at 9:39 PM, 王立生 <wa...@gmail.com> wrote:
> > > > >
> > > > > Hello,
> > > > >
> > > > > I have a question about solrCloud joining. i knew solrCloud joining
> > can
> > > > do join only when index is not splited to shards, but when i test
> it,
> > i
> > > > found a problem which make me confused.
> > > > >
> > > > > i tested it on version 8.2
> > > > >
> > > > > assuming i have 2 collections like sample about "joining" on solr
> > > > offcial website,
> > > > >
> > > > > one collection called "movies", another called "movieDirectors".
> > > > >
> > > > > movies's fields: id, title, director_id
> > > > > movieDirectors's fields: id, name, has_oscar
> > > > >
> > > > > the information of shards and replicas as below, i started two
> nodes
> > on
> > > > my laptop:
> > > > >
> > > > > moviesDirectors have 2 docs:
> > > > >
> > > > > movies also have 2 docs:
> > > > >
> > > > > everything is ok when i run query with "{!join from=id
> > > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983
> > and
> > > > 8984, i can got expacted result:
> > > > >
> > > > > but when i run "{!join from=director_id fromIndex=movies
> > > > to=id}title:"Dunkirk"" on 8983
> > > > > got 1 doc and if i filter by "title:Get Out", i got nothing. i
> > > > understood "Get Out" is not exist in 8983.
> > > > >
> > > > >
> > > > > but question is coming, when i run "{!join from=director_id
> > > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud
> join:
> > > > multiple shards not yet supported movies"
> > > > > no matter what filter value is.
> > > > >
> > > > > i found following code:
> > > > >
> > > > >
> > > > > when i run joining from movies on 8983, slice length is 2 as movies
> > > have
> > > > 2 shards. "fromReplica " was assigned in second cycle, because
> > > > zkController name is 8983 and replica name is 8984 in first cycle.
> > > > >
> > > > > but when run on 8984, "fromReplica" was assigned in first cycle,
> > > because
> > > > zkController name isand replica name both are 8984 in first cycle, so
> > > throw
> > > > "SolrCloud join: multiple shards not yet supported" in second cycle.
> > > > >
> > > > > Thanks for your patience, it's too long. i'm confused about why use
> > > this
> > > > way to judge "multiple shards", because the result is also wrong
> > running
> > > on
> > > > 8983 even if didnt throw exception. why dont use slice length>1 to
> > judge
> > > > "multiple shards" ? or maybe have other better way?
> > > > >
> > > > > Please advise.
> > > > >
> > > > > Thanks in advance!
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>
--
Sincerely yours
Mikhail Khludnev
Re: question about solrCloud joining
Posted by Lisheng Wang <wa...@gmail.com>.
Hi Mikhail
Thanks for your response, but question is not related to "title:Get Out",
maybe i did not describe clearly.
I knew solrCloud joining is not working in index which is splited to
multiple shards.
but why i run "*{!join from=director_id fromIndex=movies
to=id}title:"Dunkirk"*" on 8984 (fromIndex=movies, movies has 2 shards) i
got exception "SolrCloud join: multiple shards not yet supported movies"
but when run on 8983, i got result but it is incorrect without above
exception. i think should get same exception no matter run joining on 8983
or 8984.
Not sure my explanation is clear?
Please kindly let me know if you have any question.
Thanks!
Lisheng
Mikhail Khludnev <mk...@apache.org> 于2019年8月21日周三 上午4:41写道:
> Hello, Lisheng.
> I barely follow, but couldn't the space symbol in "title:Get Out"
> cause the problem
> ?
> Check debugQuery and nested query in local param.
>
>
> On Tue, Aug 20, 2019 at 6:35 PM Lisheng Wang <wa...@gmail.com>
> wrote:
>
> > Hi Erick
> >
> > Thanks for your quick response and remaining me about attachment issue.
> >
> > Yes, i run on 2 different jvms that not related to if they are on same
> > machine or not.
> >
> > let me describe my scenario, i have two collection:
> >
> > i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and 8984.
> >
> > 1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is on
> 8983
> > 2. movies: 2 shard, 1 replica/shard shard1 is on 8983, shard2 is on
> > 8984.
> >
> > collection movieDirectors has 2 docs:
> > {
> > "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > 1642343781358370816
> > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > 1642343828930166784
> > }
> > collection movies has 2 docs too:
> > { "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > 1642343781358370816
> > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > 1642343828930166784
> > }
> > everything is ok when i run query with "{!join from=id
> > fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983 and
> > 8984, i can got expected result:
> > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":79,
> "params":{
> > "q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
> > to=director_id}has_oscar:true", "_":"1566313944099"}},
> > "response":{"numFound
> > ":2,"start":0,"maxScore":1.0,"docs":[ { "id":"1", "title":"Dunkirk", "
> > director_id":"1", "_version_":1642343781358370816}, { "id":"2",
> > "title":"Get
> > Out", "director_id":"2", "_version_":1642343828930166784}] }}
> > but when i run "{!join from=director_id fromIndex=movies
> > to=id}title:"Dunkirk"" on 8983 got 1 doc,
> > if i filter by "title:Get Out", i got nothing. i understood "Get Out"
> is
> > not exist in 8983.
> > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3,
> "params":{
> > "
> > q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> > to=id}title:\"Dunkirk\"", "_":"1566261450613"}},
> "response":{"numFound":1,"
> > start":0,"docs":[ { "id":"1", "name":"Christopher Nolan",
> "has_oscar":true,
> > "_version_":1642343436642156544}] }}
> >
> > but question is coming, when i run "{!join from=director_id
> > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join:
> > multiple shards not yet supported movies"
> > no matter what filter value is.
> >
> > i found following code:
> >
> > private static String findLocalReplicaForFromIndex(ZkController
> > zkController, String fromIndex) {
> > String fromReplica = null;
> >
> > String nodeName = zkController.getNodeName();
> > for (Slice slice :
> >
> >
> zkController.getClusterState().getCollection(fromIndex).getActiveSlicesArr())
> > {
> > if (fromReplica != null)
> > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > "SolrCloud join: multiple shards not yet supported " +
> > fromIndex);
> > for (Replica replica : slice.getReplicas()) {
> > if (replica.getNodeName().equals(nodeName)) {
> > fromReplica = replica.getStr(ZkStateReader.CORE_NAME_PROP);
> > // found local replica, but is it Active?
> > if (replica.getState() != Replica.State.ACTIVE)
> > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > "SolrCloud join: "+fromIndex+" has a local replica
> > ("+fromReplica+
> > ") on "+nodeName+", but it is "+replica.getState());
> >
> > break;
> > }
> > }
> > }
> >
> > if (fromReplica == null)
> > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > "SolrCloud join: No active replicas for "+fromIndex+
> > " found in node " + nodeName);
> >
> > return fromReplica;
> > }
> >
> >
> > when i run joining from movies on 8983, slice length is 2 as movies have
> 2
> > shards. "fromReplica " was assigned in second cycle, because
> zkController
> > name is 8983 and replica name is 8984 in first cycle.
> >
> > but when run on 8984, "fromReplica" was assigned in first cycle, because
> > zkController name isand replica name both are 8984 in first cycle, so
> throw
> > "SolrCloud join: multiple shards not yet supported" in second cycle.
> >
> > Thanks for your patience, it's too long. i'm confused about why use this
> > way to judge "multiple shards", because the result is also wrong running
> on
> > 8983 even if didnt throw exception. why dont use slice length>1 to judge
> > "multiple shards" ? or maybe have other better way?
> >
> > Please advise.
> >
> > Thanks in advance!
> >
> > Erick Erickson <er...@gmail.com> 于2019年8月20日周二 下午7:39写道:
> >
> > > None of your images came through, the mail server aggressively strips
> > > attachments. You’ll have to put them somewhere and provide a link.
> > >
> > > Given that, I’m guessing without much data so this may be totally
> > > misguided. You mention ports 8984 and 8984. Assuming those are two
> > > different Solr JVMs, the fact that they’re running on the same machine
> is
> > > irrelevant; As far as SolrCloud is concerned, they are two separate
> > > machines. Your directors collection must be completely resident on both
> > > Solr instances for cross-collection join to work.
> > >
> > > Best,
> > > Erick
> > >
> > > > On Aug 19, 2019, at 9:39 PM, 王立生 <wa...@gmail.com> wrote:
> > > >
> > > > Hello,
> > > >
> > > > I have a question about solrCloud joining. i knew solrCloud joining
> can
> > > do join only when index is not splited to shards, but when i test it,
> i
> > > found a problem which make me confused.
> > > >
> > > > i tested it on version 8.2
> > > >
> > > > assuming i have 2 collections like sample about "joining" on solr
> > > offcial website,
> > > >
> > > > one collection called "movies", another called "movieDirectors".
> > > >
> > > > movies's fields: id, title, director_id
> > > > movieDirectors's fields: id, name, has_oscar
> > > >
> > > > the information of shards and replicas as below, i started two nodes
> on
> > > my laptop:
> > > >
> > > > moviesDirectors have 2 docs:
> > > >
> > > > movies also have 2 docs:
> > > >
> > > > everything is ok when i run query with "{!join from=id
> > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983
> and
> > > 8984, i can got expacted result:
> > > >
> > > > but when i run "{!join from=director_id fromIndex=movies
> > > to=id}title:"Dunkirk"" on 8983
> > > > got 1 doc and if i filter by "title:Get Out", i got nothing. i
> > > understood "Get Out" is not exist in 8983.
> > > >
> > > >
> > > > but question is coming, when i run "{!join from=director_id
> > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join:
> > > multiple shards not yet supported movies"
> > > > no matter what filter value is.
> > > >
> > > > i found following code:
> > > >
> > > >
> > > > when i run joining from movies on 8983, slice length is 2 as movies
> > have
> > > 2 shards. "fromReplica " was assigned in second cycle, because
> > > zkController name is 8983 and replica name is 8984 in first cycle.
> > > >
> > > > but when run on 8984, "fromReplica" was assigned in first cycle,
> > because
> > > zkController name isand replica name both are 8984 in first cycle, so
> > throw
> > > "SolrCloud join: multiple shards not yet supported" in second cycle.
> > > >
> > > > Thanks for your patience, it's too long. i'm confused about why use
> > this
> > > way to judge "multiple shards", because the result is also wrong
> running
> > on
> > > 8983 even if didnt throw exception. why dont use slice length>1 to
> judge
> > > "multiple shards" ? or maybe have other better way?
> > > >
> > > > Please advise.
> > > >
> > > > Thanks in advance!
> > > >
> > > >
> > > >
> > >
> > >
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>
Re: question about solrCloud joining
Posted by Mikhail Khludnev <mk...@apache.org>.
Hello, Lisheng.
I barely follow, but couldn't the space symbol in "title:Get Out"
cause the problem
?
Check debugQuery and nested query in local param.
On Tue, Aug 20, 2019 at 6:35 PM Lisheng Wang <wa...@gmail.com>
wrote:
> Hi Erick
>
> Thanks for your quick response and remaining me about attachment issue.
>
> Yes, i run on 2 different jvms that not related to if they are on same
> machine or not.
>
> let me describe my scenario, i have two collection:
>
> i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and 8984.
>
> 1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is on 8983
> 2. movies: 2 shard, 1 replica/shard shard1 is on 8983, shard2 is on
> 8984.
>
> collection movieDirectors has 2 docs:
> {
> "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> 1642343781358370816
> }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> 1642343828930166784
> }
> collection movies has 2 docs too:
> { "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> 1642343781358370816
> }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> 1642343828930166784
> }
> everything is ok when i run query with "{!join from=id
> fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983 and
> 8984, i can got expected result:
> { "responseHeader":{ "zkConnected":true, "status":0, "QTime":79, "params":{
> "q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
> to=director_id}has_oscar:true", "_":"1566313944099"}},
> "response":{"numFound
> ":2,"start":0,"maxScore":1.0,"docs":[ { "id":"1", "title":"Dunkirk", "
> director_id":"1", "_version_":1642343781358370816}, { "id":"2",
> "title":"Get
> Out", "director_id":"2", "_version_":1642343828930166784}] }}
> but when i run "{!join from=director_id fromIndex=movies
> to=id}title:"Dunkirk"" on 8983 got 1 doc,
> if i filter by "title:Get Out", i got nothing. i understood "Get Out" is
> not exist in 8983.
> { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3, "params":{
> "
> q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> to=id}title:\"Dunkirk\"", "_":"1566261450613"}}, "response":{"numFound":1,"
> start":0,"docs":[ { "id":"1", "name":"Christopher Nolan", "has_oscar":true,
> "_version_":1642343436642156544}] }}
>
> but question is coming, when i run "{!join from=director_id
> fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join:
> multiple shards not yet supported movies"
> no matter what filter value is.
>
> i found following code:
>
> private static String findLocalReplicaForFromIndex(ZkController
> zkController, String fromIndex) {
> String fromReplica = null;
>
> String nodeName = zkController.getNodeName();
> for (Slice slice :
>
> zkController.getClusterState().getCollection(fromIndex).getActiveSlicesArr())
> {
> if (fromReplica != null)
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> "SolrCloud join: multiple shards not yet supported " +
> fromIndex);
> for (Replica replica : slice.getReplicas()) {
> if (replica.getNodeName().equals(nodeName)) {
> fromReplica = replica.getStr(ZkStateReader.CORE_NAME_PROP);
> // found local replica, but is it Active?
> if (replica.getState() != Replica.State.ACTIVE)
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> "SolrCloud join: "+fromIndex+" has a local replica
> ("+fromReplica+
> ") on "+nodeName+", but it is "+replica.getState());
>
> break;
> }
> }
> }
>
> if (fromReplica == null)
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> "SolrCloud join: No active replicas for "+fromIndex+
> " found in node " + nodeName);
>
> return fromReplica;
> }
>
>
> when i run joining from movies on 8983, slice length is 2 as movies have 2
> shards. "fromReplica " was assigned in second cycle, because zkController
> name is 8983 and replica name is 8984 in first cycle.
>
> but when run on 8984, "fromReplica" was assigned in first cycle, because
> zkController name isand replica name both are 8984 in first cycle, so throw
> "SolrCloud join: multiple shards not yet supported" in second cycle.
>
> Thanks for your patience, it's too long. i'm confused about why use this
> way to judge "multiple shards", because the result is also wrong running on
> 8983 even if didnt throw exception. why dont use slice length>1 to judge
> "multiple shards" ? or maybe have other better way?
>
> Please advise.
>
> Thanks in advance!
>
> Erick Erickson <er...@gmail.com> 于2019年8月20日周二 下午7:39写道:
>
> > None of your images came through, the mail server aggressively strips
> > attachments. You’ll have to put them somewhere and provide a link.
> >
> > Given that, I’m guessing without much data so this may be totally
> > misguided. You mention ports 8984 and 8984. Assuming those are two
> > different Solr JVMs, the fact that they’re running on the same machine is
> > irrelevant; As far as SolrCloud is concerned, they are two separate
> > machines. Your directors collection must be completely resident on both
> > Solr instances for cross-collection join to work.
> >
> > Best,
> > Erick
> >
> > > On Aug 19, 2019, at 9:39 PM, 王立生 <wa...@gmail.com> wrote:
> > >
> > > Hello,
> > >
> > > I have a question about solrCloud joining. i knew solrCloud joining can
> > do join only when index is not splited to shards, but when i test it, i
> > found a problem which make me confused.
> > >
> > > i tested it on version 8.2
> > >
> > > assuming i have 2 collections like sample about "joining" on solr
> > offcial website,
> > >
> > > one collection called "movies", another called "movieDirectors".
> > >
> > > movies's fields: id, title, director_id
> > > movieDirectors's fields: id, name, has_oscar
> > >
> > > the information of shards and replicas as below, i started two nodes on
> > my laptop:
> > >
> > > moviesDirectors have 2 docs:
> > >
> > > movies also have 2 docs:
> > >
> > > everything is ok when i run query with "{!join from=id
> > fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983 and
> > 8984, i can got expacted result:
> > >
> > > but when i run "{!join from=director_id fromIndex=movies
> > to=id}title:"Dunkirk"" on 8983
> > > got 1 doc and if i filter by "title:Get Out", i got nothing. i
> > understood "Get Out" is not exist in 8983.
> > >
> > >
> > > but question is coming, when i run "{!join from=director_id
> > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join:
> > multiple shards not yet supported movies"
> > > no matter what filter value is.
> > >
> > > i found following code:
> > >
> > >
> > > when i run joining from movies on 8983, slice length is 2 as movies
> have
> > 2 shards. "fromReplica " was assigned in second cycle, because
> > zkController name is 8983 and replica name is 8984 in first cycle.
> > >
> > > but when run on 8984, "fromReplica" was assigned in first cycle,
> because
> > zkController name isand replica name both are 8984 in first cycle, so
> throw
> > "SolrCloud join: multiple shards not yet supported" in second cycle.
> > >
> > > Thanks for your patience, it's too long. i'm confused about why use
> this
> > way to judge "multiple shards", because the result is also wrong running
> on
> > 8983 even if didnt throw exception. why dont use slice length>1 to judge
> > "multiple shards" ? or maybe have other better way?
> > >
> > > Please advise.
> > >
> > > Thanks in advance!
> > >
> > >
> > >
> >
> >
>
--
Sincerely yours
Mikhail Khludnev
Re: question about solrCloud joining
Posted by Lisheng Wang <wa...@gmail.com>.
Hi Erick
Thanks for your quick response and remaining me about attachment issue.
Yes, i run on 2 different jvms that not related to if they are on same
machine or not.
let me describe my scenario, i have two collection:
i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and 8984.
1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is on 8983
2. movies: 2 shard, 1 replica/shard shard1 is on 8983, shard2 is on 8984.
collection movieDirectors has 2 docs:
{
"id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
1642343781358370816
}, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
1642343828930166784
}
collection movies has 2 docs too:
{ "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
1642343781358370816
}, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
1642343828930166784
}
everything is ok when i run query with "{!join from=id
fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983 and
8984, i can got expected result:
{ "responseHeader":{ "zkConnected":true, "status":0, "QTime":79, "params":{
"q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
to=director_id}has_oscar:true", "_":"1566313944099"}}, "response":{"numFound
":2,"start":0,"maxScore":1.0,"docs":[ { "id":"1", "title":"Dunkirk", "
director_id":"1", "_version_":1642343781358370816}, { "id":"2", "title":"Get
Out", "director_id":"2", "_version_":1642343828930166784}] }}
but when i run "{!join from=director_id fromIndex=movies
to=id}title:"Dunkirk"" on 8983 got 1 doc,
if i filter by "title:Get Out", i got nothing. i understood "Get Out" is
not exist in 8983.
{ "responseHeader":{ "zkConnected":true, "status":0, "QTime":3, "params":{ "
q":"*:*", "fq":"{!join from=director_id fromIndex=movies
to=id}title:\"Dunkirk\"", "_":"1566261450613"}}, "response":{"numFound":1,"
start":0,"docs":[ { "id":"1", "name":"Christopher Nolan", "has_oscar":true,
"_version_":1642343436642156544}] }}
but question is coming, when i run "{!join from=director_id
fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join:
multiple shards not yet supported movies"
no matter what filter value is.
i found following code:
private static String findLocalReplicaForFromIndex(ZkController
zkController, String fromIndex) {
String fromReplica = null;
String nodeName = zkController.getNodeName();
for (Slice slice :
zkController.getClusterState().getCollection(fromIndex).getActiveSlicesArr())
{
if (fromReplica != null)
throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
"SolrCloud join: multiple shards not yet supported " + fromIndex);
for (Replica replica : slice.getReplicas()) {
if (replica.getNodeName().equals(nodeName)) {
fromReplica = replica.getStr(ZkStateReader.CORE_NAME_PROP);
// found local replica, but is it Active?
if (replica.getState() != Replica.State.ACTIVE)
throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
"SolrCloud join: "+fromIndex+" has a local replica ("+fromReplica+
") on "+nodeName+", but it is "+replica.getState());
break;
}
}
}
if (fromReplica == null)
throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
"SolrCloud join: No active replicas for "+fromIndex+
" found in node " + nodeName);
return fromReplica;
}
when i run joining from movies on 8983, slice length is 2 as movies have 2
shards. "fromReplica " was assigned in second cycle, because zkController
name is 8983 and replica name is 8984 in first cycle.
but when run on 8984, "fromReplica" was assigned in first cycle, because
zkController name isand replica name both are 8984 in first cycle, so throw
"SolrCloud join: multiple shards not yet supported" in second cycle.
Thanks for your patience, it's too long. i'm confused about why use this
way to judge "multiple shards", because the result is also wrong running on
8983 even if didnt throw exception. why dont use slice length>1 to judge
"multiple shards" ? or maybe have other better way?
Please advise.
Thanks in advance!
Erick Erickson <er...@gmail.com> 于2019年8月20日周二 下午7:39写道:
> None of your images came through, the mail server aggressively strips
> attachments. You’ll have to put them somewhere and provide a link.
>
> Given that, I’m guessing without much data so this may be totally
> misguided. You mention ports 8984 and 8984. Assuming those are two
> different Solr JVMs, the fact that they’re running on the same machine is
> irrelevant; As far as SolrCloud is concerned, they are two separate
> machines. Your directors collection must be completely resident on both
> Solr instances for cross-collection join to work.
>
> Best,
> Erick
>
> > On Aug 19, 2019, at 9:39 PM, 王立生 <wa...@gmail.com> wrote:
> >
> > Hello,
> >
> > I have a question about solrCloud joining. i knew solrCloud joining can
> do join only when index is not splited to shards, but when i test it, i
> found a problem which make me confused.
> >
> > i tested it on version 8.2
> >
> > assuming i have 2 collections like sample about "joining" on solr
> offcial website,
> >
> > one collection called "movies", another called "movieDirectors".
> >
> > movies's fields: id, title, director_id
> > movieDirectors's fields: id, name, has_oscar
> >
> > the information of shards and replicas as below, i started two nodes on
> my laptop:
> >
> > moviesDirectors have 2 docs:
> >
> > movies also have 2 docs:
> >
> > everything is ok when i run query with "{!join from=id
> fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983 and
> 8984, i can got expacted result:
> >
> > but when i run "{!join from=director_id fromIndex=movies
> to=id}title:"Dunkirk"" on 8983
> > got 1 doc and if i filter by "title:Get Out", i got nothing. i
> understood "Get Out" is not exist in 8983.
> >
> >
> > but question is coming, when i run "{!join from=director_id
> fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join:
> multiple shards not yet supported movies"
> > no matter what filter value is.
> >
> > i found following code:
> >
> >
> > when i run joining from movies on 8983, slice length is 2 as movies have
> 2 shards. "fromReplica " was assigned in second cycle, because
> zkController name is 8983 and replica name is 8984 in first cycle.
> >
> > but when run on 8984, "fromReplica" was assigned in first cycle, because
> zkController name isand replica name both are 8984 in first cycle, so throw
> "SolrCloud join: multiple shards not yet supported" in second cycle.
> >
> > Thanks for your patience, it's too long. i'm confused about why use this
> way to judge "multiple shards", because the result is also wrong running on
> 8983 even if didnt throw exception. why dont use slice length>1 to judge
> "multiple shards" ? or maybe have other better way?
> >
> > Please advise.
> >
> > Thanks in advance!
> >
> >
> >
>
>
Re: question about solrCloud joining
Posted by Erick Erickson <er...@gmail.com>.
None of your images came through, the mail server aggressively strips attachments. You’ll have to put them somewhere and provide a link.
Given that, I’m guessing without much data so this may be totally misguided. You mention ports 8984 and 8984. Assuming those are two different Solr JVMs, the fact that they’re running on the same machine is irrelevant; As far as SolrCloud is concerned, they are two separate machines. Your directors collection must be completely resident on both Solr instances for cross-collection join to work.
Best,
Erick
> On Aug 19, 2019, at 9:39 PM, 王立生 <wa...@gmail.com> wrote:
>
> Hello,
>
> I have a question about solrCloud joining. i knew solrCloud joining can do join only when index is not splited to shards, but when i test it, i found a problem which make me confused.
>
> i tested it on version 8.2
>
> assuming i have 2 collections like sample about "joining" on solr offcial website,
>
> one collection called "movies", another called "movieDirectors".
>
> movies's fields: id, title, director_id
> movieDirectors's fields: id, name, has_oscar
>
> the information of shards and replicas as below, i started two nodes on my laptop:
>
> moviesDirectors have 2 docs:
>
> movies also have 2 docs:
>
> everything is ok when i run query with "{!join from=id fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983 and 8984, i can got expacted result:
>
> but when i run "{!join from=director_id fromIndex=movies to=id}title:"Dunkirk"" on 8983
> got 1 doc and if i filter by "title:Get Out", i got nothing. i understood "Get Out" is not exist in 8983.
>
>
> but question is coming, when i run "{!join from=director_id fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join: multiple shards not yet supported movies"
> no matter what filter value is.
>
> i found following code:
>
>
> when i run joining from movies on 8983, slice length is 2 as movies have 2 shards. "fromReplica " was assigned in second cycle, because zkController name is 8983 and replica name is 8984 in first cycle.
>
> but when run on 8984, "fromReplica" was assigned in first cycle, because zkController name isand replica name both are 8984 in first cycle, so throw "SolrCloud join: multiple shards not yet supported" in second cycle.
>
> Thanks for your patience, it's too long. i'm confused about why use this way to judge "multiple shards", because the result is also wrong running on 8983 even if didnt throw exception. why dont use slice length>1 to judge "multiple shards" ? or maybe have other better way?
>
> Please advise.
>
> Thanks in advance!
>
>
>