You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Komal Motwani <mo...@gmail.com> on 2019/08/09 09:23:33 UTC
Searches across Cores
Hi,
I have a use case where I would like a query to span across Cores
(Multi-Core); all the cores involved do have same schema. I have started
using solr just recently and have been trying to find ways to achieve this
but couldn’t find any solution so far (Distributed searches, shards are not
what I am looking for). I remember in one of the tech talks, there was a
mention of this feature to be included in future releases. Appreciate any
pointers to help me progress further.
Thanks,
Komal Motwani
Re: Searches across Cores
Posted by Sidharth Negi <si...@gmail.com>.
Hi,
If the number of cores spanned is low, I guess firing parallel queries and
taking union or intersection should work since their schema is the same. Do
you notice any perceivable difference in performance?
Best,
Sidharth
On Fri, Aug 9, 2019 at 2:54 PM Komal Motwani <mo...@gmail.com>
wrote:
> Hi,
>
>
>
> I have a use case where I would like a query to span across Cores
> (Multi-Core); all the cores involved do have same schema. I have started
> using solr just recently and have been trying to find ways to achieve this
> but couldn’t find any solution so far (Distributed searches, shards are not
> what I am looking for). I remember in one of the tech talks, there was a
> mention of this feature to be included in future releases. Appreciate any
> pointers to help me progress further.
>
>
>
> Thanks,
>
> Komal Motwani
>
Re: Searches across Cores
Posted by Nicolas Franck <Ni...@UGent.be>.
He's right. You can use the parameter "shards" for a very long time,
even before the whole solr cloud existed.
e.g. http://localhost:8983/solr/core0/select
with parameters:
shards = localhost:8983/solr/core0,example.com:8983/solr/core0<http://example.com:8983/solr/core0>
q = *:*
defType = lucene
Yes, I used same core name twice (in the path and in the parameter), but I do not see another way.
You need to start the query at a query handler..
I guess your data is generated by several parties,
each on their own core? That makes sense.
On 9 Aug 2019, at 19:21, Vadim Ivanov <va...@spb.ntk-intourist.ru>> wrote:
May be consider having one collection with implicit sharding ?
This way you can have all advantages of solrcloud and can control content of each core "manualy" as well as query them independently (&distrib=false)
... or some of them using &shards=core1,core2 as was proposed before
Quote from doc
" If you created the collection and defined the "implicit" router at the time of creation, you can additionally define a router.field parameter to use a field from each document to identify a shard where the document belongs. If the field specified is missing in the document, however, the document will be rejected. You could also use the _route_ parameter to name a specific shard."
--
Vadim
-----Original Message-----
From: Komal Motwani [mailto:motwani.komal@gmail.com]
Sent: Friday, August 09, 2019 7:57 PM
To: solr-user@lucene.apache.org<ma...@lucene.apache.org>
Subject: Re: Searches across Cores
For some good reasons, SolrCloud is not an option for me.
I need to run nested graph queries so firing parallel queries and taking
union/intersection won't work.
I am aware of achieving this via shards however I am looking for ways to
achieve this via multiple cores. We already have data existing in multiple
cores on which i need to add this feature.
Thanks,
Komal Motwani
On Fri, Aug 9, 2019 at 8:57 PM Erick Erickson <er...@gmail.com>>
wrote:
So my question is why do you have individual cores? Why not use SolrCloud
and collections and have this happen automatically?
There may be very good reasons, this is more if a sanity check….
On Aug 9, 2019, at 8:02 AM, Jan Høydahl <ja...@cominvent.com>> wrote:
USE request param &shards=core1,core2 or if on separate machines
host:port/solr/core1,host:port/solr/core2
Jan Høydahl
9. aug. 2019 kl. 11:23 skrev Komal Motwani
<mo...@gmail.com>>:
Hi,
I have a use case where I would like a query to span across Cores
(Multi-Core); all the cores involved do have same schema. I have started
using solr just recently and have been trying to find ways to achieve
this
but couldn’t find any solution so far (Distributed searches, shards are
not
what I am looking for). I remember in one of the tech talks, there was a
mention of this feature to be included in future releases. Appreciate
any
pointers to help me progress further.
Thanks,
Komal Motwani
RE: Searches across Cores
Posted by Vadim Ivanov <va...@spb.ntk-intourist.ru>.
May be consider having one collection with implicit sharding ?
This way you can have all advantages of solrcloud and can control content of each core "manualy" as well as query them independently (&distrib=false)
... or some of them using &shards=core1,core2 as was proposed before
Quote from doc
" If you created the collection and defined the "implicit" router at the time of creation, you can additionally define a router.field parameter to use a field from each document to identify a shard where the document belongs. If the field specified is missing in the document, however, the document will be rejected. You could also use the _route_ parameter to name a specific shard."
--
Vadim
> -----Original Message-----
> From: Komal Motwani [mailto:motwani.komal@gmail.com]
> Sent: Friday, August 09, 2019 7:57 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Searches across Cores
>
> For some good reasons, SolrCloud is not an option for me.
> I need to run nested graph queries so firing parallel queries and taking
> union/intersection won't work.
> I am aware of achieving this via shards however I am looking for ways to
> achieve this via multiple cores. We already have data existing in multiple
> cores on which i need to add this feature.
>
> Thanks,
> Komal Motwani
>
> On Fri, Aug 9, 2019 at 8:57 PM Erick Erickson <er...@gmail.com>
> wrote:
>
> > So my question is why do you have individual cores? Why not use SolrCloud
> > and collections and have this happen automatically?
> >
> > There may be very good reasons, this is more if a sanity check….
> >
> > > On Aug 9, 2019, at 8:02 AM, Jan Høydahl <ja...@cominvent.com> wrote:
> > >
> > > USE request param &shards=core1,core2 or if on separate machines
> > host:port/solr/core1,host:port/solr/core2
> > >
> > > Jan Høydahl
> > >
> > >> 9. aug. 2019 kl. 11:23 skrev Komal Motwani
> <mo...@gmail.com>:
> > >>
> > >> Hi,
> > >>
> > >>
> > >>
> > >> I have a use case where I would like a query to span across Cores
> > >> (Multi-Core); all the cores involved do have same schema. I have started
> > >> using solr just recently and have been trying to find ways to achieve
> > this
> > >> but couldn’t find any solution so far (Distributed searches, shards are
> > not
> > >> what I am looking for). I remember in one of the tech talks, there was a
> > >> mention of this feature to be included in future releases. Appreciate
> > any
> > >> pointers to help me progress further.
> > >>
> > >>
> > >>
> > >> Thanks,
> > >>
> > >> Komal Motwani
> >
> >
Re: Searches across Cores
Posted by Komal Motwani <mo...@gmail.com>.
For some good reasons, SolrCloud is not an option for me.
I need to run nested graph queries so firing parallel queries and taking
union/intersection won't work.
I am aware of achieving this via shards however I am looking for ways to
achieve this via multiple cores. We already have data existing in multiple
cores on which i need to add this feature.
Thanks,
Komal Motwani
On Fri, Aug 9, 2019 at 8:57 PM Erick Erickson <er...@gmail.com>
wrote:
> So my question is why do you have individual cores? Why not use SolrCloud
> and collections and have this happen automatically?
>
> There may be very good reasons, this is more if a sanity check….
>
> > On Aug 9, 2019, at 8:02 AM, Jan Høydahl <ja...@cominvent.com> wrote:
> >
> > USE request param &shards=core1,core2 or if on separate machines
> host:port/solr/core1,host:port/solr/core2
> >
> > Jan Høydahl
> >
> >> 9. aug. 2019 kl. 11:23 skrev Komal Motwani <mo...@gmail.com>:
> >>
> >> Hi,
> >>
> >>
> >>
> >> I have a use case where I would like a query to span across Cores
> >> (Multi-Core); all the cores involved do have same schema. I have started
> >> using solr just recently and have been trying to find ways to achieve
> this
> >> but couldn’t find any solution so far (Distributed searches, shards are
> not
> >> what I am looking for). I remember in one of the tech talks, there was a
> >> mention of this feature to be included in future releases. Appreciate
> any
> >> pointers to help me progress further.
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Komal Motwani
>
>
Re: Searches across Cores
Posted by Erick Erickson <er...@gmail.com>.
So my question is why do you have individual cores? Why not use SolrCloud and collections and have this happen automatically?
There may be very good reasons, this is more if a sanity check….
> On Aug 9, 2019, at 8:02 AM, Jan Høydahl <ja...@cominvent.com> wrote:
>
> USE request param &shards=core1,core2 or if on separate machines host:port/solr/core1,host:port/solr/core2
>
> Jan Høydahl
>
>> 9. aug. 2019 kl. 11:23 skrev Komal Motwani <mo...@gmail.com>:
>>
>> Hi,
>>
>>
>>
>> I have a use case where I would like a query to span across Cores
>> (Multi-Core); all the cores involved do have same schema. I have started
>> using solr just recently and have been trying to find ways to achieve this
>> but couldn’t find any solution so far (Distributed searches, shards are not
>> what I am looking for). I remember in one of the tech talks, there was a
>> mention of this feature to be included in future releases. Appreciate any
>> pointers to help me progress further.
>>
>>
>>
>> Thanks,
>>
>> Komal Motwani
Re: Searches across Cores
Posted by Jan Høydahl <ja...@cominvent.com>.
USE request param &shards=core1,core2 or if on separate machines host:port/solr/core1,host:port/solr/core2
Jan Høydahl
> 9. aug. 2019 kl. 11:23 skrev Komal Motwani <mo...@gmail.com>:
>
> Hi,
>
>
>
> I have a use case where I would like a query to span across Cores
> (Multi-Core); all the cores involved do have same schema. I have started
> using solr just recently and have been trying to find ways to achieve this
> but couldn’t find any solution so far (Distributed searches, shards are not
> what I am looking for). I remember in one of the tech talks, there was a
> mention of this feature to be included in future releases. Appreciate any
> pointers to help me progress further.
>
>
>
> Thanks,
>
> Komal Motwani