You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Mike <mz...@gmail.com> on 2022/12/28 01:40:26 UTC
Multiple cores
Hello!
I have a Solr server with multiple cores with the same schema.
For some queries I want to query several cores at the same time.
I looked at the options and tried to configure what didn't work and that's
how I ended up here.
How can I query on query A -> Core1
and at query B .-> Core1 and Core2
Please help me with the configuration, thanks.
Mike
Re: Multiple cores
Posted by Thomas Corthals <th...@klascement.net>.
For our corpus, term frequency gets in the way of how we want to rank
search results rather than being helpful.
I put this in our schema to effectively turn Okapi BM25
<https://en.wikipedia.org/wiki/Okapi_BM25> into BM15:
<similarity class="solr.BM25SimilarityFactory">
<float name="b">0</float>
</similarity>
Thomas
Op wo 28 dec. 2022 om 14:35 schreef Eric Pugh <
epugh@opensourceconnections.com>:
> For a very long time, that was what folks always say…. “The different
> IDF” is going to be an issue. My opinion is that there are many other
> things that REALLY effect your overall relevance a lot more then unbalanced
> IDF. Folks worry way too much about IDF, and not enough about “what are
> your crazy synonyms.txt or stop words.txt doing to you?”.
>
> You should go use a tool like Quepid (www.quepid.com) and set up a
> baseline relevance test case, and just try the experiment, that way instead
> of making decisions based on hunches, you have data!
>
>
>
> > On Dec 28, 2022, at 8:30 AM, Dave <ha...@gmail.com> wrote:
> >
> > Eric, that is super clever. But how does it effect ranking if you do a
> general search? Since each collection has its own idf etc?
> > -Dave
> >
> >> On Dec 28, 2022, at 7:03 AM, Eric Pugh <ep...@opensourceconnections.com>
> wrote:
> >>
> >> You may find it an easier path forward to just move to SolrCloud. You
> can run a single Solr server with multiple collections and use the embedded
> ZK to avoid setting up the full ZK ensemble….
> >>
> >>> On Dec 28, 2022, at 12:04 AM, Mike <mz...@gmail.com> wrote:
> >>>
> >>> Yes, it should be the same, it works without basic authentication.
> >>>
> >>> Thank you
> >>>
> >>>> Am Mi., 28. Dez. 2022 um 05:48 Uhr schrieb Srijan <shreejan@gmail.com
> >:
> >>>>
> >>>>
> >>>>
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-15237/comment/17626195
> >>>>
> >>>> Same issue?
> >>>>
> >>>>> On Tue, Dec 27, 2022, 19:59 Mike <mz...@gmail.com> wrote:
> >>>>
> >>>>> I get a 401 require authentication error when I query with &shards=
> >>>>>
> >>>>> Do you or anyone else have any idea why?
> >>>>>
> >>>>> Am Mi., 28. Dez. 2022 um 04:10 Uhr schrieb Shawn Heisey <
> >>>>> apache@elyograg.org
> >>>>>> :
> >>>>>
> >>>>>> On 12/27/22 19:50, Mike wrote:
> >>>>>>> The server is not in cloud mode, it is a standalone server.
> >>>>>>> I don't understand where to put the query line, in the URL, with
> what
> >>>>>> query
> >>>>>>> parameter (?=) ?
> >>>>>>>
> >>>>>>> Do I have to change something in solr.xml or solrconfig?
> >>>>>>
> >>>>>> If you put it in the URL:
> >>>>>>
> >>>>>> &shards=server:port/solr/core1,server:port/solr/core2
> >>>>>>
> >>>>>> The way I did it is created a special core with no index of its own
> and
> >>>>>> put the following line in the solrconfig.xml, in the defaults
> section
> >>>> of
> >>>>>> the search handler:
> >>>>>>
> >>>>>> <str
> >>>>>> name="shards">
> >>>>>>
> >>>>>
> >>>>
> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
> >>>>>> </str>
> >>>>>>
> >>>>>> Queries never went directly to the cores with data, they only went
> to
> >>>>>> the special core. I wrote an indexing system that would ensure
> >>>>>> documents ended up in the correct shard.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Shawn
> >>>>>>
> >>>>>
> >>>>
> >>
> >> _______________________
> >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> | http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> >> This e-mail and all contents, including attachments, is considered to
> be Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>
Re: Multiple cores
Posted by David Hastings <ha...@gmail.com>.
this is actually something I experienced using things like MLT in order to
get "similar" documents, is the corpus has to match, or else it all goes
out the window. so yeah if you have multiple cores/collections with the
same exact type of documents you can be pretty safe, but once you start
mixing a history book collection with a novel, and a news archive
collection, things get strange pretty quickly. and god forbid you to have
different languages
On Wed, Dec 28, 2022 at 1:25 PM Walter Underwood <wu...@wunderwood.org>
wrote:
> Using local IDF is usually not a problem if documents are randomly
> distributed between shards or collections. It can be a problem if terms are
> clustered in one collection/shard.
>
> Assume a news archive with one collection for the current year and one for
> everything else. A recently hot topic, like “fentanyl”, will have a lower
> IDF in the recent collection. Similar things can happen with collections
> from each part of a company, say all the printer documents are in one
> collection, so “LaserJet” is a common term there.
>
> Global IDF is very slow in Solr right now, but there is a fast method
> invented by Infoseek. That patent expired several years ago, so we should
> implement it.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
> > On Dec 28, 2022, at 5:35 AM, Eric Pugh <ep...@opensourceconnections.com>
> wrote:
> >
> > For a very long time, that was what folks always say…. “The different
> IDF” is going to be an issue. My opinion is that there are many other
> things that REALLY effect your overall relevance a lot more then unbalanced
> IDF. Folks worry way too much about IDF, and not enough about “what are
> your crazy synonyms.txt or stop words.txt doing to you?”.
> >
> > You should go use a tool like Quepid (www.quepid.com) and set up a
> baseline relevance test case, and just try the experiment, that way instead
> of making decisions based on hunches, you have data!
> >
> >
> >
> >> On Dec 28, 2022, at 8:30 AM, Dave <ha...@gmail.com> wrote:
> >>
> >> Eric, that is super clever. But how does it effect ranking if you do a
> general search? Since each collection has its own idf etc?
> >> -Dave
> >>
> >>> On Dec 28, 2022, at 7:03 AM, Eric Pugh <
> epugh@opensourceconnections.com> wrote:
> >>>
> >>> You may find it an easier path forward to just move to SolrCloud.
> You can run a single Solr server with multiple collections and use the
> embedded ZK to avoid setting up the full ZK ensemble….
> >>>
> >>>> On Dec 28, 2022, at 12:04 AM, Mike <mz...@gmail.com> wrote:
> >>>>
> >>>> Yes, it should be the same, it works without basic authentication.
> >>>>
> >>>> Thank you
> >>>>
> >>>>> Am Mi., 28. Dez. 2022 um 05:48 Uhr schrieb Srijan <
> shreejan@gmail.com>:
> >>>>>
> >>>>>
> >>>>>
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-15237/comment/17626195
> >>>>>
> >>>>> Same issue?
> >>>>>
> >>>>>> On Tue, Dec 27, 2022, 19:59 Mike <mz...@gmail.com> wrote:
> >>>>>
> >>>>>> I get a 401 require authentication error when I query with &shards=
> >>>>>>
> >>>>>> Do you or anyone else have any idea why?
> >>>>>>
> >>>>>> Am Mi., 28. Dez. 2022 um 04:10 Uhr schrieb Shawn Heisey <
> >>>>>> apache@elyograg.org
> >>>>>>> :
> >>>>>>
> >>>>>>> On 12/27/22 19:50, Mike wrote:
> >>>>>>>> The server is not in cloud mode, it is a standalone server.
> >>>>>>>> I don't understand where to put the query line, in the URL, with
> what
> >>>>>>> query
> >>>>>>>> parameter (?=) ?
> >>>>>>>>
> >>>>>>>> Do I have to change something in solr.xml or solrconfig?
> >>>>>>>
> >>>>>>> If you put it in the URL:
> >>>>>>>
> >>>>>>> &shards=server:port/solr/core1,server:port/solr/core2
> >>>>>>>
> >>>>>>> The way I did it is created a special core with no index of its
> own and
> >>>>>>> put the following line in the solrconfig.xml, in the defaults
> section
> >>>>> of
> >>>>>>> the search handler:
> >>>>>>>
> >>>>>>> <str
> >>>>>>> name="shards">
> >>>>>>>
> >>>>>>
> >>>>>
> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
> >>>>>>> </str>
> >>>>>>>
> >>>>>>> Queries never went directly to the cores with data, they only went
> to
> >>>>>>> the special core. I wrote an indexing system that would ensure
> >>>>>>> documents ended up in the correct shard.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Shawn
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>> _______________________
> >>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> | http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> >>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> >>> This e-mail and all contents, including attachments, is considered to
> be Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >>>
> >
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> > This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >
>
>
Re: Multiple cores
Posted by Walter Underwood <wu...@wunderwood.org>.
Using local IDF is usually not a problem if documents are randomly distributed between shards or collections. It can be a problem if terms are clustered in one collection/shard.
Assume a news archive with one collection for the current year and one for everything else. A recently hot topic, like “fentanyl”, will have a lower IDF in the recent collection. Similar things can happen with collections from each part of a company, say all the printer documents are in one collection, so “LaserJet” is a common term there.
Global IDF is very slow in Solr right now, but there is a fast method invented by Infoseek. That patent expired several years ago, so we should implement it.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Dec 28, 2022, at 5:35 AM, Eric Pugh <ep...@opensourceconnections.com> wrote:
>
> For a very long time, that was what folks always say…. “The different IDF” is going to be an issue. My opinion is that there are many other things that REALLY effect your overall relevance a lot more then unbalanced IDF. Folks worry way too much about IDF, and not enough about “what are your crazy synonyms.txt or stop words.txt doing to you?”.
>
> You should go use a tool like Quepid (www.quepid.com) and set up a baseline relevance test case, and just try the experiment, that way instead of making decisions based on hunches, you have data!
>
>
>
>> On Dec 28, 2022, at 8:30 AM, Dave <ha...@gmail.com> wrote:
>>
>> Eric, that is super clever. But how does it effect ranking if you do a general search? Since each collection has its own idf etc?
>> -Dave
>>
>>> On Dec 28, 2022, at 7:03 AM, Eric Pugh <ep...@opensourceconnections.com> wrote:
>>>
>>> You may find it an easier path forward to just move to SolrCloud. You can run a single Solr server with multiple collections and use the embedded ZK to avoid setting up the full ZK ensemble….
>>>
>>>> On Dec 28, 2022, at 12:04 AM, Mike <mz...@gmail.com> wrote:
>>>>
>>>> Yes, it should be the same, it works without basic authentication.
>>>>
>>>> Thank you
>>>>
>>>>> Am Mi., 28. Dez. 2022 um 05:48 Uhr schrieb Srijan <sh...@gmail.com>:
>>>>>
>>>>>
>>>>> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-15237/comment/17626195
>>>>>
>>>>> Same issue?
>>>>>
>>>>>> On Tue, Dec 27, 2022, 19:59 Mike <mz...@gmail.com> wrote:
>>>>>
>>>>>> I get a 401 require authentication error when I query with &shards=
>>>>>>
>>>>>> Do you or anyone else have any idea why?
>>>>>>
>>>>>> Am Mi., 28. Dez. 2022 um 04:10 Uhr schrieb Shawn Heisey <
>>>>>> apache@elyograg.org
>>>>>>> :
>>>>>>
>>>>>>> On 12/27/22 19:50, Mike wrote:
>>>>>>>> The server is not in cloud mode, it is a standalone server.
>>>>>>>> I don't understand where to put the query line, in the URL, with what
>>>>>>> query
>>>>>>>> parameter (?=) ?
>>>>>>>>
>>>>>>>> Do I have to change something in solr.xml or solrconfig?
>>>>>>>
>>>>>>> If you put it in the URL:
>>>>>>>
>>>>>>> &shards=server:port/solr/core1,server:port/solr/core2
>>>>>>>
>>>>>>> The way I did it is created a special core with no index of its own and
>>>>>>> put the following line in the solrconfig.xml, in the defaults section
>>>>> of
>>>>>>> the search handler:
>>>>>>>
>>>>>>> <str
>>>>>>> name="shards">
>>>>>>>
>>>>>>
>>>>> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
>>>>>>> </str>
>>>>>>>
>>>>>>> Queries never went directly to the cores with data, they only went to
>>>>>>> the special core. I wrote an indexing system that would ensure
>>>>>>> documents ended up in the correct shard.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Shawn
>>>>>>>
>>>>>>
>>>>>
>>>
>>> _______________________
>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>>> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>>>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>
Re: Multiple cores
Posted by Eric Pugh <ep...@opensourceconnections.com>.
For a very long time, that was what folks always say…. “The different IDF” is going to be an issue. My opinion is that there are many other things that REALLY effect your overall relevance a lot more then unbalanced IDF. Folks worry way too much about IDF, and not enough about “what are your crazy synonyms.txt or stop words.txt doing to you?”.
You should go use a tool like Quepid (www.quepid.com) and set up a baseline relevance test case, and just try the experiment, that way instead of making decisions based on hunches, you have data!
> On Dec 28, 2022, at 8:30 AM, Dave <ha...@gmail.com> wrote:
>
> Eric, that is super clever. But how does it effect ranking if you do a general search? Since each collection has its own idf etc?
> -Dave
>
>> On Dec 28, 2022, at 7:03 AM, Eric Pugh <ep...@opensourceconnections.com> wrote:
>>
>> You may find it an easier path forward to just move to SolrCloud. You can run a single Solr server with multiple collections and use the embedded ZK to avoid setting up the full ZK ensemble….
>>
>>> On Dec 28, 2022, at 12:04 AM, Mike <mz...@gmail.com> wrote:
>>>
>>> Yes, it should be the same, it works without basic authentication.
>>>
>>> Thank you
>>>
>>>> Am Mi., 28. Dez. 2022 um 05:48 Uhr schrieb Srijan <sh...@gmail.com>:
>>>>
>>>>
>>>> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-15237/comment/17626195
>>>>
>>>> Same issue?
>>>>
>>>>> On Tue, Dec 27, 2022, 19:59 Mike <mz...@gmail.com> wrote:
>>>>
>>>>> I get a 401 require authentication error when I query with &shards=
>>>>>
>>>>> Do you or anyone else have any idea why?
>>>>>
>>>>> Am Mi., 28. Dez. 2022 um 04:10 Uhr schrieb Shawn Heisey <
>>>>> apache@elyograg.org
>>>>>> :
>>>>>
>>>>>> On 12/27/22 19:50, Mike wrote:
>>>>>>> The server is not in cloud mode, it is a standalone server.
>>>>>>> I don't understand where to put the query line, in the URL, with what
>>>>>> query
>>>>>>> parameter (?=) ?
>>>>>>>
>>>>>>> Do I have to change something in solr.xml or solrconfig?
>>>>>>
>>>>>> If you put it in the URL:
>>>>>>
>>>>>> &shards=server:port/solr/core1,server:port/solr/core2
>>>>>>
>>>>>> The way I did it is created a special core with no index of its own and
>>>>>> put the following line in the solrconfig.xml, in the defaults section
>>>> of
>>>>>> the search handler:
>>>>>>
>>>>>> <str
>>>>>> name="shards">
>>>>>>
>>>>>
>>>> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
>>>>>> </str>
>>>>>>
>>>>>> Queries never went directly to the cores with data, they only went to
>>>>>> the special core. I wrote an indexing system that would ensure
>>>>>> documents ended up in the correct shard.
>>>>>>
>>>>>> Thanks,
>>>>>> Shawn
>>>>>>
>>>>>
>>>>
>>
>> _______________________
>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>>
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Multiple cores
Posted by Dave <ha...@gmail.com>.
Eric, that is super clever. But how does it effect ranking if you do a general search? Since each collection has its own idf etc?
-Dave
> On Dec 28, 2022, at 7:03 AM, Eric Pugh <ep...@opensourceconnections.com> wrote:
>
> You may find it an easier path forward to just move to SolrCloud. You can run a single Solr server with multiple collections and use the embedded ZK to avoid setting up the full ZK ensemble….
>
>> On Dec 28, 2022, at 12:04 AM, Mike <mz...@gmail.com> wrote:
>>
>> Yes, it should be the same, it works without basic authentication.
>>
>> Thank you
>>
>>> Am Mi., 28. Dez. 2022 um 05:48 Uhr schrieb Srijan <sh...@gmail.com>:
>>>
>>>
>>> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-15237/comment/17626195
>>>
>>> Same issue?
>>>
>>>> On Tue, Dec 27, 2022, 19:59 Mike <mz...@gmail.com> wrote:
>>>
>>>> I get a 401 require authentication error when I query with &shards=
>>>>
>>>> Do you or anyone else have any idea why?
>>>>
>>>> Am Mi., 28. Dez. 2022 um 04:10 Uhr schrieb Shawn Heisey <
>>>> apache@elyograg.org
>>>>> :
>>>>
>>>>> On 12/27/22 19:50, Mike wrote:
>>>>>> The server is not in cloud mode, it is a standalone server.
>>>>>> I don't understand where to put the query line, in the URL, with what
>>>>> query
>>>>>> parameter (?=) ?
>>>>>>
>>>>>> Do I have to change something in solr.xml or solrconfig?
>>>>>
>>>>> If you put it in the URL:
>>>>>
>>>>> &shards=server:port/solr/core1,server:port/solr/core2
>>>>>
>>>>> The way I did it is created a special core with no index of its own and
>>>>> put the following line in the solrconfig.xml, in the defaults section
>>> of
>>>>> the search handler:
>>>>>
>>>>> <str
>>>>> name="shards">
>>>>>
>>>>
>>> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
>>>>> </str>
>>>>>
>>>>> Queries never went directly to the cores with data, they only went to
>>>>> the special core. I wrote an indexing system that would ensure
>>>>> documents ended up in the correct shard.
>>>>>
>>>>> Thanks,
>>>>> Shawn
>>>>>
>>>>
>>>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>
Re: Multiple cores
Posted by Eric Pugh <ep...@opensourceconnections.com>.
You may find it an easier path forward to just move to SolrCloud. You can run a single Solr server with multiple collections and use the embedded ZK to avoid setting up the full ZK ensemble….
> On Dec 28, 2022, at 12:04 AM, Mike <mz...@gmail.com> wrote:
>
> Yes, it should be the same, it works without basic authentication.
>
> Thank you
>
> Am Mi., 28. Dez. 2022 um 05:48 Uhr schrieb Srijan <sh...@gmail.com>:
>
>>
>> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-15237/comment/17626195
>>
>> Same issue?
>>
>> On Tue, Dec 27, 2022, 19:59 Mike <mz...@gmail.com> wrote:
>>
>>> I get a 401 require authentication error when I query with &shards=
>>>
>>> Do you or anyone else have any idea why?
>>>
>>> Am Mi., 28. Dez. 2022 um 04:10 Uhr schrieb Shawn Heisey <
>>> apache@elyograg.org
>>>> :
>>>
>>>> On 12/27/22 19:50, Mike wrote:
>>>>> The server is not in cloud mode, it is a standalone server.
>>>>> I don't understand where to put the query line, in the URL, with what
>>>> query
>>>>> parameter (?=) ?
>>>>>
>>>>> Do I have to change something in solr.xml or solrconfig?
>>>>
>>>> If you put it in the URL:
>>>>
>>>> &shards=server:port/solr/core1,server:port/solr/core2
>>>>
>>>> The way I did it is created a special core with no index of its own and
>>>> put the following line in the solrconfig.xml, in the defaults section
>> of
>>>> the search handler:
>>>>
>>>> <str
>>>> name="shards">
>>>>
>>>
>> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
>>>> </str>
>>>>
>>>> Queries never went directly to the cores with data, they only went to
>>>> the special core. I wrote an indexing system that would ensure
>>>> documents ended up in the correct shard.
>>>>
>>>> Thanks,
>>>> Shawn
>>>>
>>>
>>
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Multiple cores
Posted by Mike <mz...@gmail.com>.
Yes, it should be the same, it works without basic authentication.
Thank you
Am Mi., 28. Dez. 2022 um 05:48 Uhr schrieb Srijan <sh...@gmail.com>:
>
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-15237/comment/17626195
>
> Same issue?
>
> On Tue, Dec 27, 2022, 19:59 Mike <mz...@gmail.com> wrote:
>
> > I get a 401 require authentication error when I query with &shards=
> >
> > Do you or anyone else have any idea why?
> >
> > Am Mi., 28. Dez. 2022 um 04:10 Uhr schrieb Shawn Heisey <
> > apache@elyograg.org
> > >:
> >
> > > On 12/27/22 19:50, Mike wrote:
> > > > The server is not in cloud mode, it is a standalone server.
> > > > I don't understand where to put the query line, in the URL, with what
> > > query
> > > > parameter (?=) ?
> > > >
> > > > Do I have to change something in solr.xml or solrconfig?
> > >
> > > If you put it in the URL:
> > >
> > > &shards=server:port/solr/core1,server:port/solr/core2
> > >
> > > The way I did it is created a special core with no index of its own and
> > > put the following line in the solrconfig.xml, in the defaults section
> of
> > > the search handler:
> > >
> > > <str
> > > name="shards">
> > >
> >
> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
> > > </str>
> > >
> > > Queries never went directly to the cores with data, they only went to
> > > the special core. I wrote an indexing system that would ensure
> > > documents ended up in the correct shard.
> > >
> > > Thanks,
> > > Shawn
> > >
> >
>
Re: Multiple cores
Posted by Srijan <sh...@gmail.com>.
https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-15237/comment/17626195
Same issue?
On Tue, Dec 27, 2022, 19:59 Mike <mz...@gmail.com> wrote:
> I get a 401 require authentication error when I query with &shards=
>
> Do you or anyone else have any idea why?
>
> Am Mi., 28. Dez. 2022 um 04:10 Uhr schrieb Shawn Heisey <
> apache@elyograg.org
> >:
>
> > On 12/27/22 19:50, Mike wrote:
> > > The server is not in cloud mode, it is a standalone server.
> > > I don't understand where to put the query line, in the URL, with what
> > query
> > > parameter (?=) ?
> > >
> > > Do I have to change something in solr.xml or solrconfig?
> >
> > If you put it in the URL:
> >
> > &shards=server:port/solr/core1,server:port/solr/core2
> >
> > The way I did it is created a special core with no index of its own and
> > put the following line in the solrconfig.xml, in the defaults section of
> > the search handler:
> >
> > <str
> > name="shards">
> >
> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
> > </str>
> >
> > Queries never went directly to the cores with data, they only went to
> > the special core. I wrote an indexing system that would ensure
> > documents ended up in the correct shard.
> >
> > Thanks,
> > Shawn
> >
>
Re: Multiple cores
Posted by Mike <mz...@gmail.com>.
I get a 401 require authentication error when I query with &shards=
Do you or anyone else have any idea why?
Am Mi., 28. Dez. 2022 um 04:10 Uhr schrieb Shawn Heisey <apache@elyograg.org
>:
> On 12/27/22 19:50, Mike wrote:
> > The server is not in cloud mode, it is a standalone server.
> > I don't understand where to put the query line, in the URL, with what
> query
> > parameter (?=) ?
> >
> > Do I have to change something in solr.xml or solrconfig?
>
> If you put it in the URL:
>
> &shards=server:port/solr/core1,server:port/solr/core2
>
> The way I did it is created a special core with no index of its own and
> put the following line in the solrconfig.xml, in the defaults section of
> the search handler:
>
> <str
> name="shards">
> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
> </str>
>
> Queries never went directly to the cores with data, they only went to
> the special core. I wrote an indexing system that would ensure
> documents ended up in the correct shard.
>
> Thanks,
> Shawn
>
Re: Multiple cores
Posted by Shawn Heisey <ap...@elyograg.org>.
On 12/27/22 19:50, Mike wrote:
> The server is not in cloud mode, it is a standalone server.
> I don't understand where to put the query line, in the URL, with what query
> parameter (?=) ?
>
> Do I have to change something in solr.xml or solrconfig?
If you put it in the URL:
&shards=server:port/solr/core1,server:port/solr/core2
The way I did it is created a special core with no index of its own and
put the following line in the solrconfig.xml, in the defaults section of
the search handler:
<str
name="shards">idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live</str>
Queries never went directly to the cores with data, they only went to
the special core. I wrote an indexing system that would ensure
documents ended up in the correct shard.
Thanks,
Shawn
Re: Multiple cores
Posted by Mike <mz...@gmail.com>.
Hello Shawn!
Thank you for your quick response!
The server is not in cloud mode, it is a standalone server.
I don't understand where to put the query line, in the URL, with what query
parameter (?=) ?
Do I have to change something in solr.xml or solrconfig?
Thanks mike
Am Mi., 28. Dez. 2022 um 03:44 Uhr schrieb Shawn Heisey <apache@elyograg.org
>:
> On 12/27/22 18:40, Mike wrote:
> > I have a Solr server with multiple cores with the same schema.
> > For some queries I want to query several cores at the same time.
> > I looked at the options and tried to configure what didn't work and
> that's
> > how I ended up here.
> >
> > How can I query on query A -> Core1
> > and at query B .-> Core1 and Core2
> >
> > Please help me with the configuration, thanks.
>
> Is it running in cloud mode? If it is, then that is very
> straightforward to do. Probably the way I would do it is to create a
> collection alias pointing to both of the collections.
>
> If it's not in cloud mode you can still do it with the shards parameter.
>
> This is an example of the shards parameter from a Solr cluster I used to
> maintain, with the domain name changed to "example.com":
>
>
> idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
>
> It was a sharded index, but did not use cloud mode as that did not yet
> exist in a released Solr version when I built the cluster.
>
> Thanks,
> Shawn
>
Re: Multiple cores
Posted by Shawn Heisey <ap...@elyograg.org>.
On 12/27/22 18:40, Mike wrote:
> I have a Solr server with multiple cores with the same schema.
> For some queries I want to query several cores at the same time.
> I looked at the options and tried to configure what didn't work and that's
> how I ended up here.
>
> How can I query on query A -> Core1
> and at query B .-> Core1 and Core2
>
> Please help me with the configuration, thanks.
Is it running in cloud mode? If it is, then that is very
straightforward to do. Probably the way I would do it is to create a
collection alias pointing to both of the collections.
If it's not in cloud mode you can still do it with the shards parameter.
This is an example of the shards parameter from a Solr cluster I used to
maintain, with the domain name changed to "example.com":
idxb2.example.com:8981/solr/inclive,idxb1.example.com:8981/solr/s0live,idxb1.example.com:8981/solr/s1live,idxb1.example.com:8981/solr/s2live,idxb2.example.com:8981/solr/s3live,idxb2.example.com:8981/solr/s4live,idxb2.example.com:8981/solr/s5live
It was a sharded index, but did not use cloud mode as that did not yet
exist in a released Solr version when I built the cluster.
Thanks,
Shawn
Re: Multiple cores
Posted by Mike <mz...@gmail.com>.
Unfortunately it doesn't work completely yet, the results from shards1 are
ranking first.
I set <statsCache class="org.apache.solr.search.stats.ExactStatsCache"/> in
solrconfig.xml.
Do I need to do proper scoring across two shards?
Unfortunately it didn't help. Can anyone help me please?
Am Mi., 28. Dez. 2022 um 02:40 Uhr schrieb Mike <mz...@gmail.com>:
> Hello!
>
> I have a Solr server with multiple cores with the same schema.
> For some queries I want to query several cores at the same time.
> I looked at the options and tried to configure what didn't work and that's
> how I ended up here.
>
> How can I query on query A -> Core1
> and at query B .-> Core1 and Core2
>
> Please help me with the configuration, thanks.
>
> Mike
>