You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Stanislav Sandalnikov <s....@gmail.com> on 2017/12/14 08:58:26 UTC
Limit search queries only to pull replicas
Hi,
We have a Solr 7.1 setup with SolrCloud where we have multiple shards on one server (for indexing) each shard has a pull replica on other servers.
What are the possible ways to limit search request only to pull type replicase?
At the moment the only solution I found is to append shards parameter to each query, but if new shards added later it requires to change solrconfig. Is it the only way to do this?
Thank you
Regards
Stanislav
Re: Limit search queries only to pull replicas
Posted by Ere Maijala <er...@helsinki.fi>.
Interesting indeed, but maybe in line with the idea that ES knows what
to do best without the user interfering.
My example parameter name was bad, it should have been something like
"preferReplicaTypes=TLOG,PULL". I can't see what would be bad about
that, but then to me it seems Solr has always been much more about
giving control to the administrator or developer instead of
automatically just working. This may be daunting in the beginning, but
it seems I always start to look for more control of how things are done
in the long run.
--Ere
Emir Arnautović kirjoitti 5.1.2018 klo 17.51:
> It is interesting that ES had similar feature to prefer primary/replica but it deprecating that and will remove it - could not find explanation why.
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
>> On 5 Jan 2018, at 15:22, Ere Maijala <er...@helsinki.fi> wrote:
>>
>> Hi,
>>
>> It would be really nice to have a server-side option, though. Not everyone uses Solrj, and a typical fairly dummy client just queries the server without any understanding about shards etc. Solr could be clever enough to not forward the query to NRT shards when configured to prefer PULL shards and they're available. Maybe it could be something similar to the preferLocalShards parameter, like "preferShardTypes=TLOG,PULL".
>>
>> --Ere
>>
>> Emir Arnautović kirjoitti 14.12.2017 klo 11.41:
>>> Hi Stanislav,
>>> I don’t think that there is a built in feature to do this, but that sounds like nice feature of Solrj - maybe you should check if available. You can implement it outside of Solrj - check cluster state to see which shards are available and send queries only to pull replicas.
>>> HTH,
>>> Emir
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>> On 14 Dec 2017, at 09:58, Stanislav Sandalnikov <s....@gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> We have a Solr 7.1 setup with SolrCloud where we have multiple shards on one server (for indexing) each shard has a pull replica on other servers.
>>>>
>>>> What are the possible ways to limit search request only to pull type replicase?
>>>> At the moment the only solution I found is to append shards parameter to each query, but if new shards added later it requires to change solrconfig. Is it the only way to do this?
>>>>
>>>> Thank you
>>>>
>>>> Regards
>>>> Stanislav
>>>>
>>
>> --
>> Ere Maijala
>> Kansalliskirjasto / The National Library of Finland
>
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
Re: Limit search queries only to pull replicas
Posted by Stanislav Sandalnikov <s....@gmail.com>.
Hi,
The use case for this is that our indexing node has more shards than it has CPU cores it is enough for indexing, but not enough to serve the search queries if those queries are heavy. To put it out of serving requests we are using in-house solution that routes the queries to pull replicas based on information from zookeeper.
Ere, thanks for the patch, looking forward to try it.
Regards
Stanislav
> 14 февр. 2018 г., в 18:18, Ere Maijala <er...@helsinki.fi> написал(а):
>
> I've now posted https://issues.apache.org/jira/browse/SOLR-11982 with a patch. It works just like preferLocalShards. SOLR-10880 is awesome, but my idea is not to filter out anything, so this just adjusts the order of nodes.
>
> --Ere
>
> Tomas Fernandez Lobbe kirjoitti 8.1.2018 klo 21.42:
>> This feature is not currently supported. I was thinking in implementing it by extending the work done in SOLR-10880. I still didn’t have time to work on it though. There is a patch for SOLR-10880 that doesn’t implement support for replica types, but could be used as base.
>> Tomás
>>> On Jan 8, 2018, at 12:04 AM, Ere Maijala <er...@helsinki.fi> wrote:
>>>
>>> Server load alone doesn't always indicate the server's ability to serve queries. Memory and cache state are important too, and they're not as easy to monitor. Additionally, server load at any single point in time or a short term average is not indicative of the server's ability to handle search requests if indexing happens in short but intense bursts.
>>>
>>> It can also complicate things if there are more than one Solr instance running on a single server.
>>>
>>> I'm definitely not against intelligent routing. In many cases it makes perfect sense, and I'd still like to use it, just limited to the pull replicas.
>>>
>>> --Ere
>>>
>>> Erick Erickson kirjoitti 5.1.2018 klo 19.03:
>>>> Actually, I think a much better option is to route queries to server load.
>>>> The theory of preferring pull replicas to leaders would be that the leader
>>>> will be doing the indexing work and the pull replicas would be doing less
>>>> work therefore serving queries faster. But that's a fragile assumption.
>>>> Let's say indexing stops totally. Now your leader is sitting there idle
>>>> when it could be serving queries.
>>>> The autoscaling work will allow for more intelligent routing, you can
>>>> monitor the CPU load on your servers and if the leader has some spare
>>>> cycles use them .vs. crudely routing all queries to pull replicas (or tlog
>>>> replicas for that matter). NOTE: I don't know whether this is being
>>>> actively worked on or not, but seems a logical extension of the increased
>>>> monitoring capabilities being put in place for autoscaling, but I'd rather
>>>> see effort put in there than support routing based solely on a node's type.
>>>> Best,
>>>> Erick
>>>> On Fri, Jan 5, 2018 at 7:51 AM, Emir Arnautović <
>>>> emir.arnautovic@sematext.com> wrote:
>>>>> It is interesting that ES had similar feature to prefer primary/replica
>>>>> but it deprecating that and will remove it - could not find explanation why.
>>>>>
>>>>> Emir
>>>>> --
>>>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>>>
>>>>>
>>>>>
>>>>>> On 5 Jan 2018, at 15:22, Ere Maijala <er...@helsinki.fi> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> It would be really nice to have a server-side option, though. Not
>>>>> everyone uses Solrj, and a typical fairly dummy client just queries the
>>>>> server without any understanding about shards etc. Solr could be clever
>>>>> enough to not forward the query to NRT shards when configured to prefer
>>>>> PULL shards and they're available. Maybe it could be something similar to
>>>>> the preferLocalShards parameter, like "preferShardTypes=TLOG,PULL".
>>>>>>
>>>>>> --Ere
>>>>>>
>>>>>> Emir Arnautović kirjoitti 14.12.2017 klo 11.41:
>>>>>>> Hi Stanislav,
>>>>>>> I don’t think that there is a built in feature to do this, but that
>>>>> sounds like nice feature of Solrj - maybe you should check if available.
>>>>> You can implement it outside of Solrj - check cluster state to see which
>>>>> shards are available and send queries only to pull replicas.
>>>>>>> HTH,
>>>>>>> Emir
>>>>>>> --
>>>>>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>>>>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>>>>>> On 14 Dec 2017, at 09:58, Stanislav Sandalnikov <
>>>>> s.sandalnikov@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> We have a Solr 7.1 setup with SolrCloud where we have multiple shards
>>>>> on one server (for indexing) each shard has a pull replica on other servers.
>>>>>>>>
>>>>>>>> What are the possible ways to limit search request only to pull type
>>>>> replicase?
>>>>>>>> At the moment the only solution I found is to append shards parameter
>>>>> to each query, but if new shards added later it requires to change
>>>>> solrconfig. Is it the only way to do this?
>>>>>>>>
>>>>>>>> Thank you
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Stanislav
>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Ere Maijala
>>>>>> Kansalliskirjasto / The National Library of Finland
>>>>>
>>>>>
>>>
>>> --
>>> Ere Maijala
>>> Kansalliskirjasto / The National Library of Finland
>
> --
> Ere Maijala
> Kansalliskirjasto / The National Library of Finland
Re: Limit search queries only to pull replicas
Posted by Ere Maijala <er...@helsinki.fi>.
I've now posted https://issues.apache.org/jira/browse/SOLR-11982 with a
patch. It works just like preferLocalShards. SOLR-10880 is awesome, but
my idea is not to filter out anything, so this just adjusts the order of
nodes.
--Ere
Tomas Fernandez Lobbe kirjoitti 8.1.2018 klo 21.42:
> This feature is not currently supported. I was thinking in implementing it by extending the work done in SOLR-10880. I still didn’t have time to work on it though. There is a patch for SOLR-10880 that doesn’t implement support for replica types, but could be used as base.
>
> Tomás
>
>> On Jan 8, 2018, at 12:04 AM, Ere Maijala <er...@helsinki.fi> wrote:
>>
>> Server load alone doesn't always indicate the server's ability to serve queries. Memory and cache state are important too, and they're not as easy to monitor. Additionally, server load at any single point in time or a short term average is not indicative of the server's ability to handle search requests if indexing happens in short but intense bursts.
>>
>> It can also complicate things if there are more than one Solr instance running on a single server.
>>
>> I'm definitely not against intelligent routing. In many cases it makes perfect sense, and I'd still like to use it, just limited to the pull replicas.
>>
>> --Ere
>>
>> Erick Erickson kirjoitti 5.1.2018 klo 19.03:
>>> Actually, I think a much better option is to route queries to server load.
>>> The theory of preferring pull replicas to leaders would be that the leader
>>> will be doing the indexing work and the pull replicas would be doing less
>>> work therefore serving queries faster. But that's a fragile assumption.
>>> Let's say indexing stops totally. Now your leader is sitting there idle
>>> when it could be serving queries.
>>> The autoscaling work will allow for more intelligent routing, you can
>>> monitor the CPU load on your servers and if the leader has some spare
>>> cycles use them .vs. crudely routing all queries to pull replicas (or tlog
>>> replicas for that matter). NOTE: I don't know whether this is being
>>> actively worked on or not, but seems a logical extension of the increased
>>> monitoring capabilities being put in place for autoscaling, but I'd rather
>>> see effort put in there than support routing based solely on a node's type.
>>> Best,
>>> Erick
>>> On Fri, Jan 5, 2018 at 7:51 AM, Emir Arnautović <
>>> emir.arnautovic@sematext.com> wrote:
>>>> It is interesting that ES had similar feature to prefer primary/replica
>>>> but it deprecating that and will remove it - could not find explanation why.
>>>>
>>>> Emir
>>>> --
>>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>>
>>>>
>>>>
>>>>> On 5 Jan 2018, at 15:22, Ere Maijala <er...@helsinki.fi> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> It would be really nice to have a server-side option, though. Not
>>>> everyone uses Solrj, and a typical fairly dummy client just queries the
>>>> server without any understanding about shards etc. Solr could be clever
>>>> enough to not forward the query to NRT shards when configured to prefer
>>>> PULL shards and they're available. Maybe it could be something similar to
>>>> the preferLocalShards parameter, like "preferShardTypes=TLOG,PULL".
>>>>>
>>>>> --Ere
>>>>>
>>>>> Emir Arnautović kirjoitti 14.12.2017 klo 11.41:
>>>>>> Hi Stanislav,
>>>>>> I don’t think that there is a built in feature to do this, but that
>>>> sounds like nice feature of Solrj - maybe you should check if available.
>>>> You can implement it outside of Solrj - check cluster state to see which
>>>> shards are available and send queries only to pull replicas.
>>>>>> HTH,
>>>>>> Emir
>>>>>> --
>>>>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>>>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>>>>> On 14 Dec 2017, at 09:58, Stanislav Sandalnikov <
>>>> s.sandalnikov@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> We have a Solr 7.1 setup with SolrCloud where we have multiple shards
>>>> on one server (for indexing) each shard has a pull replica on other servers.
>>>>>>>
>>>>>>> What are the possible ways to limit search request only to pull type
>>>> replicase?
>>>>>>> At the moment the only solution I found is to append shards parameter
>>>> to each query, but if new shards added later it requires to change
>>>> solrconfig. Is it the only way to do this?
>>>>>>>
>>>>>>> Thank you
>>>>>>>
>>>>>>> Regards
>>>>>>> Stanislav
>>>>>>>
>>>>>
>>>>> --
>>>>> Ere Maijala
>>>>> Kansalliskirjasto / The National Library of Finland
>>>>
>>>>
>>
>> --
>> Ere Maijala
>> Kansalliskirjasto / The National Library of Finland
>
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
Re: Limit search queries only to pull replicas
Posted by Tomas Fernandez Lobbe <tf...@apple.com>.
This feature is not currently supported. I was thinking in implementing it by extending the work done in SOLR-10880. I still didn’t have time to work on it though. There is a patch for SOLR-10880 that doesn’t implement support for replica types, but could be used as base.
Tomás
> On Jan 8, 2018, at 12:04 AM, Ere Maijala <er...@helsinki.fi> wrote:
>
> Server load alone doesn't always indicate the server's ability to serve queries. Memory and cache state are important too, and they're not as easy to monitor. Additionally, server load at any single point in time or a short term average is not indicative of the server's ability to handle search requests if indexing happens in short but intense bursts.
>
> It can also complicate things if there are more than one Solr instance running on a single server.
>
> I'm definitely not against intelligent routing. In many cases it makes perfect sense, and I'd still like to use it, just limited to the pull replicas.
>
> --Ere
>
> Erick Erickson kirjoitti 5.1.2018 klo 19.03:
>> Actually, I think a much better option is to route queries to server load.
>> The theory of preferring pull replicas to leaders would be that the leader
>> will be doing the indexing work and the pull replicas would be doing less
>> work therefore serving queries faster. But that's a fragile assumption.
>> Let's say indexing stops totally. Now your leader is sitting there idle
>> when it could be serving queries.
>> The autoscaling work will allow for more intelligent routing, you can
>> monitor the CPU load on your servers and if the leader has some spare
>> cycles use them .vs. crudely routing all queries to pull replicas (or tlog
>> replicas for that matter). NOTE: I don't know whether this is being
>> actively worked on or not, but seems a logical extension of the increased
>> monitoring capabilities being put in place for autoscaling, but I'd rather
>> see effort put in there than support routing based solely on a node's type.
>> Best,
>> Erick
>> On Fri, Jan 5, 2018 at 7:51 AM, Emir Arnautović <
>> emir.arnautovic@sematext.com> wrote:
>>> It is interesting that ES had similar feature to prefer primary/replica
>>> but it deprecating that and will remove it - could not find explanation why.
>>>
>>> Emir
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>
>>>
>>>
>>>> On 5 Jan 2018, at 15:22, Ere Maijala <er...@helsinki.fi> wrote:
>>>>
>>>> Hi,
>>>>
>>>> It would be really nice to have a server-side option, though. Not
>>> everyone uses Solrj, and a typical fairly dummy client just queries the
>>> server without any understanding about shards etc. Solr could be clever
>>> enough to not forward the query to NRT shards when configured to prefer
>>> PULL shards and they're available. Maybe it could be something similar to
>>> the preferLocalShards parameter, like "preferShardTypes=TLOG,PULL".
>>>>
>>>> --Ere
>>>>
>>>> Emir Arnautović kirjoitti 14.12.2017 klo 11.41:
>>>>> Hi Stanislav,
>>>>> I don’t think that there is a built in feature to do this, but that
>>> sounds like nice feature of Solrj - maybe you should check if available.
>>> You can implement it outside of Solrj - check cluster state to see which
>>> shards are available and send queries only to pull replicas.
>>>>> HTH,
>>>>> Emir
>>>>> --
>>>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>>>> On 14 Dec 2017, at 09:58, Stanislav Sandalnikov <
>>> s.sandalnikov@gmail.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We have a Solr 7.1 setup with SolrCloud where we have multiple shards
>>> on one server (for indexing) each shard has a pull replica on other servers.
>>>>>>
>>>>>> What are the possible ways to limit search request only to pull type
>>> replicase?
>>>>>> At the moment the only solution I found is to append shards parameter
>>> to each query, but if new shards added later it requires to change
>>> solrconfig. Is it the only way to do this?
>>>>>>
>>>>>> Thank you
>>>>>>
>>>>>> Regards
>>>>>> Stanislav
>>>>>>
>>>>
>>>> --
>>>> Ere Maijala
>>>> Kansalliskirjasto / The National Library of Finland
>>>
>>>
>
> --
> Ere Maijala
> Kansalliskirjasto / The National Library of Finland
Re: Limit search queries only to pull replicas
Posted by Ere Maijala <er...@helsinki.fi>.
Server load alone doesn't always indicate the server's ability to serve
queries. Memory and cache state are important too, and they're not as
easy to monitor. Additionally, server load at any single point in time
or a short term average is not indicative of the server's ability to
handle search requests if indexing happens in short but intense bursts.
It can also complicate things if there are more than one Solr instance
running on a single server.
I'm definitely not against intelligent routing. In many cases it makes
perfect sense, and I'd still like to use it, just limited to the pull
replicas.
--Ere
Erick Erickson kirjoitti 5.1.2018 klo 19.03:
> Actually, I think a much better option is to route queries to server load.
>
> The theory of preferring pull replicas to leaders would be that the leader
> will be doing the indexing work and the pull replicas would be doing less
> work therefore serving queries faster. But that's a fragile assumption.
> Let's say indexing stops totally. Now your leader is sitting there idle
> when it could be serving queries.
>
> The autoscaling work will allow for more intelligent routing, you can
> monitor the CPU load on your servers and if the leader has some spare
> cycles use them .vs. crudely routing all queries to pull replicas (or tlog
> replicas for that matter). NOTE: I don't know whether this is being
> actively worked on or not, but seems a logical extension of the increased
> monitoring capabilities being put in place for autoscaling, but I'd rather
> see effort put in there than support routing based solely on a node's type.
>
> Best,
> Erick
>
> On Fri, Jan 5, 2018 at 7:51 AM, Emir Arnautović <
> emir.arnautovic@sematext.com> wrote:
>
>> It is interesting that ES had similar feature to prefer primary/replica
>> but it deprecating that and will remove it - could not find explanation why.
>>
>> Emir
>> --
>> Monitoring - Log Management - Alerting - Anomaly Detection
>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>
>>
>>
>>> On 5 Jan 2018, at 15:22, Ere Maijala <er...@helsinki.fi> wrote:
>>>
>>> Hi,
>>>
>>> It would be really nice to have a server-side option, though. Not
>> everyone uses Solrj, and a typical fairly dummy client just queries the
>> server without any understanding about shards etc. Solr could be clever
>> enough to not forward the query to NRT shards when configured to prefer
>> PULL shards and they're available. Maybe it could be something similar to
>> the preferLocalShards parameter, like "preferShardTypes=TLOG,PULL".
>>>
>>> --Ere
>>>
>>> Emir Arnautović kirjoitti 14.12.2017 klo 11.41:
>>>> Hi Stanislav,
>>>> I don’t think that there is a built in feature to do this, but that
>> sounds like nice feature of Solrj - maybe you should check if available.
>> You can implement it outside of Solrj - check cluster state to see which
>> shards are available and send queries only to pull replicas.
>>>> HTH,
>>>> Emir
>>>> --
>>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>>> On 14 Dec 2017, at 09:58, Stanislav Sandalnikov <
>> s.sandalnikov@gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> We have a Solr 7.1 setup with SolrCloud where we have multiple shards
>> on one server (for indexing) each shard has a pull replica on other servers.
>>>>>
>>>>> What are the possible ways to limit search request only to pull type
>> replicase?
>>>>> At the moment the only solution I found is to append shards parameter
>> to each query, but if new shards added later it requires to change
>> solrconfig. Is it the only way to do this?
>>>>>
>>>>> Thank you
>>>>>
>>>>> Regards
>>>>> Stanislav
>>>>>
>>>
>>> --
>>> Ere Maijala
>>> Kansalliskirjasto / The National Library of Finland
>>
>>
>
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
Re: Limit search queries only to pull replicas
Posted by Erick Erickson <er...@gmail.com>.
Actually, I think a much better option is to route queries to server load.
The theory of preferring pull replicas to leaders would be that the leader
will be doing the indexing work and the pull replicas would be doing less
work therefore serving queries faster. But that's a fragile assumption.
Let's say indexing stops totally. Now your leader is sitting there idle
when it could be serving queries.
The autoscaling work will allow for more intelligent routing, you can
monitor the CPU load on your servers and if the leader has some spare
cycles use them .vs. crudely routing all queries to pull replicas (or tlog
replicas for that matter). NOTE: I don't know whether this is being
actively worked on or not, but seems a logical extension of the increased
monitoring capabilities being put in place for autoscaling, but I'd rather
see effort put in there than support routing based solely on a node's type.
Best,
Erick
On Fri, Jan 5, 2018 at 7:51 AM, Emir Arnautović <
emir.arnautovic@sematext.com> wrote:
> It is interesting that ES had similar feature to prefer primary/replica
> but it deprecating that and will remove it - could not find explanation why.
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
> > On 5 Jan 2018, at 15:22, Ere Maijala <er...@helsinki.fi> wrote:
> >
> > Hi,
> >
> > It would be really nice to have a server-side option, though. Not
> everyone uses Solrj, and a typical fairly dummy client just queries the
> server without any understanding about shards etc. Solr could be clever
> enough to not forward the query to NRT shards when configured to prefer
> PULL shards and they're available. Maybe it could be something similar to
> the preferLocalShards parameter, like "preferShardTypes=TLOG,PULL".
> >
> > --Ere
> >
> > Emir Arnautović kirjoitti 14.12.2017 klo 11.41:
> >> Hi Stanislav,
> >> I don’t think that there is a built in feature to do this, but that
> sounds like nice feature of Solrj - maybe you should check if available.
> You can implement it outside of Solrj - check cluster state to see which
> shards are available and send queries only to pull replicas.
> >> HTH,
> >> Emir
> >> --
> >> Monitoring - Log Management - Alerting - Anomaly Detection
> >> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> >>> On 14 Dec 2017, at 09:58, Stanislav Sandalnikov <
> s.sandalnikov@gmail.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> We have a Solr 7.1 setup with SolrCloud where we have multiple shards
> on one server (for indexing) each shard has a pull replica on other servers.
> >>>
> >>> What are the possible ways to limit search request only to pull type
> replicase?
> >>> At the moment the only solution I found is to append shards parameter
> to each query, but if new shards added later it requires to change
> solrconfig. Is it the only way to do this?
> >>>
> >>> Thank you
> >>>
> >>> Regards
> >>> Stanislav
> >>>
> >
> > --
> > Ere Maijala
> > Kansalliskirjasto / The National Library of Finland
>
>
Re: Limit search queries only to pull replicas
Posted by Emir Arnautović <em...@sematext.com>.
It is interesting that ES had similar feature to prefer primary/replica but it deprecating that and will remove it - could not find explanation why.
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 5 Jan 2018, at 15:22, Ere Maijala <er...@helsinki.fi> wrote:
>
> Hi,
>
> It would be really nice to have a server-side option, though. Not everyone uses Solrj, and a typical fairly dummy client just queries the server without any understanding about shards etc. Solr could be clever enough to not forward the query to NRT shards when configured to prefer PULL shards and they're available. Maybe it could be something similar to the preferLocalShards parameter, like "preferShardTypes=TLOG,PULL".
>
> --Ere
>
> Emir Arnautović kirjoitti 14.12.2017 klo 11.41:
>> Hi Stanislav,
>> I don’t think that there is a built in feature to do this, but that sounds like nice feature of Solrj - maybe you should check if available. You can implement it outside of Solrj - check cluster state to see which shards are available and send queries only to pull replicas.
>> HTH,
>> Emir
>> --
>> Monitoring - Log Management - Alerting - Anomaly Detection
>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>> On 14 Dec 2017, at 09:58, Stanislav Sandalnikov <s....@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> We have a Solr 7.1 setup with SolrCloud where we have multiple shards on one server (for indexing) each shard has a pull replica on other servers.
>>>
>>> What are the possible ways to limit search request only to pull type replicase?
>>> At the moment the only solution I found is to append shards parameter to each query, but if new shards added later it requires to change solrconfig. Is it the only way to do this?
>>>
>>> Thank you
>>>
>>> Regards
>>> Stanislav
>>>
>
> --
> Ere Maijala
> Kansalliskirjasto / The National Library of Finland
Re: Limit search queries only to pull replicas
Posted by Ere Maijala <er...@helsinki.fi>.
Hi,
It would be really nice to have a server-side option, though. Not
everyone uses Solrj, and a typical fairly dummy client just queries the
server without any understanding about shards etc. Solr could be clever
enough to not forward the query to NRT shards when configured to prefer
PULL shards and they're available. Maybe it could be something similar
to the preferLocalShards parameter, like "preferShardTypes=TLOG,PULL".
--Ere
Emir Arnautović kirjoitti 14.12.2017 klo 11.41:
> Hi Stanislav,
> I don’t think that there is a built in feature to do this, but that sounds like nice feature of Solrj - maybe you should check if available. You can implement it outside of Solrj - check cluster state to see which shards are available and send queries only to pull replicas.
>
> HTH,
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
>> On 14 Dec 2017, at 09:58, Stanislav Sandalnikov <s....@gmail.com> wrote:
>>
>> Hi,
>>
>> We have a Solr 7.1 setup with SolrCloud where we have multiple shards on one server (for indexing) each shard has a pull replica on other servers.
>>
>> What are the possible ways to limit search request only to pull type replicase?
>> At the moment the only solution I found is to append shards parameter to each query, but if new shards added later it requires to change solrconfig. Is it the only way to do this?
>>
>> Thank you
>>
>> Regards
>> Stanislav
>>
>
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
Re: Limit search queries only to pull replicas
Posted by Emir Arnautović <em...@sematext.com>.
Hi Stanislav,
I don’t think that there is a built in feature to do this, but that sounds like nice feature of Solrj - maybe you should check if available. You can implement it outside of Solrj - check cluster state to see which shards are available and send queries only to pull replicas.
HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 14 Dec 2017, at 09:58, Stanislav Sandalnikov <s....@gmail.com> wrote:
>
> Hi,
>
> We have a Solr 7.1 setup with SolrCloud where we have multiple shards on one server (for indexing) each shard has a pull replica on other servers.
>
> What are the possible ways to limit search request only to pull type replicase?
> At the moment the only solution I found is to append shards parameter to each query, but if new shards added later it requires to change solrconfig. Is it the only way to do this?
>
> Thank you
>
> Regards
> Stanislav
>