You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "sivaganesh (Created) (JIRA)" <ji...@apache.org> on 2011/11/19 10:16:51 UTC
[jira] [Created] (SOLR-2908) To push the terms.limit parameter from
the master core to all the shard cores.
To push the terms.limit parameter from the master core to all the shard cores.
------------------------------------------------------------------------------
Key: SOLR-2908
URL: https://issues.apache.org/jira/browse/SOLR-2908
Project: Solr
Issue Type: Improvement
Components: SearchComponents - other
Affects Versions: 1.4.1
Environment: Linux server. 64 bit processor and 16GB Ram.
Reporter: sivaganesh
Priority: Critical
Fix For: 1.4.1
When we pass the terms.limit parameter to the master (which has many shard cores), it's not getting pushed down to the individual cores. Instead the default value of -1 is assigned to Terms.limit parameter is assigned in the underlying shard cores. The issue being the time taken by the Master core to return the required limit of terms is higher when we are having more number of underlying shard cores. This affects the performances of the auto suggest feature.
Can thought we can have a parameter to explicitly override the -1 being set to Terms.limit in shards core.
We saw the source code(TermsComponent.java) and concluded that the same. Please help us in pushing the terms.limit parameter to shard cores.
PFB code snippet.
private ShardRequest createShardQuery(SolrParams params) {
ShardRequest sreq = new ShardRequest();
sreq.purpose = ShardRequest.PURPOSE_GET_TERMS;
// base shard request on original parameters
sreq.params = new ModifiableSolrParams(params);
// remove any limits for shards, we want them to return all possible
// responses
// we want this so we can calculate the correct counts
// dont sort by count to avoid that unnecessary overhead on the shards
sreq.params.remove(TermsParams.TERMS_MAXCOUNT);
sreq.params.remove(TermsParams.TERMS_MINCOUNT);
sreq.params.set(TermsParams.TERMS_LIMIT, -1);
sreq.params.set(TermsParams.TERMS_SORT, TermsParams.TERMS_SORT_INDEX);
return sreq;
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2908) To push the terms.limit parameter from
the master core to all the shard cores.
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-2908:
------------------------------
Fix Version/s: (was: 1.4.1)
5.0
4.1
> To push the terms.limit parameter from the master core to all the shard cores.
> ------------------------------------------------------------------------------
>
> Key: SOLR-2908
> URL: https://issues.apache.org/jira/browse/SOLR-2908
> Project: Solr
> Issue Type: Improvement
> Components: SearchComponents - other
> Affects Versions: 1.4.1
> Environment: Linux server. 64 bit processor and 16GB Ram.
> Reporter: sivaganesh
> Priority: Critical
> Labels: patch
> Fix For: 4.1, 5.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> When we pass the terms.limit parameter to the master (which has many shard cores), it's not getting pushed down to the individual cores. Instead the default value of -1 is assigned to Terms.limit parameter is assigned in the underlying shard cores. The issue being the time taken by the Master core to return the required limit of terms is higher when we are having more number of underlying shard cores. This affects the performances of the auto suggest feature.
> Can thought we can have a parameter to explicitly override the -1 being set to Terms.limit in shards core.
> We saw the source code(TermsComponent.java) and concluded that the same. Please help us in pushing the terms.limit parameter to shard cores.
> PFB code snippet.
> private ShardRequest createShardQuery(SolrParams params) {
> ShardRequest sreq = new ShardRequest();
> sreq.purpose = ShardRequest.PURPOSE_GET_TERMS;
> // base shard request on original parameters
> sreq.params = new ModifiableSolrParams(params);
> // remove any limits for shards, we want them to return all possible
> // responses
> // we want this so we can calculate the correct counts
> // dont sort by count to avoid that unnecessary overhead on the shards
> sreq.params.remove(TermsParams.TERMS_MAXCOUNT);
> sreq.params.remove(TermsParams.TERMS_MINCOUNT);
> sreq.params.set(TermsParams.TERMS_LIMIT, -1);
> sreq.params.set(TermsParams.TERMS_SORT, TermsParams.TERMS_SORT_INDEX);
> return sreq;
> }
> Solr Version:
> Solr Specification Version: 1.4.0.2010.01.13.08.09.44
> Solr Implementation Version: 1.5-dev exported - yonik - 2010-01-13 08:09:44
> Lucene Specification Version: 2.9.1-dev
> Lucene Implementation Version: 2.9.1-dev 888785 - 2009-12-09 18:03:31
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2908) To push the terms.limit parameter from
the master core to all the shard cores.
Posted by "sivaganesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
sivaganesh updated SOLR-2908:
-----------------------------
Description:
When we pass the terms.limit parameter to the master (which has many shard cores), it's not getting pushed down to the individual cores. Instead the default value of -1 is assigned to Terms.limit parameter is assigned in the underlying shard cores. The issue being the time taken by the Master core to return the required limit of terms is higher when we are having more number of underlying shard cores. This affects the performances of the auto suggest feature.
Can thought we can have a parameter to explicitly override the -1 being set to Terms.limit in shards core.
We saw the source code(TermsComponent.java) and concluded that the same. Please help us in pushing the terms.limit parameter to shard cores.
PFB code snippet.
private ShardRequest createShardQuery(SolrParams params) {
ShardRequest sreq = new ShardRequest();
sreq.purpose = ShardRequest.PURPOSE_GET_TERMS;
// base shard request on original parameters
sreq.params = new ModifiableSolrParams(params);
// remove any limits for shards, we want them to return all possible
// responses
// we want this so we can calculate the correct counts
// dont sort by count to avoid that unnecessary overhead on the shards
sreq.params.remove(TermsParams.TERMS_MAXCOUNT);
sreq.params.remove(TermsParams.TERMS_MINCOUNT);
sreq.params.set(TermsParams.TERMS_LIMIT, -1);
sreq.params.set(TermsParams.TERMS_SORT, TermsParams.TERMS_SORT_INDEX);
return sreq;
}
Solr Version:
Solr Specification Version: 1.4.0.2010.01.13.08.09.44
Solr Implementation Version: 1.5-dev exported - yonik - 2010-01-13 08:09:44
Lucene Specification Version: 2.9.1-dev
Lucene Implementation Version: 2.9.1-dev 888785 - 2009-12-09 18:03:31
was:
When we pass the terms.limit parameter to the master (which has many shard cores), it's not getting pushed down to the individual cores. Instead the default value of -1 is assigned to Terms.limit parameter is assigned in the underlying shard cores. The issue being the time taken by the Master core to return the required limit of terms is higher when we are having more number of underlying shard cores. This affects the performances of the auto suggest feature.
Can thought we can have a parameter to explicitly override the -1 being set to Terms.limit in shards core.
We saw the source code(TermsComponent.java) and concluded that the same. Please help us in pushing the terms.limit parameter to shard cores.
PFB code snippet.
private ShardRequest createShardQuery(SolrParams params) {
ShardRequest sreq = new ShardRequest();
sreq.purpose = ShardRequest.PURPOSE_GET_TERMS;
// base shard request on original parameters
sreq.params = new ModifiableSolrParams(params);
// remove any limits for shards, we want them to return all possible
// responses
// we want this so we can calculate the correct counts
// dont sort by count to avoid that unnecessary overhead on the shards
sreq.params.remove(TermsParams.TERMS_MAXCOUNT);
sreq.params.remove(TermsParams.TERMS_MINCOUNT);
sreq.params.set(TermsParams.TERMS_LIMIT, -1);
sreq.params.set(TermsParams.TERMS_SORT, TermsParams.TERMS_SORT_INDEX);
return sreq;
}
> To push the terms.limit parameter from the master core to all the shard cores.
> ------------------------------------------------------------------------------
>
> Key: SOLR-2908
> URL: https://issues.apache.org/jira/browse/SOLR-2908
> Project: Solr
> Issue Type: Improvement
> Components: SearchComponents - other
> Affects Versions: 1.4.1
> Environment: Linux server. 64 bit processor and 16GB Ram.
> Reporter: sivaganesh
> Priority: Critical
> Labels: patch
> Fix For: 1.4.1
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> When we pass the terms.limit parameter to the master (which has many shard cores), it's not getting pushed down to the individual cores. Instead the default value of -1 is assigned to Terms.limit parameter is assigned in the underlying shard cores. The issue being the time taken by the Master core to return the required limit of terms is higher when we are having more number of underlying shard cores. This affects the performances of the auto suggest feature.
> Can thought we can have a parameter to explicitly override the -1 being set to Terms.limit in shards core.
> We saw the source code(TermsComponent.java) and concluded that the same. Please help us in pushing the terms.limit parameter to shard cores.
> PFB code snippet.
> private ShardRequest createShardQuery(SolrParams params) {
> ShardRequest sreq = new ShardRequest();
> sreq.purpose = ShardRequest.PURPOSE_GET_TERMS;
> // base shard request on original parameters
> sreq.params = new ModifiableSolrParams(params);
> // remove any limits for shards, we want them to return all possible
> // responses
> // we want this so we can calculate the correct counts
> // dont sort by count to avoid that unnecessary overhead on the shards
> sreq.params.remove(TermsParams.TERMS_MAXCOUNT);
> sreq.params.remove(TermsParams.TERMS_MINCOUNT);
> sreq.params.set(TermsParams.TERMS_LIMIT, -1);
> sreq.params.set(TermsParams.TERMS_SORT, TermsParams.TERMS_SORT_INDEX);
> return sreq;
> }
> Solr Version:
> Solr Specification Version: 1.4.0.2010.01.13.08.09.44
> Solr Implementation Version: 1.5-dev exported - yonik - 2010-01-13 08:09:44
> Lucene Specification Version: 2.9.1-dev
> Lucene Implementation Version: 2.9.1-dev 888785 - 2009-12-09 18:03:31
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org