You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shawn Heisey (Created) (JIRA)" <ji...@apache.org> on 2012/04/06 14:57:23 UTC
[jira] [Created] (SOLR-3333) Create an option that allows a query
to be cached, but not used for warming
Create an option that allows a query to be cached, but not used for warming
---------------------------------------------------------------------------
Key: SOLR-3333
URL: https://issues.apache.org/jira/browse/SOLR-3333
Project: Solr
Issue Type: New Feature
Affects Versions: 3.5, 4.0
Reporter: Shawn Heisey
The application that uses my Solr install builds complex filter queries for employees because they have access to everything, whereas most users have access to a small subset.
Because of this, autowarming on the filterCache can take 30-60 seconds even though autoWarm is set to just 4 queries.
If we had a way (probably a localparam) to tell Solr to not use those filters when autowarming, but to go ahead and put them in the filterCache and use them until there's a new commit, that would eliminate this problem. Employees might have their queries take longer, but regular users would not be affected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3333) Create an option that allows a query
to be cached, but not used for warming
Posted by "Shawn Heisey (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254664#comment-13254664 ]
Shawn Heisey commented on SOLR-3333:
------------------------------------
I just thought of a localparam syntax for this: {!cache=nowarm}
> Create an option that allows a query to be cached, but not used for warming
> ---------------------------------------------------------------------------
>
> Key: SOLR-3333
> URL: https://issues.apache.org/jira/browse/SOLR-3333
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 3.5, 4.0
> Reporter: Shawn Heisey
>
> The application that uses my Solr install builds complex filter queries for employees because they have access to everything, whereas most users have access to a small subset.
> Because of this, autowarming on the filterCache can take 30-60 seconds even though autoWarm is set to just 4 queries.
> If we had a way (probably a localparam) to tell Solr to not use those filters when autowarming, but to go ahead and put them in the filterCache and use them until there's a new commit, that would eliminate this problem. Employees might have their queries take longer, but regular users would not be affected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3333) Create an option that allows a query
to be cached, but not used for warming
Posted by "Shawn Heisey (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248579#comment-13248579 ]
Shawn Heisey commented on SOLR-3333:
------------------------------------
I would like to have our application code tag those nasty employee filters with something that makes them ineligible for autowarming, but still eligible for caching, which would keep them around until the next commit. I am pretty sure our code is capable of knowing that the user is a special user, typically admin or system.
An update cycle runs once a minute for the index as a whole, but changes are tracked on a per-shard basis. Commits on each shard are only done if something on that particular shard actually changes. The large shards where this is a problem typically go several minutes between commits, and that might extend to an hour or more.
I will talk to our developers about using the cache=false localparam for now, but I am hoping for the ability to use the cache for those nasty filters but not include them for warming. Having recently toyed with the cache code (SOLR-2906), I know this may not be trivial.
> Create an option that allows a query to be cached, but not used for warming
> ---------------------------------------------------------------------------
>
> Key: SOLR-3333
> URL: https://issues.apache.org/jira/browse/SOLR-3333
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 3.5, 4.0
> Reporter: Shawn Heisey
>
> The application that uses my Solr install builds complex filter queries for employees because they have access to everything, whereas most users have access to a small subset.
> Because of this, autowarming on the filterCache can take 30-60 seconds even though autoWarm is set to just 4 queries.
> If we had a way (probably a localparam) to tell Solr to not use those filters when autowarming, but to go ahead and put them in the filterCache and use them until there's a new commit, that would eliminate this problem. Employees might have their queries take longer, but regular users would not be affected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3333) Create an option that allows a query
to be cached, but not used for warming
Posted by "Shawn Heisey (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248602#comment-13248602 ]
Shawn Heisey commented on SOLR-3333:
------------------------------------
I never actually answered your first question. Yes, I do want most entries in the filter cache to be usable for autowarming. Most users have relatively few boolean clauses in their filter queries. Employees are the common exception. We get a few hundred boolean clauses in ours. Plans are being discussed to greatly reduce that, but I'm not sure we'll ever get away from it entirely.
> Create an option that allows a query to be cached, but not used for warming
> ---------------------------------------------------------------------------
>
> Key: SOLR-3333
> URL: https://issues.apache.org/jira/browse/SOLR-3333
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 3.5, 4.0
> Reporter: Shawn Heisey
>
> The application that uses my Solr install builds complex filter queries for employees because they have access to everything, whereas most users have access to a small subset.
> Because of this, autowarming on the filterCache can take 30-60 seconds even though autoWarm is set to just 4 queries.
> If we had a way (probably a localparam) to tell Solr to not use those filters when autowarming, but to go ahead and put them in the filterCache and use them until there's a new commit, that would eliminate this problem. Employees might have their queries take longer, but regular users would not be affected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3333) Create an option that allows a query
to be cached, but not used for warming
Posted by "Erick Erickson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248385#comment-13248385 ]
Erick Erickson commented on SOLR-3333:
--------------------------------------
Are there other auto-warming queries you want to have done? Because it almost sounds like you just want to turn off autowarming in the filter cache.
Or if they're unlikely to be re-used anyway, would it work to set cache=false on the original fq?
> Create an option that allows a query to be cached, but not used for warming
> ---------------------------------------------------------------------------
>
> Key: SOLR-3333
> URL: https://issues.apache.org/jira/browse/SOLR-3333
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 3.5, 4.0
> Reporter: Shawn Heisey
>
> The application that uses my Solr install builds complex filter queries for employees because they have access to everything, whereas most users have access to a small subset.
> Because of this, autowarming on the filterCache can take 30-60 seconds even though autoWarm is set to just 4 queries.
> If we had a way (probably a localparam) to tell Solr to not use those filters when autowarming, but to go ahead and put them in the filterCache and use them until there's a new commit, that would eliminate this problem. Employees might have their queries take longer, but regular users would not be affected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3333) Create an option that allows a query
to be cached, but not used for warming
Posted by "Shawn Heisey (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248292#comment-13248292 ]
Shawn Heisey commented on SOLR-3333:
------------------------------------
I don't think I can implement this. My knowledge of Solr internals simply isn't strong enough.
> Create an option that allows a query to be cached, but not used for warming
> ---------------------------------------------------------------------------
>
> Key: SOLR-3333
> URL: https://issues.apache.org/jira/browse/SOLR-3333
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 3.5, 4.0
> Reporter: Shawn Heisey
>
> The application that uses my Solr install builds complex filter queries for employees because they have access to everything, whereas most users have access to a small subset.
> Because of this, autowarming on the filterCache can take 30-60 seconds even though autoWarm is set to just 4 queries.
> If we had a way (probably a localparam) to tell Solr to not use those filters when autowarming, but to go ahead and put them in the filterCache and use them until there's a new commit, that would eliminate this problem. Employees might have their queries take longer, but regular users would not be affected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org