You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Sanjay Dutt (Jira)" <ji...@apache.org> on 2024/02/03 11:24:00 UTC
[jira] [Comment Edited] (SOLR-17148) Config API lies about setting cache enabled=false
[ https://issues.apache.org/jira/browse/SOLR-17148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17813912#comment-17813912 ]
Sanjay Dutt edited comment on SOLR-17148 at 2/3/24 11:23 AM:
-------------------------------------------------------------
Thank you for sharing all the info [~hossman] !
I looked into this issue and below is the method that being called to initialized the cache objects (documentCache, queryResultCache e.t.c)
{code:java}
public static CacheConfig getConfig(SolrConfig solrConfig, ConfigNode node, String xpath) {
if (!node.exists() || !"true".equals(node.attributes().get("enabled", "true"))) {
Map<String, Object> m = solrConfig.getOverlay().getEditableSubProperties(xpath);
if (m == null) return null;
List<String> parts = StrUtils.splitSmart(xpath, '/');
return getConfig(solrConfig, parts.get(parts.size() - 1), Collections.emptyMap(), xpath);
}
return getConfig(solrConfig, node.name(), node.attributes().asMap(), xpath);
}{code}
Now we disable the cache using below command and configoverlay.json file gets updated as you mentioned.
{code:java}
curl -X POST 'http://localhost:8983/solr/gettingstarted/config/' -H'Content-type: application/json' -d '{"set-property":{"query.documentCache.enabled":false}}'{code}
As per the investigation, the issue present in the "IF" condition,
{code:java}
if (!node.exists() || !"true".equals(node.attributes().get("enabled", "true"))) {{code}
The node object passing to getConfig is an instance of OverlaidConfigNode (extends ConfigNode), a special object containing parent properties(from SolrConfig.xml) and new properties(from overlay), Only If there are new properties present in overlay file.
But even after disabling the cache, *node.attributes().get("enabled", "true")* is always returning the default value "true" because *node.attributes()* searching for property only in parent properties (from Solrconfig.xml) and not in overlay properties and therefore it is not able to disable the cache. Because to do that, they have to see the enabled flag with a false value.
Now we can change above If condition to below updated code where we are calling *node.attr* method which would first look for enabled flag in overlay and then delegate to the parent properties.
{code}
var enabled = node.attr("enabled") == null ? "true": node.attr("enabled");
if (!node.exists() || !"true".equals(enabled)) {
{code}
But I believe there is one more issue,If we look at the getConfig method again. Then once the code entered into the IF block, It will fetch properties from the overlay file and If it's not null then they would initialize Cache object with overlay properties and some default properties, even if the cache is disabled.
IMO, The fix would be to look for enabled flag in overlay and If it's false then return null.
was (Author: duttsanjay):
Thank you for sharing all the info [~hossman] !
I looked into this issue and below is the method that being called to initialized the cache objects (documentCache, queryResultCache e.t.c)
{code:java}
public static CacheConfig getConfig(SolrConfig solrConfig, ConfigNode node, String xpath) {
if (!node.exists() || !"true".equals(node.attributes().get("enabled", "true"))) {
Map<String, Object> m = solrConfig.getOverlay().getEditableSubProperties(xpath);
if (m == null) return null;
List<String> parts = StrUtils.splitSmart(xpath, '/');
return getConfig(solrConfig, parts.get(parts.size() - 1), Collections.emptyMap(), xpath);
}
return getConfig(solrConfig, node.name(), node.attributes().asMap(), xpath);
}{code}
Now we disable the cache using below command and configoverlay.json file gets updated as you mentioned.
{code:java}
curl -X POST 'http://localhost:8983/solr/gettingstarted/config/' -H'Content-type: application/json' -d '{"set-property":{"query.documentCache.enabled":false}}'{code}
As per the investigation, the issue present in the "IF" condition,
{code:java}
if (!node.exists() || !"true".equals(node.attributes().get("enabled", "true"))) {{code}
The node object passing to getConfig is an instance of OverlaidConfigNode (extends ConfigNode), a special object containing parent properties(from SolrConfig.xml) and new properties(from overlay), Only If there are new properties present in overlay file.
But even after disabling the cache, *node.attributes().get("enabled", "true")* is always returning the default value "true" because *node.attributes()* searching for property only in parent properties (from Solrconfig.xml) and not in overlay properties and therefore it is not able to disable the cache. Because to do that, they have to see the enabled flag with a false value.
Now we can change above If condition to below updated code where we are calling *node.attr* method which would first look for enabled flag in overlay and then delegate to the parent properties.
{code:keyword}
var enabled = node.attr("enabled") == null ? "true": node.attr("enabled");
if (!node.exists() || !"true".equals(enabled)) {
{code}
But I believe there is one more issue,If we look at the getConfig method again. Then once the code entered into the IF block, It will fetch properties from the overlay file and If it's not null then they would initialize Cache object with overlay properties and some default properties, even if the cache is disabled.
IMO, The fix would be to look for enabled flag in overlay and If it's false then return null.
> Config API lies about setting cache enabled=false
> -------------------------------------------------
>
> Key: SOLR-17148
> URL: https://issues.apache.org/jira/browse/SOLR-17148
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: config-api
> Reporter: Chris M. Hostetter
> Priority: Major
>
> Using the stock solr {{gettingstarted}} example, you can run the command below and it will give all evidence that it has disabled the document cache:
> {noformat}
> curl -X POST 'http://localhost:8983/solr/gettingstarted/config/' -H'Content-type: application/json' -d '{"set-property":{"query.documentCache.enabled":false}}'
> {noformat}
> * Api command succeeds
> * {{configoverlay.json}} is updated in ZK
> * SolrCore reloads happen for all of the replicas of this collection
> ...but doesn't actually disable the cache. It's still there and in use (easy to verify by executing queries and looking at metrics)
> Explicitly stopping & restarting the solr nodes doesn't fix the problem either.
> They {{enabled}} property name appears to be explicitly allowed by the Config API, because trying to set other arbitrary properties fails (even when supported by the {{CaffieneCache}} impl)...
> {noformat}
> $ curl -X POST 'http://localhost:8983/solr/gettingstarted/config/' -H'Content-type: application/json' -d '{"set-property":{"query.documentCache.async":false}}'
> {
> "responseHeader":{
> "status":400,
> "QTime":8},
> "errorMessages":["error processing commands, errors: [{errorMessages=['query.documentCache.async' is not an editable property], set-property={query.documentCache.async=false}}], \n"],
> {noformat}
> ...suggesting that modifying the {{enabled}} property is intentionally supported – but doesn't work.
>
> ----
>
> A quick & dirty workaround is setting the size=-1 and ignoring the ERROR in the logs on every newSearcher...
> {noformat}
> curl -X POST 'http://localhost:8983/solr/gettingstarted/config/' -H'Content-type: application/json' -d '{"set-property":{"query.documentCache.size":-1}}'
> {noformat}
> (Which seems to indicate that whatever the problem is, it's specific to {{enabled}} – and not a general problem with modifying cache configuration via the Config API)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org