You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by dinesh naik <di...@gmail.com> on 2021/01/05 15:26:46 UTC
maxBooleanClauses change in solr.xml not reflecting in solr 8.4.1
Hi all,
I want to update the maxBooleanClauses to 2048 (from default value 1024).
Below are the steps tried:
1. updated solrconfig.xml :
<maxBooleanClauses>${solr.max.booleanClauses:2048}</maxBooleanClauses>
2. updated solr.xml :
<int name="maxBooleanClauses">${solr.max.booleanClauses:2048}</int>
3. Restarted the solr nodes.
4. Tried query with more than 2000 OR clauses and getting below waring
message in solr logs:
2021-01-05 14:03:59.603 WARN (qtp1545077099-27) x:col1_shard1_replica_n3
o.a.s.c.SolrConfig solrconfig.xml: <maxBooleanClauses> of 2048 is greater
than global limit of 1024 and will have no effect
2021-01-05 14:03:59.603 WARN (qtp1545077099-27) x:col1_shard1_replica_n3
o.a.s.c.SolrConfig set 'maxBooleanClauses' in solr.xml to increase global
limit
Note: In 7.6.1 version we just need to change the solrconfig.xml and it
works.
Kindly let me know if i am missing something for making it work in 8.4.1
version.
--
Best Regards,
Dinesh Naik
Re: maxBooleanClauses change in solr.xml not reflecting in solr 8.4.1
Posted by dinesh naik <di...@gmail.com>.
Thanks Hoss,
Yes, i was making the change in solr.xml in wrong directory earlier.
Also as you said:
: You need to update EVERY solrconfig.xml that the JVM is loading for this
to
: actually work.
that has not been true for a while, see SOLR-13336 / SOLR-10921 ...
I validated this and it's working as expected. We don't need to update
every solrconfig.xml.
The value mentioned in solr.xml is global and if maxBooleanClauses for any
collection in solrconfig.xml exceeds the limit specified in solr.xml then
we get the exception.
Thanks for replying.
On Wed, Jan 6, 2021 at 10:57 PM dinesh naik <di...@gmail.com>
wrote:
> Thanks Shawn,
>
> This entry <int name="maxBooleanClauses">${sol
> r.max.booleanClauses:2048}</int> in solr.xml was introduced only in solr
> 8.x version and were not present in 7.6 version.
>
> We have this in solrconfig.xml in 8.4.1 version.
> <!-- Maximum number of clauses allowed when parsing a boolean query
> string. This limit only impacts boolean queries specified by a user as part
> of a query string, and provides per-collection controls on how complex user
> specified boolean queries can be. Query strings that specify more clauses
> then this will result in an error. If this per-collection limit is greater
> then the global `maxBooleanClauses` limit specified in `solr.xml`, it will
> have no effect, as that setting also limits the size of user specified
> boolean queries. --> <maxBooleanClauses>${solr.max.booleanClauses:2048}</
> maxBooleanClauses>
> i was updating the solr.xml in the installation directory and not the
> installed data directory, hence the change was not reflecting.
> After updating the correct solr.xml and restarting the Solr nodes the new
> value is working as expected.
>
> On Wed, Jan 6, 2021 at 10:34 PM Chris Hostetter <ho...@fucit.org>
> wrote:
>
>>
>> : You need to update EVERY solrconfig.xml that the JVM is loading for
>> this to
>> : actually work.
>>
>> that has not been true for a while, see SOLR-13336 / SOLR-10921 ...
>>
>> : > 2. updated solr.xml :
>> : > <int name="maxBooleanClauses">${solr.max.booleanClauses:2048}</int>
>> :
>> : I don't think it's currently possible to set the value with solr.xml.
>>
>> Not only is it possible, it's neccessary -- the value in solr.xml acts as
>> a hard upper limit (and affects all queries, even internally expanded
>> queries) on the "soft limit" in solrconfig.xml (that only affects
>> explicitly supplied boolean queries from users)
>>
>> As to the original question...
>>
>> > 2021-01-05 14:03:59.603 WARN (qtp1545077099-27)
>> x:col1_shard1_replica_n3
>> > o.a.s.c.SolrConfig solrconfig.xml: <maxBooleanClauses> of 2048 is
>> greater
>> > than global limit of 1024 and will have no effect
>>
>> I attempted to reproduce this with 8.4.1 and did not see the probem you
>> are describing.
>>
>> Are you 100% certain you are updating the correct solr.xml file? If you
>> add some non-xml giberish to the solr.xml you are editing does the solr
>> node fail to start up?
>>
>> Remember that when using SolrCloud, solr will try to load solr.xml from
>> zk
>> first, and only look on local disk if it can't be found in ZK ... look
>> for
>> log messages like "solr.xml found in ZooKeeper. Loading..." vs "Loading
>> solr.xml from SolrHome (not found in ZooKeeper)"
>>
>>
>>
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>
>
> --
> Best Regards,
> Dinesh Naik
>
--
Best Regards,
Dinesh Naik
Re: maxBooleanClauses change in solr.xml not reflecting in solr 8.4.1
Posted by dinesh naik <di...@gmail.com>.
Thanks Shawn,
This entry <int name="maxBooleanClauses">${solr.max.booleanClauses:2048}</
int> in solr.xml was introduced only in solr 8.x version and were not
present in 7.6 version.
We have this in solrconfig.xml in 8.4.1 version.
<!-- Maximum number of clauses allowed when parsing a boolean query string.
This limit only impacts boolean queries specified by a user as part of a
query string, and provides per-collection controls on how complex user
specified boolean queries can be. Query strings that specify more clauses
then this will result in an error. If this per-collection limit is greater
then the global `maxBooleanClauses` limit specified in `solr.xml`, it will
have no effect, as that setting also limits the size of user specified
boolean queries. --> <maxBooleanClauses>${solr.max.booleanClauses:2048}</
maxBooleanClauses>
i was updating the solr.xml in the installation directory and not the
installed data directory, hence the change was not reflecting.
After updating the correct solr.xml and restarting the Solr nodes the new
value is working as expected.
On Wed, Jan 6, 2021 at 10:34 PM Chris Hostetter <ho...@fucit.org>
wrote:
>
> : You need to update EVERY solrconfig.xml that the JVM is loading for this
> to
> : actually work.
>
> that has not been true for a while, see SOLR-13336 / SOLR-10921 ...
>
> : > 2. updated solr.xml :
> : > <int name="maxBooleanClauses">${solr.max.booleanClauses:2048}</int>
> :
> : I don't think it's currently possible to set the value with solr.xml.
>
> Not only is it possible, it's neccessary -- the value in solr.xml acts as
> a hard upper limit (and affects all queries, even internally expanded
> queries) on the "soft limit" in solrconfig.xml (that only affects
> explicitly supplied boolean queries from users)
>
> As to the original question...
>
> > 2021-01-05 14:03:59.603 WARN (qtp1545077099-27) x:col1_shard1_replica_n3
> > o.a.s.c.SolrConfig solrconfig.xml: <maxBooleanClauses> of 2048 is greater
> > than global limit of 1024 and will have no effect
>
> I attempted to reproduce this with 8.4.1 and did not see the probem you
> are describing.
>
> Are you 100% certain you are updating the correct solr.xml file? If you
> add some non-xml giberish to the solr.xml you are editing does the solr
> node fail to start up?
>
> Remember that when using SolrCloud, solr will try to load solr.xml from zk
> first, and only look on local disk if it can't be found in ZK ... look for
> log messages like "solr.xml found in ZooKeeper. Loading..." vs "Loading
> solr.xml from SolrHome (not found in ZooKeeper)"
>
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
--
Best Regards,
Dinesh Naik
Re: maxBooleanClauses change in solr.xml not reflecting in solr
8.4.1
Posted by Chris Hostetter <ho...@fucit.org>.
: You need to update EVERY solrconfig.xml that the JVM is loading for this to
: actually work.
that has not been true for a while, see SOLR-13336 / SOLR-10921 ...
: > 2. updated solr.xml :
: > <int name="maxBooleanClauses">${solr.max.booleanClauses:2048}</int>
:
: I don't think it's currently possible to set the value with solr.xml.
Not only is it possible, it's neccessary -- the value in solr.xml acts as
a hard upper limit (and affects all queries, even internally expanded
queries) on the "soft limit" in solrconfig.xml (that only affects
explicitly supplied boolean queries from users)
As to the original question...
> 2021-01-05 14:03:59.603 WARN (qtp1545077099-27) x:col1_shard1_replica_n3
> o.a.s.c.SolrConfig solrconfig.xml: <maxBooleanClauses> of 2048 is greater
> than global limit of 1024 and will have no effect
I attempted to reproduce this with 8.4.1 and did not see the probem you
are describing.
Are you 100% certain you are updating the correct solr.xml file? If you
add some non-xml giberish to the solr.xml you are editing does the solr
node fail to start up?
Remember that when using SolrCloud, solr will try to load solr.xml from zk
first, and only look on local disk if it can't be found in ZK ... look for
log messages like "solr.xml found in ZooKeeper. Loading..." vs "Loading
solr.xml from SolrHome (not found in ZooKeeper)"
-Hoss
http://www.lucidworks.com/
Re: maxBooleanClauses change in solr.xml not reflecting in solr 8.4.1
Posted by Shawn Heisey <ap...@elyograg.org>.
On 1/5/2021 8:26 AM, dinesh naik wrote:
> Hi all,
> I want to update the maxBooleanClauses to 2048 (from default value 1024).
> Below are the steps tried:
> 1. updated solrconfig.xml :
> <maxBooleanClauses>${solr.max.booleanClauses:2048}</maxBooleanClauses>
You need to update EVERY solrconfig.xml that the JVM is loading for this
to actually work.
maxBooleanClauses is an odd duck. At the Lucene level, where this
matters, it is a global (JVM-wide) variable. So whenever Solr sets this
value, it applies to ALL of the Lucene indexes that are being accessed
by that JVM.
When you havet multiple Solr cores, the last core that was loaded will
set the max clauses value for ALL cores. If any of your solrconfig.xml
files don't have that config, then it will be set to the default of 1024
when that core is loaded or reloaded. Leaving the config out is not a
solution.
So if any of your configs don't have the setting or set it to something
lower than you need, you run the risk of having the max value
incorrectly set across the board.
Here are the ways that I think this could be fixed:
1) Make the value per-index in Lucene, (or maybe even per-query) instead
of global.
2) Have Solr only change the global Lucene value if the config is
*higher* than the current global value.
3) Eliminate the limit entirely. Remove the config option from Solr and
have Solr hard-set it to the maximum value.
4) Move the maxBooleanClauses config to solr.xml instead of solrconfig.xml
I think that option 1 is the best way to do it, but this problem has
been around for many years, so it's probably not easy to do. I don't
think it's going to happen. There are a number of existing issues in
the Solr bug tracker for changing how the limit is configured.
> 2. updated solr.xml :
> <int name="maxBooleanClauses">${solr.max.booleanClauses:2048}</int>
I don't think it's currently possible to set the value with solr.xml.
Thanks,
Shawn
Re: maxBooleanClauses change in solr.xml not reflecting in solr 8.4.1
Posted by ANNAMANENI RAVEENDRA <a....@gmail.com>.
I experienced the same thing in solr-8.7 , it worked for me using system
property.
Set system property in solr.in.sh file
On Tue, Jan 5, 2021 at 8:58 PM dinesh naik <di...@gmail.com>
wrote:
> Hi all,
> I want to update the maxBooleanClauses to 2048 (from default value 1024).
> Below are the steps tried:
> 1. updated solrconfig.xml :
> <maxBooleanClauses>${solr.max.booleanClauses:2048}</maxBooleanClauses>
>
> 2. updated solr.xml :
> <int name="maxBooleanClauses">${solr.max.booleanClauses:2048}</int>
>
> 3. Restarted the solr nodes.
>
> 4. Tried query with more than 2000 OR clauses and getting below waring
> message in solr logs:
>
> 2021-01-05 14:03:59.603 WARN (qtp1545077099-27) x:col1_shard1_replica_n3
> o.a.s.c.SolrConfig solrconfig.xml: <maxBooleanClauses> of 2048 is greater
> than global limit of 1024 and will have no effect
>
> 2021-01-05 14:03:59.603 WARN (qtp1545077099-27) x:col1_shard1_replica_n3
> o.a.s.c.SolrConfig set 'maxBooleanClauses' in solr.xml to increase global
> limit
>
> Note: In 7.6.1 version we just need to change the solrconfig.xml and it
> works.
>
> Kindly let me know if i am missing something for making it work in 8.4.1
> version.
> --
> Best Regards,
> Dinesh Naik
>