You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by rgummadi <ra...@gmail.com> on 2019/06/18 09:57:19 UTC
Delete with Solrj deleteByQuery - Boolean clauses
There is a situation where we have to delete a lot of assets for a customer
from solr index. There are occasions where the number of assets run into
thousands. So I am constructing the query as below. If the number of ‘OR’
clauses cross a certain limit (like 50), the delete is not working.
We are using the SolrJ for this (solrServer.deleteByQuery(stb.toString())).
Also the MaxBooleanClause parameter is set to 3000 in solrconfig.xml.
Why is the query failing even though the number of clauses is below 3000. Is
there a better way of doing this.
CustomerCode:BACU1 AND ( Barcode:(000006743476636 OR 000006743482288 OR
000006881406272 OR 000006881406315 OR 000006881406343 OR 000006881406383 OR
000006881406432 OR 000006857852700 OR 000006857852756 OR 000006857852783 OR
000006857852740 OR 000006857852768 OR 000006857852801 OR 000006857852810 OR
000006857852819 OR 000006857852828 OR 000006857852844 OR 000006857852859 OR
000006857852873 OR 000006857852887 OR 000006857852894 OR 000006857852904 OR
000006851761567 OR 000006851761572 OR 000006851761573 OR 000006851761575 OR
000006851761576 OR 000006851761577 OR 000006851761578 OR 000006851761580 OR
000006851761582 OR 000006851761583 OR 000006852758372 OR 000006852758354 OR
000006852758338 OR 000006852758318 OR 000006852758276 OR 000006857789194 OR
000006857789198 OR 000006857789204 OR 000006857789200 OR 000006857789220 OR
000006856559248 OR 000006856559309 OR 000006743482260 OR 000006743482254 OR
000006743482294 OR 000006743482232 OR 000006743478792 OR 000006743482235 OR
000006743482250 OR 000006743482259 OR 000006743482262 OR 000006743482256 OR
000006743482237 OR 000006743478796 OR 000006743482018 OR 000006743482033 OR
000006743482063 OR 000006743496937 OR 000006743482044 OR 000006743481998 OR
000006743482066 OR 000006743482070 OR 000006743482037 OR 000006743482001 OR
000006743482004 OR 000006743482021 OR 000006743482056 OR 000006743495615 OR
000006743482007 OR 000006743482028 OR 000006743482047 OR 000006743496939 OR
000006743482009 OR 000006743482038 OR 000006743496736 OR 000006743495617 OR
000006743482025 OR 000006743482057 OR 000006743482049 OR 000006743482075 OR
000006743482051 OR 000006743482031 OR 000006743496737 OR 000006743496419 OR
000006743495619 OR 000006743482059 OR 000006743482041 OR 000006743482060 OR
000006743482062 OR 000006743482012 OR 000006743482053 OR 000006743495620 OR
000006743496738 OR 000006743496940 OR 000006743482081 OR 000006743482104 OR
000006743482130 OR 000006743482121 OR 000006743482107 OR 000006743482094 OR
000006743495622 OR 000006743482136 OR 000006743482096 OR 000006743482078 OR
000006743482079 OR 000006743482085 OR 000006743495624 OR 000006743482140 OR
000006743482141 OR 000006743482086 OR 000006743482123 OR 000006838138542 OR
000006743495626 OR 000006743495627 OR 000006743495628 OR 000006743495630 OR
000006743482126 OR 000006743482089 OR 000006743482133 OR 000006743482100 OR
000006743482110 OR 000006743495631 OR 000006743495632 OR 000006743482134 OR
000006743495634 OR 000006743495635 OR 000006743482137 OR 000006743495636 OR
000006743495638 OR 000006743482128 OR 000006743482102 OR 000006743482116 OR
000006743482091 OR 000006743482119 OR 000006743495640 OR 000006743497254 OR
000006743496421 OR 000006743496486 OR 000006743496487 OR 000006743496488 OR
000006743496489 OR 000006743496422 OR 000006743497255 OR 000006743496491 OR
000006743496423 OR 000006743497256 OR 000006743496424 OR 000006743496492 OR
000006743496494 OR 000006743496425))
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Delete with Solrj deleteByQuery - Boolean clauses
Posted by rgummadi <ra...@gmail.com>.
Thanks Erick. I will try the terms query.
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Delete with Solrj deleteByQuery - Boolean clauses
Posted by Shawn Heisey <ap...@elyograg.org>.
On 6/19/2019 8:33 AM, rgummadi wrote:
> Still using Solr 4.6. Terms query parser does not exist in this version
> right?
Correct. It was added in version 4.10.0.
https://issues.apache.org/jira/browse/SOLR-6318
Thanks,
Shawn
Re: Delete with Solrj deleteByQuery - Boolean clauses
Posted by rgummadi <ra...@gmail.com>.
Hi Erick,
Still using Solr 4.6. Terms query parser does not exist in this version
right?
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Delete with Solrj deleteByQuery - Boolean clauses
Posted by Erick Erickson <er...@gmail.com>.
If these are the id field (i.e. <uniqueKey>), then delete by id is
much less painful. That aside:
1> check how the query is parsed with just one or two Barcodes. If
you are pushing this through edismax or similar, you might be getting
surprising results
2> try putting that massive OR clause inside a Terms query parser, It’d
look something like:
CustomerCode:BACU1 AND _query_:”{!terms f=barcode}0001,0002,0003…..”
Best,
Erick
> On Jun 18, 2019, at 2:57 AM, rgummadi <ra...@gmail.com> wrote:
>
> There is a situation where we have to delete a lot of assets for a customer
> from solr index. There are occasions where the number of assets run into
> thousands. So I am constructing the query as below. If the number of ‘OR’
> clauses cross a certain limit (like 50), the delete is not working.
>
> We are using the SolrJ for this (solrServer.deleteByQuery(stb.toString())).
>
> Also the MaxBooleanClause parameter is set to 3000 in solrconfig.xml.
>
> Why is the query failing even though the number of clauses is below 3000. Is
> there a better way of doing this.
>
> CustomerCode:BACU1 AND ( Barcode:(000006743476636 OR 000006743482288 OR
> 000006881406272 OR 000006881406315 OR 000006881406343 OR 000006881406383 OR
> 000006881406432 OR 000006857852700 OR 000006857852756 OR 000006857852783 OR
> 000006857852740 OR 000006857852768 OR 000006857852801 OR 000006857852810 OR
> 000006857852819 OR 000006857852828 OR 000006857852844 OR 000006857852859 OR
> 000006857852873 OR 000006857852887 OR 000006857852894 OR 000006857852904 OR
> 000006851761567 OR 000006851761572 OR 000006851761573 OR 000006851761575 OR
> 000006851761576 OR 000006851761577 OR 000006851761578 OR 000006851761580 OR
> 000006851761582 OR 000006851761583 OR 000006852758372 OR 000006852758354 OR
> 000006852758338 OR 000006852758318 OR 000006852758276 OR 000006857789194 OR
> 000006857789198 OR 000006857789204 OR 000006857789200 OR 000006857789220 OR
> 000006856559248 OR 000006856559309 OR 000006743482260 OR 000006743482254 OR
> 000006743482294 OR 000006743482232 OR 000006743478792 OR 000006743482235 OR
> 000006743482250 OR 000006743482259 OR 000006743482262 OR 000006743482256 OR
> 000006743482237 OR 000006743478796 OR 000006743482018 OR 000006743482033 OR
> 000006743482063 OR 000006743496937 OR 000006743482044 OR 000006743481998 OR
> 000006743482066 OR 000006743482070 OR 000006743482037 OR 000006743482001 OR
> 000006743482004 OR 000006743482021 OR 000006743482056 OR 000006743495615 OR
> 000006743482007 OR 000006743482028 OR 000006743482047 OR 000006743496939 OR
> 000006743482009 OR 000006743482038 OR 000006743496736 OR 000006743495617 OR
> 000006743482025 OR 000006743482057 OR 000006743482049 OR 000006743482075 OR
> 000006743482051 OR 000006743482031 OR 000006743496737 OR 000006743496419 OR
> 000006743495619 OR 000006743482059 OR 000006743482041 OR 000006743482060 OR
> 000006743482062 OR 000006743482012 OR 000006743482053 OR 000006743495620 OR
> 000006743496738 OR 000006743496940 OR 000006743482081 OR 000006743482104 OR
> 000006743482130 OR 000006743482121 OR 000006743482107 OR 000006743482094 OR
> 000006743495622 OR 000006743482136 OR 000006743482096 OR 000006743482078 OR
> 000006743482079 OR 000006743482085 OR 000006743495624 OR 000006743482140 OR
> 000006743482141 OR 000006743482086 OR 000006743482123 OR 000006838138542 OR
> 000006743495626 OR 000006743495627 OR 000006743495628 OR 000006743495630 OR
> 000006743482126 OR 000006743482089 OR 000006743482133 OR 000006743482100 OR
> 000006743482110 OR 000006743495631 OR 000006743495632 OR 000006743482134 OR
> 000006743495634 OR 000006743495635 OR 000006743482137 OR 000006743495636 OR
> 000006743495638 OR 000006743482128 OR 000006743482102 OR 000006743482116 OR
> 000006743482091 OR 000006743482119 OR 000006743495640 OR 000006743497254 OR
> 000006743496421 OR 000006743496486 OR 000006743496487 OR 000006743496488 OR
> 000006743496489 OR 000006743496422 OR 000006743497255 OR 000006743496491 OR
> 000006743496423 OR 000006743497256 OR 000006743496424 OR 000006743496492 OR
> 000006743496494 OR 000006743496425))
>
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html