You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Andres de la Peña <ad...@stratio.com> on 2016/08/02 17:07:30 UTC
Re: Filter strategy in Lucene 6.0
Thanks Adrien, this is very helpful.
I have just read your blog post about this
<https://www.elastic.co/blog/better-query-execution-coming-elasticsearch-2-0>,
the two-phase iteration is really cool!
2016-07-31 20:17 GMT+01:00 Adrien Grand <jp...@gmail.com>:
> Lucene 5.0 introduced two-phase iteration
> (see Scorer.twoPhaseIterator()) as a way to tackle slow queries. In short,
> queries can be split into a fast approximation and a slower confirmation
> and Lucene makes sure to reach agreement between the different
> approximations of a BooleanQuery before checking the confirmations.
>
> Both approximations and confirmations have an API that allows them to
> expose a "cost" and the least costly components are always evaluated before
> the most costly ones. So the filter strategy is not useful anymore, every
> decision is made based on these cost APIs.
>
>
> Le ven. 22 juil. 2016 à 15:32, Andres de la Peña <ad...@stratio.com> a
> écrit :
>
> > Hi all,
> >
> > Suppose that we have a boolean query composed by two filtering queries,
> > where one of them is fast and the other is slow:
> >
> > BooleanQuery.Builder builder = new BooleanQuery.Builder();
> > builder.add(fastQuery, FILTER);
> > builder.add(slowQuery, FILTER);
> > Query query = builder.build();
> >
> >
> > How is the intersection between the two sub-queries calculated? Is the
> > order in which they are added to the boolean query relevant? Is there
> > something we could do to take advantage of my knowledge about the
> expected
> > performance of the sub-queries?
> >
> > Prior to Lucene 6.0 there was a FilteredQuery.FilterStrategy providing
> some
> > control about this. Is there something analogous in Lucene 6.0?
> >
> > Thanks in advance,
> >
> > --
> > Andrés de la Peña
> >
> > Vía de las dos Castillas, 33, Ática 4, 3ª Planta
> > 28224 Pozuelo de Alarcón, Madrid
> > Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
> > <https://twitter.com/StratioBD>*
> >
>
--
Andrés de la Peña
Vía de las dos Castillas, 33, Ática 4, 3ª Planta
28224 Pozuelo de Alarcón, Madrid
Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
<https://twitter.com/StratioBD>*
Re: Filter strategy in Lucene 6.0
Posted by Adrien Grand <jp...@gmail.com>.
You can read about the inception of the feature at
https://issues.apache.org/jira/browse/LUCENE-6198 and since two-phase
iteration is mostly useful for conjunctions, you could look at
ConjunctionDISI which is the class that takes care of intersecting multiple
iterators. I am afraid there is not much besides that.
Le mer. 3 août 2016 à 09:42, Parit Bansal <Pa...@sib.swiss> a écrit :
> Hi,
>
> Could you point to some resource where I can read about two-phase
> iterators in slightly more depth? There are still confusions for me as
> to how exactly it works.
>
> - Best
> Parit
>
> On 08/02/2016 07:07 PM, Andres de la Peña wrote:
> > Thanks Adrien, this is very helpful.
> >
> > I have just read your blog post about this
> > <
> https://www.elastic.co/blog/better-query-execution-coming-elasticsearch-2-0
> >,
> > the two-phase iteration is really cool!
> >
> > 2016-07-31 20:17 GMT+01:00 Adrien Grand <jp...@gmail.com>:
> >
> >> Lucene 5.0 introduced two-phase iteration
> >> (see Scorer.twoPhaseIterator()) as a way to tackle slow queries. In
> short,
> >> queries can be split into a fast approximation and a slower confirmation
> >> and Lucene makes sure to reach agreement between the different
> >> approximations of a BooleanQuery before checking the confirmations.
> >>
> >> Both approximations and confirmations have an API that allows them to
> >> expose a "cost" and the least costly components are always evaluated
> before
> >> the most costly ones. So the filter strategy is not useful anymore,
> every
> >> decision is made based on these cost APIs.
> >>
> >>
> >> Le ven. 22 juil. 2016 à 15:32, Andres de la Peña <ad...@stratio.com>
> a
> >> écrit :
> >>
> >>> Hi all,
> >>>
> >>> Suppose that we have a boolean query composed by two filtering queries,
> >>> where one of them is fast and the other is slow:
> >>>
> >>> BooleanQuery.Builder builder = new BooleanQuery.Builder();
> >>> builder.add(fastQuery, FILTER);
> >>> builder.add(slowQuery, FILTER);
> >>> Query query = builder.build();
> >>>
> >>>
> >>> How is the intersection between the two sub-queries calculated? Is the
> >>> order in which they are added to the boolean query relevant? Is there
> >>> something we could do to take advantage of my knowledge about the
> >> expected
> >>> performance of the sub-queries?
> >>>
> >>> Prior to Lucene 6.0 there was a FilteredQuery.FilterStrategy providing
> >> some
> >>> control about this. Is there something analogous in Lucene 6.0?
> >>>
> >>> Thanks in advance,
> >>>
> >>> --
> >>> Andrés de la Peña
> >>>
> >>> Vía de las dos Castillas, 33, Ática 4, 3ª Planta
> >>> 28224 Pozuelo de Alarcón, Madrid
> >>> Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
> >>> <https://twitter.com/StratioBD>*
> >>>
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Filter strategy in Lucene 6.0
Posted by Parit Bansal <Pa...@sib.swiss>.
Hi,
Could you point to some resource where I can read about two-phase
iterators in slightly more depth? There are still confusions for me as
to how exactly it works.
- Best
Parit
On 08/02/2016 07:07 PM, Andres de la Pe�a wrote:
> Thanks Adrien, this is very helpful.
>
> I have just read your blog post about this
> <https://www.elastic.co/blog/better-query-execution-coming-elasticsearch-2-0>,
> the two-phase iteration is really cool!
>
> 2016-07-31 20:17 GMT+01:00 Adrien Grand <jp...@gmail.com>:
>
>> Lucene 5.0 introduced two-phase iteration
>> (see Scorer.twoPhaseIterator()) as a way to tackle slow queries. In short,
>> queries can be split into a fast approximation and a slower confirmation
>> and Lucene makes sure to reach agreement between the different
>> approximations of a BooleanQuery before checking the confirmations.
>>
>> Both approximations and confirmations have an API that allows them to
>> expose a "cost" and the least costly components are always evaluated before
>> the most costly ones. So the filter strategy is not useful anymore, every
>> decision is made based on these cost APIs.
>>
>>
>> Le ven. 22 juil. 2016 � 15:32, Andres de la Pe�a <ad...@stratio.com> a
>> �crit :
>>
>>> Hi all,
>>>
>>> Suppose that we have a boolean query composed by two filtering queries,
>>> where one of them is fast and the other is slow:
>>>
>>> BooleanQuery.Builder builder = new BooleanQuery.Builder();
>>> builder.add(fastQuery, FILTER);
>>> builder.add(slowQuery, FILTER);
>>> Query query = builder.build();
>>>
>>>
>>> How is the intersection between the two sub-queries calculated? Is the
>>> order in which they are added to the boolean query relevant? Is there
>>> something we could do to take advantage of my knowledge about the
>> expected
>>> performance of the sub-queries?
>>>
>>> Prior to Lucene 6.0 there was a FilteredQuery.FilterStrategy providing
>> some
>>> control about this. Is there something analogous in Lucene 6.0?
>>>
>>> Thanks in advance,
>>>
>>> --
>>> Andr�s de la Pe�a
>>>
>>> V�a de las dos Castillas, 33, �tica 4, 3� Planta
>>> 28224 Pozuelo de Alarc�n, Madrid
>>> Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
>>> <https://twitter.com/StratioBD>*
>>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org