You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Trejkaz <tr...@trypticon.org> on 2016/01/15 01:52:27 UTC
Fwd: Replacement for Filter-as-abstract-class in Lucene 5.4?
Hi all.
Filter is now deprecated, which I already knew was in the pipeline.
The docs say:
"Use Query objects instead: when queries are wrapped in a
ConstantScoreQuery or in a BooleanClause.Occur.FILTER clause,
they automatically disable the score computation so the Filter
class does not provide benefits compared to queries anymore."
That's fair enough and an easy change to do on the caller side.
The other thing we are using Filter for is the other thing it mentions
in the Javadoc:
"Convenient base class for building queries that only perform
matching, but no scoring. The scorer produced by such queries
always returns 0 as score."
What is the new convenient way to implement your own queries that
don't do scoring?
TX
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Replacement for Filter-as-abstract-class in Lucene 5.4?
Posted by Trejkaz <tr...@trypticon.org>.
On Thu, Jan 12, 2017 at 1:02 PM, Kumaran Ramasubramanian
<ku...@gmail.com> wrote:
> I always use filter when i need to add more than 1024 ( for no scoring
> cases ). If filter is removed in lucene 6, what will happen to
> maxbooleanclauses limit? Am i missing anything?
That sounds like a BooleanQuery with FILTER clauses to me, a totally
different use case. I'm talking about using Filter as a base class to
implement queries where you already have the results as a set of some
sort without scoring.
TX
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Replacement for Filter-as-abstract-class in Lucene 5.4?
Posted by Kumaran Ramasubramanian <ku...@gmail.com>.
I always use filter when i need to add more than 1024 ( for no scoring
cases ). If filter is removed in lucene 6, what will happen to
maxbooleanclauses limit? Am i missing anything?
-
Kumaran R
On Jan 12, 2017 5:01 AM, "Trejkaz" <tr...@trypticon.org> wrote:
On Thu, Jan 21, 2016 at 4:25 AM, Adrien Grand <jp...@gmail.com> wrote:
> Uwe, maybe we could promote ConstantScoreWeight to an experimental API and
> document how to build simple queries based on it?
In the future now, looking at Lucene 6.3 Javadocs, where Filter is now
gone, and it seems that ConstantScoreWeight is still @lucene.internal
(and awfully hard to understand how it can do much at all...). Did we
ever get a replacement class for this use case for Filter? I read
something about solr taking a copy of the class over in its code,
which might be what we have to do here, but I wanted to check first.
TX
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Replacement for Filter-as-abstract-class in Lucene 5.4?
Posted by Trejkaz <tr...@trypticon.org>.
On Wed, Jan 18, 2017 at 6:07 AM, Adrien Grand <jp...@gmail.com> wrote:
>
> We are open to feedback, what issues are you having with
> ConstantScoreWeight? It is true that it does not bring much compared to
> Weight anymore now that we removed query normalization. The only useful
> thing it has is the default explain() implementation.
I guess I just don't have any good examples to follow for how to
implement it, and
Filter itself, for example, wasn't using it either. Plus there was additional
convenience in not having to make a Query subclass...
At the moment I have pulled solr's copy of it and rolled the code from
that up into
the next abstract class up which we had.
Actually, I'm looking at the direct subclasses of that as well, and it
seems like
there are two common cases:
1) Queries where we get the DocIdSet from some external source like a
database, which we could possibly switch to some kind of numeric values
/ point set query.
2) Queries where we do something like a TermsQuery but without keeping all
the terms in memory at the same time... which there might be another way
to do, but I'm not really sure.
TX
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Replacement for Filter-as-abstract-class in Lucene 5.4?
Posted by Adrien Grand <jp...@gmail.com>.
Le jeu. 12 janv. 2017 à 00:31, Trejkaz <tr...@trypticon.org> a écrit :
> In the future now, looking at Lucene 6.3 Javadocs, where Filter is now
> gone, and it seems that ConstantScoreWeight is still @lucene.internal
> (and awfully hard to understand how it can do much at all...). Did we
> ever get a replacement class for this use case for Filter? I read
> something about solr taking a copy of the class over in its code,
> which might be what we have to do here, but I wanted to check first.
>
We are open to feedback, what issues are you having with
ConstantScoreWeight? It is true that it does not bring much compared to
Weight anymore now that we removed query normalization. The only useful
thing it has is the default explain() implementation.
Regarding Filter, it was moved to Solr because Solr has strong interactions
between the Filter class and its filter cache, which were not easy to fix
at the same time as we wanted to remove Filter from Lucene. It should go
away eventually hopefully. One thing that made filters nice in the past is
that you could decide to optionally implement some Bits, but this is now
supported in queries too in a more flexible way (but harder to implement I
agree) through two-phase iteration: you can provide an approximation in
addition to the ability to check whether a document matches. Also, the
match cost API helps execute the costly bits last, which used to be an
issue with filters. For instance, a bitset-based filter should be
intersected in a leap-frog fashion with the query while a doc-values-based
filter should be executed last in a random-access fashion but the filter
API had no way to propagate that information. Solr initially added ways to
optimize it through the ExtendedQuery/PostFilter classes which allowed to
do filtering at the collector level but this should now be replaced with
two-phase iteration too in my opinion.
Re: Replacement for Filter-as-abstract-class in Lucene 5.4?
Posted by Trejkaz <tr...@trypticon.org>.
On Thu, Jan 21, 2016 at 4:25 AM, Adrien Grand <jp...@gmail.com> wrote:
> Uwe, maybe we could promote ConstantScoreWeight to an experimental API and
> document how to build simple queries based on it?
In the future now, looking at Lucene 6.3 Javadocs, where Filter is now
gone, and it seems that ConstantScoreWeight is still @lucene.internal
(and awfully hard to understand how it can do much at all...). Did we
ever get a replacement class for this use case for Filter? I read
something about solr taking a copy of the class over in its code,
which might be what we have to do here, but I wanted to check first.
TX
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Replacement for Filter-as-abstract-class in Lucene 5.4?
Posted by Adrien Grand <jp...@gmail.com>.
Uwe, maybe we could promote ConstantScoreWeight to an experimental API and
document how to build simple queries based on it?
Le ven. 15 janv. 2016 à 09:51, Christoph Kaser <lu...@iconparc.de> a
écrit :
> Nevermind, I missed the part about it being a base class for own queries.
>
> Sorry for the confusion!
>
> Am 15.01.2016 um 09:49 schrieb Christoph Kaser:
> > Isn't that what ConstantScoreQuery does? The only difference is that
> > it returns 1.0f as score instead of 0.0f.
> >
> > Regards
> > Christoph
> >
> > Am 15.01.2016 um 09:27 schrieb Uwe Schindler:
> >> I had the same problem while migrating old code. Filter is very
> >> convenient to use, so why is it deprecated? I agree we should convert
> >> all internal filters to use this, but people from the outside, that
> >> just quickly want to create a Filter based on simple stuff like
> >> bitsets, should get an easy api without multi-pass stuff or the need
> >> to implement a full scorer returning 0 as score.
> >>
> >> So I tend to rename the current filter class as BaseNonScoringQuery.
> >> In 5.x keep Filter deprecated (maybe simply as empty abstract
> >> subclass of this one).
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: uwe@thetaphi.de
> >>
> >>
> >>> -----Original Message-----
> >>> From: Trejkaz [mailto:trejkaz@trypticon.org]
> >>> Sent: Friday, January 15, 2016 1:52 AM
> >>> To: Lucene Users Mailing List <ja...@lucene.apache.org>
> >>> Subject: Fwd: Replacement for Filter-as-abstract-class in Lucene 5.4?
> >>>
> >>> Hi all.
> >>>
> >>> Filter is now deprecated, which I already knew was in the pipeline.
> >>>
> >>> The docs say:
> >>>
> >>> "Use Query objects instead: when queries are wrapped in a
> >>> ConstantScoreQuery or in a BooleanClause.Occur.FILTER clause,
> >>> they automatically disable the score computation so the Filter
> >>> class does not provide benefits compared to queries anymore."
> >>>
> >>> That's fair enough and an easy change to do on the caller side.
> >>>
> >>> The other thing we are using Filter for is the other thing it mentions
> >>> in the Javadoc:
> >>>
> >>> "Convenient base class for building queries that only perform
> >>> matching, but no scoring. The scorer produced by such queries
> >>> always returns 0 as score."
> >>>
> >>> What is the new convenient way to implement your own queries that
> >>> don't do scoring?
> >>>
> >>> TX
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Replacement for Filter-as-abstract-class in Lucene 5.4?
Posted by Christoph Kaser <lu...@iconparc.de>.
Nevermind, I missed the part about it being a base class for own queries.
Sorry for the confusion!
Am 15.01.2016 um 09:49 schrieb Christoph Kaser:
> Isn't that what ConstantScoreQuery does? The only difference is that
> it returns 1.0f as score instead of 0.0f.
>
> Regards
> Christoph
>
> Am 15.01.2016 um 09:27 schrieb Uwe Schindler:
>> I had the same problem while migrating old code. Filter is very
>> convenient to use, so why is it deprecated? I agree we should convert
>> all internal filters to use this, but people from the outside, that
>> just quickly want to create a Filter based on simple stuff like
>> bitsets, should get an easy api without multi-pass stuff or the need
>> to implement a full scorer returning 0 as score.
>>
>> So I tend to rename the current filter class as BaseNonScoringQuery.
>> In 5.x keep Filter deprecated (maybe simply as empty abstract
>> subclass of this one).
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>>> -----Original Message-----
>>> From: Trejkaz [mailto:trejkaz@trypticon.org]
>>> Sent: Friday, January 15, 2016 1:52 AM
>>> To: Lucene Users Mailing List <ja...@lucene.apache.org>
>>> Subject: Fwd: Replacement for Filter-as-abstract-class in Lucene 5.4?
>>>
>>> Hi all.
>>>
>>> Filter is now deprecated, which I already knew was in the pipeline.
>>>
>>> The docs say:
>>>
>>> "Use Query objects instead: when queries are wrapped in a
>>> ConstantScoreQuery or in a BooleanClause.Occur.FILTER clause,
>>> they automatically disable the score computation so the Filter
>>> class does not provide benefits compared to queries anymore."
>>>
>>> That's fair enough and an easy change to do on the caller side.
>>>
>>> The other thing we are using Filter for is the other thing it mentions
>>> in the Javadoc:
>>>
>>> "Convenient base class for building queries that only perform
>>> matching, but no scoring. The scorer produced by such queries
>>> always returns 0 as score."
>>>
>>> What is the new convenient way to implement your own queries that
>>> don't do scoring?
>>>
>>> TX
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Replacement for Filter-as-abstract-class in Lucene 5.4?
Posted by Christoph Kaser <lu...@iconparc.de>.
Isn't that what ConstantScoreQuery does? The only difference is that it
returns 1.0f as score instead of 0.0f.
Regards
Christoph
Am 15.01.2016 um 09:27 schrieb Uwe Schindler:
> I had the same problem while migrating old code. Filter is very convenient to use, so why is it deprecated? I agree we should convert all internal filters to use this, but people from the outside, that just quickly want to create a Filter based on simple stuff like bitsets, should get an easy api without multi-pass stuff or the need to implement a full scorer returning 0 as score.
>
> So I tend to rename the current filter class as BaseNonScoringQuery. In 5.x keep Filter deprecated (maybe simply as empty abstract subclass of this one).
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Trejkaz [mailto:trejkaz@trypticon.org]
>> Sent: Friday, January 15, 2016 1:52 AM
>> To: Lucene Users Mailing List <ja...@lucene.apache.org>
>> Subject: Fwd: Replacement for Filter-as-abstract-class in Lucene 5.4?
>>
>> Hi all.
>>
>> Filter is now deprecated, which I already knew was in the pipeline.
>>
>> The docs say:
>>
>> "Use Query objects instead: when queries are wrapped in a
>> ConstantScoreQuery or in a BooleanClause.Occur.FILTER clause,
>> they automatically disable the score computation so the Filter
>> class does not provide benefits compared to queries anymore."
>>
>> That's fair enough and an easy change to do on the caller side.
>>
>> The other thing we are using Filter for is the other thing it mentions
>> in the Javadoc:
>>
>> "Convenient base class for building queries that only perform
>> matching, but no scoring. The scorer produced by such queries
>> always returns 0 as score."
>>
>> What is the new convenient way to implement your own queries that
>> don't do scoring?
>>
>> TX
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Replacement for Filter-as-abstract-class in Lucene 5.4?
Posted by Uwe Schindler <uw...@thetaphi.de>.
I had the same problem while migrating old code. Filter is very convenient to use, so why is it deprecated? I agree we should convert all internal filters to use this, but people from the outside, that just quickly want to create a Filter based on simple stuff like bitsets, should get an easy api without multi-pass stuff or the need to implement a full scorer returning 0 as score.
So I tend to rename the current filter class as BaseNonScoringQuery. In 5.x keep Filter deprecated (maybe simply as empty abstract subclass of this one).
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Trejkaz [mailto:trejkaz@trypticon.org]
> Sent: Friday, January 15, 2016 1:52 AM
> To: Lucene Users Mailing List <ja...@lucene.apache.org>
> Subject: Fwd: Replacement for Filter-as-abstract-class in Lucene 5.4?
>
> Hi all.
>
> Filter is now deprecated, which I already knew was in the pipeline.
>
> The docs say:
>
> "Use Query objects instead: when queries are wrapped in a
> ConstantScoreQuery or in a BooleanClause.Occur.FILTER clause,
> they automatically disable the score computation so the Filter
> class does not provide benefits compared to queries anymore."
>
> That's fair enough and an easy change to do on the caller side.
>
> The other thing we are using Filter for is the other thing it mentions
> in the Javadoc:
>
> "Convenient base class for building queries that only perform
> matching, but no scoring. The scorer produced by such queries
> always returns 0 as score."
>
> What is the new convenient way to implement your own queries that
> don't do scoring?
>
> TX
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org