[jira] [Updated] (LUCENE-10007) CommonGramsQueryFilter: javadoc example does not produce output indicated

["Chris M. Hostetter (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-10007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris M. Hostetter updated LUCENE-10007:
----------------------------------------
    Attachment: LUCENE-10007.patch
        Status: Open  (was: Open)

patch with test case addition that matches the javadoc example (and some nocommit reminders to also edit the various javadocs to fix the dash/underscore discrepency...

{code}
    // example in javadocs
    assertAnalyzesTo(a, "the rain in spain falls mainly", new String[] { "the_rain", "rain_in" ,"in_spain", "falls", "mainly" });
{code}
{noformat}
org.apache.lucene.analysis.commongrams.TestCommonGramsFilter > testCommonGramsQueryFilter FAILED
    org.junit.ComparisonFailure: term 3 expected:<[falls]> but was:<[spain]>

{noformat}

> CommonGramsQueryFilter: javadoc example does not produce output indicated
> -------------------------------------------------------------------------
>
>                 Key: LUCENE-10007
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10007
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Chris M. Hostetter
>            Priority: Major
>         Attachments: LUCENE-10007.patch
>
>
> CommonGramsQueryFilter has the following explanation of it's behavior + example twice in it's javadocs (both at the class level and the incrementToken method level)..
> {noformat}
> /**
>  * Wrap a CommonGramsFilter optimizing phrase queries by only returning single words when they are
>  * not a member of a bigram.
>  *
>  * <p>Example:
>  *
>  * <ul>
>  *   <li>query input to CommonGramsFilter: "the rain in spain falls mainly"
>  *   <li>output of CommomGramsFilter/input to CommonGramsQueryFilter: |"the, "the-rain"|"rain"
>  *       "rain-in"|"in, "in-spain"|"spain"|"falls"|"mainly"
>  *   <li>output of CommonGramsQueryFilter:"the-rain", "rain-in" ,"in-spain", "falls", "mainly"
>  * </ul>
>  */
> {noformat}
> But this input doesn't actually produce the documented output: CommonGramsQueryFilter does in fact produce a token for "spain" even though it is part of the "in-spain" bigram.
> I'm not really sure which is "wrong" – the implementation or the documentation – but something seems to be out of whack.
>  ----
> _(a more trivial discrepancy is the use of "_" [underscore] in the CommomGramsFilter vs "-" [dash] in the javadoc example)_



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org