CommonGramsQuery and proximity searches issue

[Sjoerd Smeets <ss...@gmail.com>]

Hi,

We are using a CommonGrams filter for both indexing and querying. We
provide a list of words that should be treated as a common gram. In the
list the following words exist:

- new
- york

When we do queries like:

"new amsterdam"~3
"old york"~3
"new york"~3

all give not the expected results. After further investigation, it seems
like that the CommonGramsQuery filter should not be applied for proximity
searches where you want to have this applied for normal searches.

When we remove the CommonGramsQuery filter out of the config and run the
queries again, it works as expected.

Any suggestions on how to overcome this?

Thanks,
Sjoerd

Re: CommonGramsQuery and proximity searches issue

[Sjoerd Smeets <ss...@gmail.com>]

Thank Mikhail,

Yeah, we have checked the debug output and also both filters are
configured. I'll see if I can add a test case using a slope to illustrate
the bug.

Cheers,
Sjoerd



On Mon, Dec 9, 2024 at 4:32 PM Mikhail Khludnev <mk...@apache.org> wrote:

> Hi Sjoerd.
> I've checked the guide
>
> https://solr.apache.org/guide/solr/latest/indexing-guide/filters.html#common-grams-filter
> it suggests the different filters for indexing and query analyser. You
> didn't mention using them both differently.
>
> Note, index and query time analysis maybe checked via solr Admin analysis
> page. And parsed query is shown via debugQuery=true. Hope it helps to
> troubleshoot.
> I've found a test proving some usage of commongrams, but there are no
> slopes over there.
>
> https://github.com/apache/solr/blob/8479d6da5e514fcd42720c42d14ebfb08ceed3f2/solr/core/src/test/org/apache/solr/analysis/CommonGramsPhraseQueryTest.java#L30
>
>
> On Mon, Dec 9, 2024 at 6:02 PM Sjoerd Smeets <ss...@gmail.com> wrote:
>
> > Hi,
> >
> > We are using a CommonGrams filter for both indexing and querying. We
> > provide a list of words that should be treated as a common gram. In the
> > list the following words exist:
> >
> > - new
> > - york
> >
> > When we do queries like:
> >
> > "new amsterdam"~3
> > "old york"~3
> > "new york"~3
> >
> > all give not the expected results. After further investigation, it seems
> > like that the CommonGramsQuery filter should not be applied for proximity
> > searches where you want to have this applied for normal searches.
> >
> > When we remove the CommonGramsQuery filter out of the config and run the
> > queries again, it works as expected.
> >
> > Any suggestions on how to overcome this?
> >
> > Thanks,
> > Sjoerd
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>

Re: CommonGramsQuery and proximity searches issue

[Mikhail Khludnev <mk...@apache.org>]

Hi Sjoerd.
I've checked the guide
https://solr.apache.org/guide/solr/latest/indexing-guide/filters.html#common-grams-filter
it suggests the different filters for indexing and query analyser. You
didn't mention using them both differently.

Note, index and query time analysis maybe checked via solr Admin analysis
page. And parsed query is shown via debugQuery=true. Hope it helps to
troubleshoot.
I've found a test proving some usage of commongrams, but there are no
slopes over there.
https://github.com/apache/solr/blob/8479d6da5e514fcd42720c42d14ebfb08ceed3f2/solr/core/src/test/org/apache/solr/analysis/CommonGramsPhraseQueryTest.java#L30


On Mon, Dec 9, 2024 at 6:02 PM Sjoerd Smeets <ss...@gmail.com> wrote:

> Hi,
>
> We are using a CommonGrams filter for both indexing and querying. We
> provide a list of words that should be treated as a common gram. In the
> list the following words exist:
>
> - new
> - york
>
> When we do queries like:
>
> "new amsterdam"~3
> "old york"~3
> "new york"~3
>
> all give not the expected results. After further investigation, it seems
> like that the CommonGramsQuery filter should not be applied for proximity
> searches where you want to have this applied for normal searches.
>
> When we remove the CommonGramsQuery filter out of the config and run the
> queries again, it works as expected.
>
> Any suggestions on how to overcome this?
>
> Thanks,
> Sjoerd
>


-- 
Sincerely yours
Mikhail Khludnev