You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Aloke Ghoshal <al...@gmail.com> on 2013/06/11 14:29:27 UTC
Filtering down terms in suggest
Hi,
Trying to find a way to filter down the suggested terms set based on the
term value of another indexed field?
Let's say we have the following documents indexed in Solr:
userid:1, groupid:1, content:"alpha beta gamma"
userid:2, groupid:1, content:"alternate better garden"
userid:3, groupid:2, content:"altruism bent garner"
Now a query on (with a dictionary built using terms in the content field):
q:groupid:1 AND content:al
should suggest alpha & alternate, (not altruism, since it has a different
groupid).
The option to have a separate dictionary per group gets ruled out due to
the high number of distinct groups (50K+).
Kindly suggest ways to get this working.
Thanks,
Aloke
Re: Filtering down terms in suggest
Posted by Aloke Ghoshal <al...@gmail.com>.
Thanks Barani. Could also work out this way provided we start with a large
set of suggestions initially to increase the likelihood of getting some
matches when filtering down with the second query.
On Wed, Jun 12, 2013 at 10:51 PM, bbarani <bb...@gmail.com> wrote:
> I would suggest you to take the suggested string and create another query
> to
> solr along with the filter parameter.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Filtering-down-terms-in-suggest-tp4069627p4069997.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Filtering down terms in suggest
Posted by bbarani <bb...@gmail.com>.
I would suggest you to take the suggested string and create another query to
solr along with the filter parameter.
--
View this message in context: http://lucene.472066.n3.nabble.com/Filtering-down-terms-in-suggest-tp4069627p4069997.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Filtering down terms in suggest
Posted by Aloke Ghoshal <al...@gmail.com>.
Thanks Jason, querying would be a good way to approach this. Though not
NGram, thinking of doing a wildcard based search & use the highlighted text
for suggestions.
On Wed, Jun 12, 2013 at 6:49 PM, Jason Hellman <
jhellman@innoventsolutions.com> wrote:
> Aloke,
>
> It may be best to simply run a query to populate the suggestion list.
> While not as fast as the terms component (and suggester offshoots) it can
> still be tuned to be very, very fast.
>
> In this way, you can generate any fq/q combination required to meet your
> needs. You can play with wildcard searches, or better yet NGram
> (EdgeNGram) behavior to get the right suggestion data back.
>
> I would suggest an additional core to accomplish this (fed via
> replication) to avoid cache entry collision with your normal queries.
>
> Hope that's useful to you.
>
> Jason
>
> On Jun 12, 2013, at 7:43 AM, Aloke Ghoshal <al...@gmail.com> wrote:
>
> > Barani - the fq option doesn't work.
> > Jason - the dynamic field option won't work due to the high number of
> > groups and users.
> >
> >
> >
> > On Wed, Jun 12, 2013 at 1:12 AM, Jason Hellman <
> > jhellman@innoventsolutions.com> wrote:
> >
> >> Aloke,
> >>
> >> If you do not have a factorial problem in the combination of userid and
> >> groupid (which I can imagine you might) you could consider creating a
> field
> >> for each combination (u1g1, u2g2) which can easily be done via dynamic
> >> fields. Use CopyField to get data into these various constructs (again,
> >> easily configured via wildcard patterns) and then send the suggestion
> query
> >> to the right field.
> >>
> >> Obviously this will get out of hand if you have too many of these...so
> >> this has limits.
> >>
> >> Jason
> >>
> >> On Jun 11, 2013, at 8:29 AM, Aloke Ghoshal <al...@gmail.com> wrote:
> >>
> >>> Hi,
> >>>
> >>> Trying to find a way to filter down the suggested terms set based on
> the
> >>> term value of another indexed field?
> >>>
> >>> Let's say we have the following documents indexed in Solr:
> >>> userid:1, groupid:1, content:"alpha beta gamma"
> >>> userid:2, groupid:1, content:"alternate better garden"
> >>> userid:3, groupid:2, content:"altruism bent garner"
> >>>
> >>> Now a query on (with a dictionary built using terms in the content
> >> field):
> >>> q:groupid:1 AND content:al
> >>>
> >>> should suggest alpha & alternate, (not altruism, since it has a
> different
> >>> groupid).
> >>>
> >>> The option to have a separate dictionary per group gets ruled out due
> to
> >>> the high number of distinct groups (50K+).
> >>>
> >>> Kindly suggest ways to get this working.
> >>>
> >>> Thanks,
> >>> Aloke
> >>
> >>
>
>
Re: Filtering down terms in suggest
Posted by Jason Hellman <jh...@innoventsolutions.com>.
Aloke,
It may be best to simply run a query to populate the suggestion list. While not as fast as the terms component (and suggester offshoots) it can still be tuned to be very, very fast.
In this way, you can generate any fq/q combination required to meet your needs. You can play with wildcard searches, or better yet NGram (EdgeNGram) behavior to get the right suggestion data back.
I would suggest an additional core to accomplish this (fed via replication) to avoid cache entry collision with your normal queries.
Hope that's useful to you.
Jason
On Jun 12, 2013, at 7:43 AM, Aloke Ghoshal <al...@gmail.com> wrote:
> Barani - the fq option doesn't work.
> Jason - the dynamic field option won't work due to the high number of
> groups and users.
>
>
>
> On Wed, Jun 12, 2013 at 1:12 AM, Jason Hellman <
> jhellman@innoventsolutions.com> wrote:
>
>> Aloke,
>>
>> If you do not have a factorial problem in the combination of userid and
>> groupid (which I can imagine you might) you could consider creating a field
>> for each combination (u1g1, u2g2) which can easily be done via dynamic
>> fields. Use CopyField to get data into these various constructs (again,
>> easily configured via wildcard patterns) and then send the suggestion query
>> to the right field.
>>
>> Obviously this will get out of hand if you have too many of these...so
>> this has limits.
>>
>> Jason
>>
>> On Jun 11, 2013, at 8:29 AM, Aloke Ghoshal <al...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Trying to find a way to filter down the suggested terms set based on the
>>> term value of another indexed field?
>>>
>>> Let's say we have the following documents indexed in Solr:
>>> userid:1, groupid:1, content:"alpha beta gamma"
>>> userid:2, groupid:1, content:"alternate better garden"
>>> userid:3, groupid:2, content:"altruism bent garner"
>>>
>>> Now a query on (with a dictionary built using terms in the content
>> field):
>>> q:groupid:1 AND content:al
>>>
>>> should suggest alpha & alternate, (not altruism, since it has a different
>>> groupid).
>>>
>>> The option to have a separate dictionary per group gets ruled out due to
>>> the high number of distinct groups (50K+).
>>>
>>> Kindly suggest ways to get this working.
>>>
>>> Thanks,
>>> Aloke
>>
>>
Re: Filtering down terms in suggest
Posted by Aloke Ghoshal <al...@gmail.com>.
Barani - the fq option doesn't work.
Jason - the dynamic field option won't work due to the high number of
groups and users.
On Wed, Jun 12, 2013 at 1:12 AM, Jason Hellman <
jhellman@innoventsolutions.com> wrote:
> Aloke,
>
> If you do not have a factorial problem in the combination of userid and
> groupid (which I can imagine you might) you could consider creating a field
> for each combination (u1g1, u2g2) which can easily be done via dynamic
> fields. Use CopyField to get data into these various constructs (again,
> easily configured via wildcard patterns) and then send the suggestion query
> to the right field.
>
> Obviously this will get out of hand if you have too many of these...so
> this has limits.
>
> Jason
>
> On Jun 11, 2013, at 8:29 AM, Aloke Ghoshal <al...@gmail.com> wrote:
>
> > Hi,
> >
> > Trying to find a way to filter down the suggested terms set based on the
> > term value of another indexed field?
> >
> > Let's say we have the following documents indexed in Solr:
> > userid:1, groupid:1, content:"alpha beta gamma"
> > userid:2, groupid:1, content:"alternate better garden"
> > userid:3, groupid:2, content:"altruism bent garner"
> >
> > Now a query on (with a dictionary built using terms in the content
> field):
> > q:groupid:1 AND content:al
> >
> > should suggest alpha & alternate, (not altruism, since it has a different
> > groupid).
> >
> > The option to have a separate dictionary per group gets ruled out due to
> > the high number of distinct groups (50K+).
> >
> > Kindly suggest ways to get this working.
> >
> > Thanks,
> > Aloke
>
>
Re: Filtering down terms in suggest
Posted by Jason Hellman <jh...@innoventsolutions.com>.
Aloke,
If you do not have a factorial problem in the combination of userid and groupid (which I can imagine you might) you could consider creating a field for each combination (u1g1, u2g2) which can easily be done via dynamic fields. Use CopyField to get data into these various constructs (again, easily configured via wildcard patterns) and then send the suggestion query to the right field.
Obviously this will get out of hand if you have too many of these...so this has limits.
Jason
On Jun 11, 2013, at 8:29 AM, Aloke Ghoshal <al...@gmail.com> wrote:
> Hi,
>
> Trying to find a way to filter down the suggested terms set based on the
> term value of another indexed field?
>
> Let's say we have the following documents indexed in Solr:
> userid:1, groupid:1, content:"alpha beta gamma"
> userid:2, groupid:1, content:"alternate better garden"
> userid:3, groupid:2, content:"altruism bent garner"
>
> Now a query on (with a dictionary built using terms in the content field):
> q:groupid:1 AND content:al
>
> should suggest alpha & alternate, (not altruism, since it has a different
> groupid).
>
> The option to have a separate dictionary per group gets ruled out due to
> the high number of distinct groups (50K+).
>
> Kindly suggest ways to get this working.
>
> Thanks,
> Aloke
Re: Filtering down terms in suggest
Posted by bbarani <bb...@gmail.com>.
I suppose you can use fq with SpellCheckComponent but I haven't tried it yet.
https://issues.apache.org/jira/browse/SOLR-2010
--
View this message in context: http://lucene.472066.n3.nabble.com/Filtering-down-terms-in-suggest-tp4069627p4069690.html
Sent from the Solr - User mailing list archive at Nabble.com.