You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ashwin Ramesh <as...@canva.com.INVALID> on 2020/02/18 00:50:20 UTC
Best Practises around relevance tuning per query
Hi,
We are in the process of applying a scoring model to our search results. In
particular, we would like to add scores for documents per query and user
context.
For example, we want to have a score from 500 to 1 for the top 500
documents for the query “dog” for users who speak US English.
We believe it becomes infeasible to store these scores in Solr because we
want to update the scores regularly, and the number of scores increases
rapidly with increased user attributes.
One solution we explored was to store these scores in a secondary data
store, and use this at Solr query time with a boost function such as:
`bf=mul(termfreq(id,’ID-1'),500) mul(termfreq(id,'ID-2'),499) …
mul(termfreq(id,'ID-500'),1)`
We have over a hundred thousand documents in one Solr collection, and about
fifty million in another Solr collection. We have some queries for which
roughly 80% of the results match, although this is an edge case. We wanted
to know the worst case performance, so we tested with such a query. For
both of these collections we found the a message similar to the following
in the Solr cloud logs (tested on a laptop):
Elapsed time: 5020. Exceeded allowed search time: 5000 ms.
We then tried using the following boost, which seemed simpler:
`boost=if(query($qq), 10, 1)&qq=id:(ID-1 OR ID-2 OR … OR ID-500)`
We then saw the following in the Solr cloud logs:
`The request took too long to iterate over terms.`
All responses above took over 5000 milliseconds to return.
We are considering Solr’s re-ranker, but I don’t know how we would use this
without pushing all the query-context-document scores to Solr.
The alternative solution that we are currently considering involves
invoking multiple solr queries.
This means we would make a request to solr to fetch the top N results (id,
score) for the query. E.g. q=dog, fq=featureA:foo, fq=featureB=bar, limit=N.
Another request would be made using a filter query with a set of doc ids
that we know are high value for the user’s query. E.g. q=*:*,
fq=featureA:foo, fq=featureB:bar, fq=id:(d1, d2, d3), limit=N.
We would then do a reranking phase in our service layer.
Do you have any suggestions for known patterns of how we can store and
retrieve scores per user context and query?
Regards,
Ash & Spirit.
--
**
** <https://www.canva.com/>Empowering the world to design
Also, we're
hiring. Apply here! <https://about.canva.com/careers/>
<https://twitter.com/canva> <https://facebook.com/canva>
<https://au.linkedin.com/company/canva> <https://twitter.com/canva>
<https://facebook.com/canva> <https://au.linkedin.com/company/canva>
<https://instagram.com/canva>
Re: Best Practises around relevance tuning per query
Posted by Mikhail Khludnev <mk...@apache.org>.
Note, {!terms} query is more efficient for long ids list. I'd try to group
ids by boost, and cache long ids lists. Something like:
q=filter({!terms f=id}1,3,5)^=100 filter({!terms f=id}2,4,6)^=-1
Thus, it let to reuse heavy terms lists between queries.
Another idea, extract boost score to the separate core/index (strictly
single shard in SolrCloud so far), and use {!join score=sum} to bring ranks
to the main index. It let to update smaller core faster. Although, it might
require some hack to decouple updates and cache invalidation.
Also, Solr has in-place updates, which might update columns with boosts and
score by this column.
On Tue, Feb 18, 2020 at 9:27 PM Ashwin Ramesh <as...@canva.com.invalid>
wrote:
> ping on this :)
>
> On Tue, Feb 18, 2020 at 11:50 AM Ashwin Ramesh <as...@canva.com> wrote:
>
> > Hi,
> >
> > We are in the process of applying a scoring model to our search results.
> > In particular, we would like to add scores for documents per query and
> user
> > context.
> >
> > For example, we want to have a score from 500 to 1 for the top 500
> > documents for the query “dog” for users who speak US English.
> >
> > We believe it becomes infeasible to store these scores in Solr because we
> > want to update the scores regularly, and the number of scores increases
> > rapidly with increased user attributes.
> >
> > One solution we explored was to store these scores in a secondary data
> > store, and use this at Solr query time with a boost function such as:
> >
> > `bf=mul(termfreq(id,’ID-1'),500) mul(termfreq(id,'ID-2'),499) …
> > mul(termfreq(id,'ID-500'),1)`
> >
> > We have over a hundred thousand documents in one Solr collection, and
> > about fifty million in another Solr collection. We have some queries for
> > which roughly 80% of the results match, although this is an edge case. We
> > wanted to know the worst case performance, so we tested with such a
> query.
> > For both of these collections we found the a message similar to the
> > following in the Solr cloud logs (tested on a laptop):
> >
> > Elapsed time: 5020. Exceeded allowed search time: 5000 ms.
> >
> > We then tried using the following boost, which seemed simpler:
> >
> > `boost=if(query($qq), 10, 1)&qq=id:(ID-1 OR ID-2 OR … OR ID-500)`
> >
> > We then saw the following in the Solr cloud logs:
> >
> > `The request took too long to iterate over terms.`
> >
> > All responses above took over 5000 milliseconds to return.
> >
> > We are considering Solr’s re-ranker, but I don’t know how we would use
> > this without pushing all the query-context-document scores to Solr.
> >
> >
> > The alternative solution that we are currently considering involves
> > invoking multiple solr queries.
> >
> > This means we would make a request to solr to fetch the top N results
> (id,
> > score) for the query. E.g. q=dog, fq=featureA:foo, fq=featureB=bar,
> limit=N.
> >
> > Another request would be made using a filter query with a set of doc ids
> > that we know are high value for the user’s query. E.g. q=*:*,
> > fq=featureA:foo, fq=featureB:bar, fq=id:(d1, d2, d3), limit=N.
> >
> > We would then do a reranking phase in our service layer.
> >
> > Do you have any suggestions for known patterns of how we can store and
> > retrieve scores per user context and query?
> >
> > Regards,
> > Ash & Spirit.
> >
>
> --
> **
> ** <https://www.canva.com/>Empowering the world to design
> Also, we're
> hiring. Apply here! <https://about.canva.com/careers/>
>
> <https://twitter.com/canva> <https://facebook.com/canva>
> <https://au.linkedin.com/company/canva> <https://twitter.com/canva>
> <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> <https://instagram.com/canva>
>
>
>
>
>
>
>
>
>
>
>
--
Sincerely yours
Mikhail Khludnev
Re: Best Practises around relevance tuning per query
Posted by Walter Underwood <wu...@wunderwood.org>.
I didn’t respond because it seemed like you were stuck on an approach that would
never be efficient in Solr. It requires massive amounts of data applied to documents
in a fine-grained way. Maybe it makes the math easier, but the data management
is impractical. I could not see any way to make that fast.
Here are four alternative approaches.
1. Instead of a 1-500 scale, use a 0/1 scale. Either this result is good for this query term or not.
That can be implemented with a single multivalued field containing the query terms. If the
query matches that field, it gets a boost.
2. How much of that score is really different between query terms? Split out a common document
score that is independent of the query term. Use the boost parameter with that document
quality score and see how close it is to the ideal ranking.
3. Group your queries and documents into categories. Give each document a score for each
category. That could be boolean (in the category or not) or a quality score for that category.
That can be stored in a dynamic field, so topic_score_1, topic_score_42, etc. A query for
topic 42 fetches the matching field. We did this for three different sets of topics each with
thousands of categories. It was a ton of fields, but ran really fast.
You can categorize the query by seeing which documents it matches. Check the category
memberships of the first k results and choose the top-scoring category. This is a kNN
(k Nearest Neighbors) classifier. Then take that category and run a second query using
the category scores.
4. Pre-calculate the top 50 results for each category with the slow algorithm and use the
elevate component to force that ranking for that term.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Feb 18, 2020, at 9:27 PM, Ashwin Ramesh <as...@canva.com.INVALID> wrote:
>
> ping on this :)
>
> On Tue, Feb 18, 2020 at 11:50 AM Ashwin Ramesh <as...@canva.com> wrote:
>
>> Hi,
>>
>> We are in the process of applying a scoring model to our search results.
>> In particular, we would like to add scores for documents per query and user
>> context.
>>
>> For example, we want to have a score from 500 to 1 for the top 500
>> documents for the query “dog” for users who speak US English.
>>
>> We believe it becomes infeasible to store these scores in Solr because we
>> want to update the scores regularly, and the number of scores increases
>> rapidly with increased user attributes.
>>
>> One solution we explored was to store these scores in a secondary data
>> store, and use this at Solr query time with a boost function such as:
>>
>> `bf=mul(termfreq(id,’ID-1'),500) mul(termfreq(id,'ID-2'),499) …
>> mul(termfreq(id,'ID-500'),1)`
>>
>> We have over a hundred thousand documents in one Solr collection, and
>> about fifty million in another Solr collection. We have some queries for
>> which roughly 80% of the results match, although this is an edge case. We
>> wanted to know the worst case performance, so we tested with such a query.
>> For both of these collections we found the a message similar to the
>> following in the Solr cloud logs (tested on a laptop):
>>
>> Elapsed time: 5020. Exceeded allowed search time: 5000 ms.
>>
>> We then tried using the following boost, which seemed simpler:
>>
>> `boost=if(query($qq), 10, 1)&qq=id:(ID-1 OR ID-2 OR … OR ID-500)`
>>
>> We then saw the following in the Solr cloud logs:
>>
>> `The request took too long to iterate over terms.`
>>
>> All responses above took over 5000 milliseconds to return.
>>
>> We are considering Solr’s re-ranker, but I don’t know how we would use
>> this without pushing all the query-context-document scores to Solr.
>>
>>
>> The alternative solution that we are currently considering involves
>> invoking multiple solr queries.
>>
>> This means we would make a request to solr to fetch the top N results (id,
>> score) for the query. E.g. q=dog, fq=featureA:foo, fq=featureB=bar, limit=N.
>>
>> Another request would be made using a filter query with a set of doc ids
>> that we know are high value for the user’s query. E.g. q=*:*,
>> fq=featureA:foo, fq=featureB:bar, fq=id:(d1, d2, d3), limit=N.
>>
>> We would then do a reranking phase in our service layer.
>>
>> Do you have any suggestions for known patterns of how we can store and
>> retrieve scores per user context and query?
>>
>> Regards,
>> Ash & Spirit.
>>
>
> --
> **
> ** <https://www.canva.com/>Empowering the world to design
> Also, we're
> hiring. Apply here! <https://about.canva.com/careers/>
>
> <https://twitter.com/canva> <https://facebook.com/canva>
> <https://au.linkedin.com/company/canva> <https://twitter.com/canva>
> <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> <https://instagram.com/canva>
>
>
>
>
>
>
>
>
>
>
Re: [SUSPICIOUS] Re: Best Practises around relevance tuning per
query
Posted by David Hastings <dh...@wshein.com>.
I don’t think anyone is responding because it’s too focused of a use case, where you just simply have to figure out an alternative on your own.
> On Feb 19, 2020, at 12:28 AM, Ashwin Ramesh <as...@canva.com.invalid> wrote:
>
> ping on this :)
>
>> On Tue, Feb 18, 2020 at 11:50 AM Ashwin Ramesh <as...@canva.com> wrote:
>>
>> Hi,
>>
>> We are in the process of applying a scoring model to our search results.
>> In particular, we would like to add scores for documents per query and user
>> context.
>>
>> For example, we want to have a score from 500 to 1 for the top 500
>> documents for the query “dog” for users who speak US English.
>>
>> We believe it becomes infeasible to store these scores in Solr because we
>> want to update the scores regularly, and the number of scores increases
>> rapidly with increased user attributes.
>>
>> One solution we explored was to store these scores in a secondary data
>> store, and use this at Solr query time with a boost function such as:
>>
>> `bf=mul(termfreq(id,’ID-1'),500) mul(termfreq(id,'ID-2'),499) …
>> mul(termfreq(id,'ID-500'),1)`
>>
>> We have over a hundred thousand documents in one Solr collection, and
>> about fifty million in another Solr collection. We have some queries for
>> which roughly 80% of the results match, although this is an edge case. We
>> wanted to know the worst case performance, so we tested with such a query.
>> For both of these collections we found the a message similar to the
>> following in the Solr cloud logs (tested on a laptop):
>>
>> Elapsed time: 5020. Exceeded allowed search time: 5000 ms.
>>
>> We then tried using the following boost, which seemed simpler:
>>
>> `boost=if(query($qq), 10, 1)&qq=id:(ID-1 OR ID-2 OR … OR ID-500)`
>>
>> We then saw the following in the Solr cloud logs:
>>
>> `The request took too long to iterate over terms.`
>>
>> All responses above took over 5000 milliseconds to return.
>>
>> We are considering Solr’s re-ranker, but I don’t know how we would use
>> this without pushing all the query-context-document scores to Solr.
>>
>>
>> The alternative solution that we are currently considering involves
>> invoking multiple solr queries.
>>
>> This means we would make a request to solr to fetch the top N results (id,
>> score) for the query. E.g. q=dog, fq=featureA:foo, fq=featureB=bar, limit=N.
>>
>> Another request would be made using a filter query with a set of doc ids
>> that we know are high value for the user’s query. E.g. q=*:*,
>> fq=featureA:foo, fq=featureB:bar, fq=id:(d1, d2, d3), limit=N.
>>
>> We would then do a reranking phase in our service layer.
>>
>> Do you have any suggestions for known patterns of how we can store and
>> retrieve scores per user context and query?
>>
>> Regards,
>> Ash & Spirit.
>>
>
> --
> **
> ** <https://www.canva.com/>Empowering the world to design
> Also, we're
> hiring. Apply here! <https://about.canva.com/careers/>
>
> <https://twitter.com/canva> <https://facebook.com/canva>
> <https://au.linkedin.com/company/canva> <https://twitter.com/canva>
> <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> <https://instagram.com/canva>
>
>
>
>
>
>
>
>
>
>
Re: Best Practises around relevance tuning per query
Posted by Ashwin Ramesh <as...@canva.com.INVALID>.
ping on this :)
On Tue, Feb 18, 2020 at 11:50 AM Ashwin Ramesh <as...@canva.com> wrote:
> Hi,
>
> We are in the process of applying a scoring model to our search results.
> In particular, we would like to add scores for documents per query and user
> context.
>
> For example, we want to have a score from 500 to 1 for the top 500
> documents for the query “dog” for users who speak US English.
>
> We believe it becomes infeasible to store these scores in Solr because we
> want to update the scores regularly, and the number of scores increases
> rapidly with increased user attributes.
>
> One solution we explored was to store these scores in a secondary data
> store, and use this at Solr query time with a boost function such as:
>
> `bf=mul(termfreq(id,’ID-1'),500) mul(termfreq(id,'ID-2'),499) …
> mul(termfreq(id,'ID-500'),1)`
>
> We have over a hundred thousand documents in one Solr collection, and
> about fifty million in another Solr collection. We have some queries for
> which roughly 80% of the results match, although this is an edge case. We
> wanted to know the worst case performance, so we tested with such a query.
> For both of these collections we found the a message similar to the
> following in the Solr cloud logs (tested on a laptop):
>
> Elapsed time: 5020. Exceeded allowed search time: 5000 ms.
>
> We then tried using the following boost, which seemed simpler:
>
> `boost=if(query($qq), 10, 1)&qq=id:(ID-1 OR ID-2 OR … OR ID-500)`
>
> We then saw the following in the Solr cloud logs:
>
> `The request took too long to iterate over terms.`
>
> All responses above took over 5000 milliseconds to return.
>
> We are considering Solr’s re-ranker, but I don’t know how we would use
> this without pushing all the query-context-document scores to Solr.
>
>
> The alternative solution that we are currently considering involves
> invoking multiple solr queries.
>
> This means we would make a request to solr to fetch the top N results (id,
> score) for the query. E.g. q=dog, fq=featureA:foo, fq=featureB=bar, limit=N.
>
> Another request would be made using a filter query with a set of doc ids
> that we know are high value for the user’s query. E.g. q=*:*,
> fq=featureA:foo, fq=featureB:bar, fq=id:(d1, d2, d3), limit=N.
>
> We would then do a reranking phase in our service layer.
>
> Do you have any suggestions for known patterns of how we can store and
> retrieve scores per user context and query?
>
> Regards,
> Ash & Spirit.
>
--
**
** <https://www.canva.com/>Empowering the world to design
Also, we're
hiring. Apply here! <https://about.canva.com/careers/>
<https://twitter.com/canva> <https://facebook.com/canva>
<https://au.linkedin.com/company/canva> <https://twitter.com/canva>
<https://facebook.com/canva> <https://au.linkedin.com/company/canva>
<https://instagram.com/canva>
Re: Best Practises around relevance tuning per query
Posted by Paras Lehana <pa...@indiamart.com>.
Hi Ashwin,
If I'm understanding your requirement correctly, I think you should read
about Payloads <https://lucidworks.com/post/solr-payloads/>.
On Thu, 27 Feb 2020 at 09:41, Ashwin Ramesh <as...@canva.com.invalid>
wrote:
> Hi everybody,
>
> Thank you for all the amazing feedback. I apologize for the formatting of
> my question.
>
> I guess if I was to generalize my question, 'What is the most common
> approaches to storing query level features in Solr documents?'
>
> For example, a normalized_click_score is a document level feature, but how
> would you scalably also do the same for specific queries? E.g. How do you
> define, *For the query 'ipod' this specific document is very relevant*.
>
> Thanks again!
>
> Regards,
>
> Ash
>
> On Wed, Feb 19, 2020 at 6:14 PM Jörn Franke <jo...@gmail.com> wrote:
>
> > You are too much focus on the solution. If you would describe the
> business
> > case in more detail without including the solution itself more people
> could
> > help.
> >
> > Eg it ie not clear why you have a scoring model and why this can address
> > business needs.
> >
> > > Am 18.02.2020 um 01:50 schrieb Ashwin Ramesh <ashwin@canva.com.invalid
> >:
> > >
> > > Hi,
> > >
> > > We are in the process of applying a scoring model to our search
> results.
> > In
> > > particular, we would like to add scores for documents per query and
> user
> > > context.
> > >
> > > For example, we want to have a score from 500 to 1 for the top 500
> > > documents for the query “dog” for users who speak US English.
> > >
> > > We believe it becomes infeasible to store these scores in Solr because
> we
> > > want to update the scores regularly, and the number of scores increases
> > > rapidly with increased user attributes.
> > >
> > > One solution we explored was to store these scores in a secondary data
> > > store, and use this at Solr query time with a boost function such as:
> > >
> > > `bf=mul(termfreq(id,’ID-1'),500) mul(termfreq(id,'ID-2'),499) …
> > > mul(termfreq(id,'ID-500'),1)`
> > >
> > > We have over a hundred thousand documents in one Solr collection, and
> > about
> > > fifty million in another Solr collection. We have some queries for
> which
> > > roughly 80% of the results match, although this is an edge case. We
> > wanted
> > > to know the worst case performance, so we tested with such a query. For
> > > both of these collections we found the a message similar to the
> following
> > > in the Solr cloud logs (tested on a laptop):
> > >
> > > Elapsed time: 5020. Exceeded allowed search time: 5000 ms.
> > >
> > > We then tried using the following boost, which seemed simpler:
> > >
> > > `boost=if(query($qq), 10, 1)&qq=id:(ID-1 OR ID-2 OR … OR ID-500)`
> > >
> > > We then saw the following in the Solr cloud logs:
> > >
> > > `The request took too long to iterate over terms.`
> > >
> > > All responses above took over 5000 milliseconds to return.
> > >
> > > We are considering Solr’s re-ranker, but I don’t know how we would use
> > this
> > > without pushing all the query-context-document scores to Solr.
> > >
> > >
> > > The alternative solution that we are currently considering involves
> > > invoking multiple solr queries.
> > >
> > > This means we would make a request to solr to fetch the top N results
> > (id,
> > > score) for the query. E.g. q=dog, fq=featureA:foo, fq=featureB=bar,
> > limit=N.
> > >
> > > Another request would be made using a filter query with a set of doc
> ids
> > > that we know are high value for the user’s query. E.g. q=*:*,
> > > fq=featureA:foo, fq=featureB:bar, fq=id:(d1, d2, d3), limit=N.
> > >
> > > We would then do a reranking phase in our service layer.
> > >
> > > Do you have any suggestions for known patterns of how we can store and
> > > retrieve scores per user context and query?
> > >
> > > Regards,
> > > Ash & Spirit.
> > >
> > > --
> > > **
> > > ** <https://www.canva.com/>Empowering the world to design
> > > Also, we're
> > > hiring. Apply here! <https://about.canva.com/careers/>
> > >
> > > <https://twitter.com/canva> <https://facebook.com/canva>
> > > <https://au.linkedin.com/company/canva> <https://twitter.com/canva>
> > > <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> > > <https://instagram.com/canva>
> >
>
> --
> **
> ** <https://www.canva.com/>Empowering the world to design
> Also, we're
> hiring. Apply here! <https://about.canva.com/careers/>
>
> <https://twitter.com/canva> <https://facebook.com/canva>
> <https://au.linkedin.com/company/canva> <https://twitter.com/canva>
> <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> <https://instagram.com/canva>
>
>
>
>
>
>
>
>
>
>
>
--
--
Regards,
*Paras Lehana* [65871]
Development Engineer, *Auto-Suggest*,
IndiaMART InterMESH Ltd,
11th Floor, Tower 2, Assotech Business Cresterra,
Plot No. 22, Sector 135, Noida, Uttar Pradesh, India 201305
Mob.: +91-9560911996
Work: 0120-4056700 | Extn:
*1196*
--
*
*
<https://www.facebook.com/IndiaMART/videos/578196442936091/>
Re: Best Practises around relevance tuning per query
Posted by Ashwin Ramesh <as...@canva.com.INVALID>.
Hi everybody,
Thank you for all the amazing feedback. I apologize for the formatting of
my question.
I guess if I was to generalize my question, 'What is the most common
approaches to storing query level features in Solr documents?'
For example, a normalized_click_score is a document level feature, but how
would you scalably also do the same for specific queries? E.g. How do you
define, *For the query 'ipod' this specific document is very relevant*.
Thanks again!
Regards,
Ash
On Wed, Feb 19, 2020 at 6:14 PM Jörn Franke <jo...@gmail.com> wrote:
> You are too much focus on the solution. If you would describe the business
> case in more detail without including the solution itself more people could
> help.
>
> Eg it ie not clear why you have a scoring model and why this can address
> business needs.
>
> > Am 18.02.2020 um 01:50 schrieb Ashwin Ramesh <as...@canva.com.invalid>:
> >
> > Hi,
> >
> > We are in the process of applying a scoring model to our search results.
> In
> > particular, we would like to add scores for documents per query and user
> > context.
> >
> > For example, we want to have a score from 500 to 1 for the top 500
> > documents for the query “dog” for users who speak US English.
> >
> > We believe it becomes infeasible to store these scores in Solr because we
> > want to update the scores regularly, and the number of scores increases
> > rapidly with increased user attributes.
> >
> > One solution we explored was to store these scores in a secondary data
> > store, and use this at Solr query time with a boost function such as:
> >
> > `bf=mul(termfreq(id,’ID-1'),500) mul(termfreq(id,'ID-2'),499) …
> > mul(termfreq(id,'ID-500'),1)`
> >
> > We have over a hundred thousand documents in one Solr collection, and
> about
> > fifty million in another Solr collection. We have some queries for which
> > roughly 80% of the results match, although this is an edge case. We
> wanted
> > to know the worst case performance, so we tested with such a query. For
> > both of these collections we found the a message similar to the following
> > in the Solr cloud logs (tested on a laptop):
> >
> > Elapsed time: 5020. Exceeded allowed search time: 5000 ms.
> >
> > We then tried using the following boost, which seemed simpler:
> >
> > `boost=if(query($qq), 10, 1)&qq=id:(ID-1 OR ID-2 OR … OR ID-500)`
> >
> > We then saw the following in the Solr cloud logs:
> >
> > `The request took too long to iterate over terms.`
> >
> > All responses above took over 5000 milliseconds to return.
> >
> > We are considering Solr’s re-ranker, but I don’t know how we would use
> this
> > without pushing all the query-context-document scores to Solr.
> >
> >
> > The alternative solution that we are currently considering involves
> > invoking multiple solr queries.
> >
> > This means we would make a request to solr to fetch the top N results
> (id,
> > score) for the query. E.g. q=dog, fq=featureA:foo, fq=featureB=bar,
> limit=N.
> >
> > Another request would be made using a filter query with a set of doc ids
> > that we know are high value for the user’s query. E.g. q=*:*,
> > fq=featureA:foo, fq=featureB:bar, fq=id:(d1, d2, d3), limit=N.
> >
> > We would then do a reranking phase in our service layer.
> >
> > Do you have any suggestions for known patterns of how we can store and
> > retrieve scores per user context and query?
> >
> > Regards,
> > Ash & Spirit.
> >
> > --
> > **
> > ** <https://www.canva.com/>Empowering the world to design
> > Also, we're
> > hiring. Apply here! <https://about.canva.com/careers/>
> >
> > <https://twitter.com/canva> <https://facebook.com/canva>
> > <https://au.linkedin.com/company/canva> <https://twitter.com/canva>
> > <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> > <https://instagram.com/canva>
>
--
**
** <https://www.canva.com/>Empowering the world to design
Also, we're
hiring. Apply here! <https://about.canva.com/careers/>
<https://twitter.com/canva> <https://facebook.com/canva>
<https://au.linkedin.com/company/canva> <https://twitter.com/canva>
<https://facebook.com/canva> <https://au.linkedin.com/company/canva>
<https://instagram.com/canva>
Re: Best Practises around relevance tuning per query
Posted by Jörn Franke <jo...@gmail.com>.
You are too much focus on the solution. If you would describe the business case in more detail without including the solution itself more people could help.
Eg it ie not clear why you have a scoring model and why this can address business needs.
> Am 18.02.2020 um 01:50 schrieb Ashwin Ramesh <as...@canva.com.invalid>:
>
> Hi,
>
> We are in the process of applying a scoring model to our search results. In
> particular, we would like to add scores for documents per query and user
> context.
>
> For example, we want to have a score from 500 to 1 for the top 500
> documents for the query “dog” for users who speak US English.
>
> We believe it becomes infeasible to store these scores in Solr because we
> want to update the scores regularly, and the number of scores increases
> rapidly with increased user attributes.
>
> One solution we explored was to store these scores in a secondary data
> store, and use this at Solr query time with a boost function such as:
>
> `bf=mul(termfreq(id,’ID-1'),500) mul(termfreq(id,'ID-2'),499) …
> mul(termfreq(id,'ID-500'),1)`
>
> We have over a hundred thousand documents in one Solr collection, and about
> fifty million in another Solr collection. We have some queries for which
> roughly 80% of the results match, although this is an edge case. We wanted
> to know the worst case performance, so we tested with such a query. For
> both of these collections we found the a message similar to the following
> in the Solr cloud logs (tested on a laptop):
>
> Elapsed time: 5020. Exceeded allowed search time: 5000 ms.
>
> We then tried using the following boost, which seemed simpler:
>
> `boost=if(query($qq), 10, 1)&qq=id:(ID-1 OR ID-2 OR … OR ID-500)`
>
> We then saw the following in the Solr cloud logs:
>
> `The request took too long to iterate over terms.`
>
> All responses above took over 5000 milliseconds to return.
>
> We are considering Solr’s re-ranker, but I don’t know how we would use this
> without pushing all the query-context-document scores to Solr.
>
>
> The alternative solution that we are currently considering involves
> invoking multiple solr queries.
>
> This means we would make a request to solr to fetch the top N results (id,
> score) for the query. E.g. q=dog, fq=featureA:foo, fq=featureB=bar, limit=N.
>
> Another request would be made using a filter query with a set of doc ids
> that we know are high value for the user’s query. E.g. q=*:*,
> fq=featureA:foo, fq=featureB:bar, fq=id:(d1, d2, d3), limit=N.
>
> We would then do a reranking phase in our service layer.
>
> Do you have any suggestions for known patterns of how we can store and
> retrieve scores per user context and query?
>
> Regards,
> Ash & Spirit.
>
> --
> **
> ** <https://www.canva.com/>Empowering the world to design
> Also, we're
> hiring. Apply here! <https://about.canva.com/careers/>
>
> <https://twitter.com/canva> <https://facebook.com/canva>
> <https://au.linkedin.com/company/canva> <https://twitter.com/canva>
> <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> <https://instagram.com/canva>