You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net> on 2024/11/13 12:49:06 UTC
Re:Is it possible to retrieve partial search results and highlight snippets from
Just throwing this out there, have you tried the unifiedHighlighter with termVectors/character offsets? The default configuration re-analyzes the text of the result-set which can be cpu-bound for large docs/fields. But there are ways to get around this by storing the character offsets and termVectors in the index, which can be a good tradeoff in many situations, i.e. "Postings"
https://solr.apache.org/guide/solr/latest/query-guide/highlighting.html#:~:text=over%20highlighting%20performance.-,Postings,-%3A%20Supported%20by%20the
I know this doesn't answer your question but just wanted to mention an alternative solution to relying on time-outs/partial-results, which may be frustrating for other reasons.
Sent from Bloomberg Professional for Android
----- Original Message -----
From: Mingchun Zhao <us...@solr.apache.org>
To: users@solr.apache.org
At: 11/12/24 21:55:36 UTC-05:00
Hi,
If a query is interrupted due to timeAllowed, does it still return the
partially matched documents and highlight snippets on it? Or does the
behavior vary depending on the Solr version?
In my testing, when the response header includes "partialResults=true", the
partial search results or snippets were not included in the response body.
Regards,
Zhao
Re: Is it possible to retrieve partial search results and highlight snippets fro
Posted by Mingchun Zhao <mi...@gmail.com>.
Hi Mikhail,
> Presumably, if an app got just partial results without highlightings, it
can make the second request searching by IDs of found results, but specify
initial query for highlighting via hl.q.
Thank you for the great idea! I'll consider this approach as well.
Regards,
Mingchun
2024年11月16日(土) 19:21 Mikhail Khludnev <mk...@apache.org>:
> Presumably, if an app got just partial results without highlightings, it
> can make the second request searching by IDs of found results, but specify
> initial query for highlighting via hl.q.
>
> On Sat, Nov 16, 2024 at 11:13 AM Mingchun Zhao <mi...@gmail.com>
> wrote:
>
> > Hi Luke,
> >
> > Thank you for analyzing the code and kindly providing a detailed
> > explanation.
> >
> > I now understand that it is not possible to retrieve the partial result
> set
> > or its highlighted snippets for a query interrupted due to exceeding the
> > "timeAllowed" setting value.
> >
> > I truly appreciate your help once again.
> >
> > Regards,
> > Zhao
> >
> > 2024年11月15日(金) 21:46 Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <
> > lkotzaniewsk@bloomberg.net>:
> >
> > > From reading the code I don't think it is possible. If your timeout is
> > > occurring in the core search the SolrIndexSearcher will throw a certain
> > > type of ExitingReaderException that is caught very high up (in the
> > > SearchHandler). This happens as the SearchHandler is looping through
> > > "SearchComponents" one of which is the HighlightComponent. Effectively
> > this
> > > means the search processing is short-circuited as no downstream
> > components
> > > get invoked. Not sure if changing this behavior is the right thing
> since
> > > highlighting itself can be expensive so the timeout would be
> misleading.
> > I
> > > suppose you could consider implementing more granular timeouts but not
> > sure
> > > about the return on investment of something like that.
> > >
> > > Sent from Bloomberg Professional for Android
> > >
> > > ----- Original Message -----
> > > From: Mingchun Zhao <us...@solr.apache.org>
> > > To: users@solr.apache.org
> > > At: 11/13/24 20:38:49 UTC-05:00
> > >
> > >
> > > Hi Luke,
> > >
> > > Thank you very much for the valuable information. I’ll give the Unified
> > > Highlighter’s Postings feature a try.
> > >
> > > That said, I’d still like to know more about the behavior of the
> > > "timeAllowed" parameter, especially whether it’s possible to retrieve
> > > partial search results (both matching documents and highlighted
> snippets)
> > > when a query times out.
> > >
> > > If you have any information on this, I’d appreciate it.
> > >
> > > Regards,
> > > Zhao
> > >
> > > 2024年11月13日(水) 21:50 Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <
> > > lkotzaniewsk@bloomberg.net>:
> > >
> > > > Just throwing this out there, have you tried the unifiedHighlighter
> > with
> > > > termVectors/character offsets? The default configuration re-analyzes
> > the
> > > > text of the result-set which can be cpu-bound for large docs/fields.
> > But
> > > > there are ways to get around this by storing the character offsets
> and
> > > > termVectors in the index, which can be a good tradeoff in many
> > > situations,
> > > > i.e. "Postings"
> > > >
> > > >
> > >
> >
> https://solr.apache.org/guide/solr/latest/query-guide/highlighting.html#:~:text=over%20highlighting%20performance.-,Postings,-%3A%20Supported%20by%20the
> > > >
> > > > I know this doesn't answer your question but just wanted to mention
> an
> > > > alternative solution to relying on time-outs/partial-results, which
> may
> > > be
> > > > frustrating for other reasons.
> > > > Sent from Bloomberg Professional for Android
> > > >
> > > > ----- Original Message -----
> > > > From: Mingchun Zhao <us...@solr.apache.org>
> > > > To: users@solr.apache.org
> > > > At: 11/12/24 21:55:36 UTC-05:00
> > > >
> > > >
> > > > Hi,
> > > >
> > > > If a query is interrupted due to timeAllowed, does it still return
> the
> > > > partially matched documents and highlight snippets on it? Or does the
> > > > behavior vary depending on the Solr version?
> > > > In my testing, when the response header includes
> "partialResults=true",
> > > the
> > > > partial search results or snippets were not included in the response
> > > body.
> > > >
> > > > Regards,
> > > > Zhao
> > > >
> > >
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>
Re: Is it possible to retrieve partial search results and highlight snippets fro
Posted by Mikhail Khludnev <mk...@apache.org>.
Presumably, if an app got just partial results without highlightings, it
can make the second request searching by IDs of found results, but specify
initial query for highlighting via hl.q.
On Sat, Nov 16, 2024 at 11:13 AM Mingchun Zhao <mi...@gmail.com>
wrote:
> Hi Luke,
>
> Thank you for analyzing the code and kindly providing a detailed
> explanation.
>
> I now understand that it is not possible to retrieve the partial result set
> or its highlighted snippets for a query interrupted due to exceeding the
> "timeAllowed" setting value.
>
> I truly appreciate your help once again.
>
> Regards,
> Zhao
>
> 2024年11月15日(金) 21:46 Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <
> lkotzaniewsk@bloomberg.net>:
>
> > From reading the code I don't think it is possible. If your timeout is
> > occurring in the core search the SolrIndexSearcher will throw a certain
> > type of ExitingReaderException that is caught very high up (in the
> > SearchHandler). This happens as the SearchHandler is looping through
> > "SearchComponents" one of which is the HighlightComponent. Effectively
> this
> > means the search processing is short-circuited as no downstream
> components
> > get invoked. Not sure if changing this behavior is the right thing since
> > highlighting itself can be expensive so the timeout would be misleading.
> I
> > suppose you could consider implementing more granular timeouts but not
> sure
> > about the return on investment of something like that.
> >
> > Sent from Bloomberg Professional for Android
> >
> > ----- Original Message -----
> > From: Mingchun Zhao <us...@solr.apache.org>
> > To: users@solr.apache.org
> > At: 11/13/24 20:38:49 UTC-05:00
> >
> >
> > Hi Luke,
> >
> > Thank you very much for the valuable information. I’ll give the Unified
> > Highlighter’s Postings feature a try.
> >
> > That said, I’d still like to know more about the behavior of the
> > "timeAllowed" parameter, especially whether it’s possible to retrieve
> > partial search results (both matching documents and highlighted snippets)
> > when a query times out.
> >
> > If you have any information on this, I’d appreciate it.
> >
> > Regards,
> > Zhao
> >
> > 2024年11月13日(水) 21:50 Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <
> > lkotzaniewsk@bloomberg.net>:
> >
> > > Just throwing this out there, have you tried the unifiedHighlighter
> with
> > > termVectors/character offsets? The default configuration re-analyzes
> the
> > > text of the result-set which can be cpu-bound for large docs/fields.
> But
> > > there are ways to get around this by storing the character offsets and
> > > termVectors in the index, which can be a good tradeoff in many
> > situations,
> > > i.e. "Postings"
> > >
> > >
> >
> https://solr.apache.org/guide/solr/latest/query-guide/highlighting.html#:~:text=over%20highlighting%20performance.-,Postings,-%3A%20Supported%20by%20the
> > >
> > > I know this doesn't answer your question but just wanted to mention an
> > > alternative solution to relying on time-outs/partial-results, which may
> > be
> > > frustrating for other reasons.
> > > Sent from Bloomberg Professional for Android
> > >
> > > ----- Original Message -----
> > > From: Mingchun Zhao <us...@solr.apache.org>
> > > To: users@solr.apache.org
> > > At: 11/12/24 21:55:36 UTC-05:00
> > >
> > >
> > > Hi,
> > >
> > > If a query is interrupted due to timeAllowed, does it still return the
> > > partially matched documents and highlight snippets on it? Or does the
> > > behavior vary depending on the Solr version?
> > > In my testing, when the response header includes "partialResults=true",
> > the
> > > partial search results or snippets were not included in the response
> > body.
> > >
> > > Regards,
> > > Zhao
> > >
> >
>
--
Sincerely yours
Mikhail Khludnev
Re: Is it possible to retrieve partial search results and highlight snippets fro
Posted by Mingchun Zhao <mi...@gmail.com>.
Hi Luke,
Thank you for analyzing the code and kindly providing a detailed
explanation.
I now understand that it is not possible to retrieve the partial result set
or its highlighted snippets for a query interrupted due to exceeding the
"timeAllowed" setting value.
I truly appreciate your help once again.
Regards,
Zhao
2024年11月15日(金) 21:46 Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <
lkotzaniewsk@bloomberg.net>:
> From reading the code I don't think it is possible. If your timeout is
> occurring in the core search the SolrIndexSearcher will throw a certain
> type of ExitingReaderException that is caught very high up (in the
> SearchHandler). This happens as the SearchHandler is looping through
> "SearchComponents" one of which is the HighlightComponent. Effectively this
> means the search processing is short-circuited as no downstream components
> get invoked. Not sure if changing this behavior is the right thing since
> highlighting itself can be expensive so the timeout would be misleading. I
> suppose you could consider implementing more granular timeouts but not sure
> about the return on investment of something like that.
>
> Sent from Bloomberg Professional for Android
>
> ----- Original Message -----
> From: Mingchun Zhao <us...@solr.apache.org>
> To: users@solr.apache.org
> At: 11/13/24 20:38:49 UTC-05:00
>
>
> Hi Luke,
>
> Thank you very much for the valuable information. I’ll give the Unified
> Highlighter’s Postings feature a try.
>
> That said, I’d still like to know more about the behavior of the
> "timeAllowed" parameter, especially whether it’s possible to retrieve
> partial search results (both matching documents and highlighted snippets)
> when a query times out.
>
> If you have any information on this, I’d appreciate it.
>
> Regards,
> Zhao
>
> 2024年11月13日(水) 21:50 Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <
> lkotzaniewsk@bloomberg.net>:
>
> > Just throwing this out there, have you tried the unifiedHighlighter with
> > termVectors/character offsets? The default configuration re-analyzes the
> > text of the result-set which can be cpu-bound for large docs/fields. But
> > there are ways to get around this by storing the character offsets and
> > termVectors in the index, which can be a good tradeoff in many
> situations,
> > i.e. "Postings"
> >
> >
> https://solr.apache.org/guide/solr/latest/query-guide/highlighting.html#:~:text=over%20highlighting%20performance.-,Postings,-%3A%20Supported%20by%20the
> >
> > I know this doesn't answer your question but just wanted to mention an
> > alternative solution to relying on time-outs/partial-results, which may
> be
> > frustrating for other reasons.
> > Sent from Bloomberg Professional for Android
> >
> > ----- Original Message -----
> > From: Mingchun Zhao <us...@solr.apache.org>
> > To: users@solr.apache.org
> > At: 11/12/24 21:55:36 UTC-05:00
> >
> >
> > Hi,
> >
> > If a query is interrupted due to timeAllowed, does it still return the
> > partially matched documents and highlight snippets on it? Or does the
> > behavior vary depending on the Solr version?
> > In my testing, when the response header includes "partialResults=true",
> the
> > partial search results or snippets were not included in the response
> body.
> >
> > Regards,
> > Zhao
> >
>
Re: Is it possible to retrieve partial search results and highlight snippets fro
Posted by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net>.
From reading the code I don't think it is possible. If your timeout is occurring in the core search the SolrIndexSearcher will throw a certain type of ExitingReaderException that is caught very high up (in the SearchHandler). This happens as the SearchHandler is looping through "SearchComponents" one of which is the HighlightComponent. Effectively this means the search processing is short-circuited as no downstream components get invoked. Not sure if changing this behavior is the right thing since highlighting itself can be expensive so the timeout would be misleading. I suppose you could consider implementing more granular timeouts but not sure about the return on investment of something like that.
Sent from Bloomberg Professional for Android
----- Original Message -----
From: Mingchun Zhao <us...@solr.apache.org>
To: users@solr.apache.org
At: 11/13/24 20:38:49 UTC-05:00
Hi Luke,
Thank you very much for the valuable information. I’ll give the Unified
Highlighter’s Postings feature a try.
That said, I’d still like to know more about the behavior of the
"timeAllowed" parameter, especially whether it’s possible to retrieve
partial search results (both matching documents and highlighted snippets)
when a query times out.
If you have any information on this, I’d appreciate it.
Regards,
Zhao
2024年11月13日(水) 21:50 Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <
lkotzaniewsk@bloomberg.net>:
> Just throwing this out there, have you tried the unifiedHighlighter with
> termVectors/character offsets? The default configuration re-analyzes the
> text of the result-set which can be cpu-bound for large docs/fields. But
> there are ways to get around this by storing the character offsets and
> termVectors in the index, which can be a good tradeoff in many situations,
> i.e. "Postings"
>
> https://solr.apache.org/guide/solr/latest/query-guide/highlighting.html#:~:text=over%20highlighting%20performance.-,Postings,-%3A%20Supported%20by%20the
>
> I know this doesn't answer your question but just wanted to mention an
> alternative solution to relying on time-outs/partial-results, which may be
> frustrating for other reasons.
> Sent from Bloomberg Professional for Android
>
> ----- Original Message -----
> From: Mingchun Zhao <us...@solr.apache.org>
> To: users@solr.apache.org
> At: 11/12/24 21:55:36 UTC-05:00
>
>
> Hi,
>
> If a query is interrupted due to timeAllowed, does it still return the
> partially matched documents and highlight snippets on it? Or does the
> behavior vary depending on the Solr version?
> In my testing, when the response header includes "partialResults=true", the
> partial search results or snippets were not included in the response body.
>
> Regards,
> Zhao
>
Re: Is it possible to retrieve partial search results and highlight snippets from
Posted by Mingchun Zhao <mi...@gmail.com>.
Hi Luke,
Thank you very much for the valuable information. I’ll give the Unified
Highlighter’s Postings feature a try.
That said, I’d still like to know more about the behavior of the
"timeAllowed" parameter, especially whether it’s possible to retrieve
partial search results (both matching documents and highlighted snippets)
when a query times out.
If you have any information on this, I’d appreciate it.
Regards,
Zhao
2024年11月13日(水) 21:50 Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <
lkotzaniewsk@bloomberg.net>:
> Just throwing this out there, have you tried the unifiedHighlighter with
> termVectors/character offsets? The default configuration re-analyzes the
> text of the result-set which can be cpu-bound for large docs/fields. But
> there are ways to get around this by storing the character offsets and
> termVectors in the index, which can be a good tradeoff in many situations,
> i.e. "Postings"
>
> https://solr.apache.org/guide/solr/latest/query-guide/highlighting.html#:~:text=over%20highlighting%20performance.-,Postings,-%3A%20Supported%20by%20the
>
> I know this doesn't answer your question but just wanted to mention an
> alternative solution to relying on time-outs/partial-results, which may be
> frustrating for other reasons.
> Sent from Bloomberg Professional for Android
>
> ----- Original Message -----
> From: Mingchun Zhao <us...@solr.apache.org>
> To: users@solr.apache.org
> At: 11/12/24 21:55:36 UTC-05:00
>
>
> Hi,
>
> If a query is interrupted due to timeAllowed, does it still return the
> partially matched documents and highlight snippets on it? Or does the
> behavior vary depending on the Solr version?
> In my testing, when the response header includes "partialResults=true", the
> partial search results or snippets were not included in the response body.
>
> Regards,
> Zhao
>