You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Rick Leir <rl...@leirtech.com> on 2017/04/01 12:49:47 UTC

Re: storing the analyzed value

stored="true"  (the default)

https://cwiki.apache.org/confluence/display/solr/Field+Type+Definitions+and+Properties

On 2017-03-31 01:55 PM, John Blythe wrote:
> hey all
>
> i'm wanting to store one of my field's analyzed token for retrieval. is
> there any way to do this? the preliminary googling i'd done had discussions
> from 2007-2010, i didn't notice anything very recent touching on the
> concept.
>
> thanks-
>


Re: storing the analyzed value

Posted by John Blythe <jo...@curvolabs.com>.
All good info guys. Appreciate the responses.

Some more on the use case: I'm wanting to display the platonic ideal of
sorts for a given set of potential vendor names.

In some cases I can get away w a lookup in our database based on the term
in the document (not its other synonyms) brought back from the search (in
the analytics related query we do, for instance, leveraging the stats
component). But when we show the user the constituent data id like to show
the same platonic ideal. I could do the same lookup mentioned above for
display purposes. But if they want to sort on that column then my tricks
are suddenly exposed and I'm at a loss.

Does that make sense?

On Sat, Apr 1, 2017 at 6:28 PM Joel Bernstein <jo...@gmail.com> wrote:

> You may find this blog interesting.
>
> http://joelsolr.blogspot.com/2017/03/streaming-nlp-is-coming-in-solr-66.html
>
> It deals with how the analyzer chain can now be applied in Streaming
> Expressions. It will be part of the 6.6 release and is in master and
> branch_6x already.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Sat, Apr 1, 2017 at 5:23 PM, Rick Leir <rl...@leirtech.com> wrote:
>
> > On 2017-04-01 10:51 AM, John Blythe wrote:
> >
> >> Hi Rick. I should explain further. I'm not looking to have the input
> >> stored
> >> but rather the final product, specifically the synonym that an input may
> >> be
> >> mapped to.
> >>
> >> If I have McDonald, McD's, and Mac Donald all mapped to "McDonald's" I'd
> >> like to be able to not only access which one was sent to solr for search
> >> (e.g. "McD's") but _also_ the synonym it mapped to: "McDonald's"
> >>
> >> Does this make more sense?
> >>
> >> Thanks for any continued discussion
> >>
> > Shawn answered well, as always.
> > Here is possibly another way. You can enable highlighting, and scan the
> > output to find the passage which is highlighted. But it can be a
> challenge
> > to get highlighting working correctly (I have not yet used the new
> > highlighter).
> > cheers -- Rick
> >
>
-- 
-- 
*John Blythe*
Product Manager & Lead Developer

251.605.3071 | john@curvolabs.com
www.curvolabs.com

58 Adams Ave
Evansville, IN 47713

Re: storing the analyzed value

Posted by Joel Bernstein <jo...@gmail.com>.
You may find this blog interesting.
http://joelsolr.blogspot.com/2017/03/streaming-nlp-is-coming-in-solr-66.html

It deals with how the analyzer chain can now be applied in Streaming
Expressions. It will be part of the 6.6 release and is in master and
branch_6x already.

Joel Bernstein
http://joelsolr.blogspot.com/

On Sat, Apr 1, 2017 at 5:23 PM, Rick Leir <rl...@leirtech.com> wrote:

> On 2017-04-01 10:51 AM, John Blythe wrote:
>
>> Hi Rick. I should explain further. I'm not looking to have the input
>> stored
>> but rather the final product, specifically the synonym that an input may
>> be
>> mapped to.
>>
>> If I have McDonald, McD's, and Mac Donald all mapped to "McDonald's" I'd
>> like to be able to not only access which one was sent to solr for search
>> (e.g. "McD's") but _also_ the synonym it mapped to: "McDonald's"
>>
>> Does this make more sense?
>>
>> Thanks for any continued discussion
>>
> Shawn answered well, as always.
> Here is possibly another way. You can enable highlighting, and scan the
> output to find the passage which is highlighted. But it can be a challenge
> to get highlighting working correctly (I have not yet used the new
> highlighter).
> cheers -- Rick
>

Re: storing the analyzed value

Posted by Rick Leir <rl...@leirtech.com>.
On 2017-04-01 10:51 AM, John Blythe wrote:
> Hi Rick. I should explain further. I'm not looking to have the input stored
> but rather the final product, specifically the synonym that an input may be
> mapped to.
>
> If I have McDonald, McD's, and Mac Donald all mapped to "McDonald's" I'd
> like to be able to not only access which one was sent to solr for search
> (e.g. "McD's") but _also_ the synonym it mapped to: "McDonald's"
>
> Does this make more sense?
>
> Thanks for any continued discussion
Shawn answered well, as always.
Here is possibly another way. You can enable highlighting, and scan the 
output to find the passage which is highlighted. But it can be a 
challenge to get highlighting working correctly (I have not yet used the 
new highlighter).
cheers -- Rick

Re: storing the analyzed value

Posted by Shawn Heisey <ap...@elyograg.org>.
On 4/1/2017 8:51 AM, John Blythe wrote:
> Hi Rick. I should explain further. I'm not looking to have the input stored
> but rather the final product, specifically the synonym that an input may be
> mapped to.
>
> If I have McDonald, McD's, and Mac Donald all mapped to "McDonald's" I'd
> like to be able to not only access which one was sent to solr for search
> (e.g. "McD's") but _also_ the synonym it mapped to: "McDonald's"

Thought of an alternate interpretation of what you asked about -- info
on the *query* side, not the *index* side.  This addresses that
alternate interpretation:

If you add "&debugQuery=true" to the URL, you'll see a lot of debug
information.  One piece of useful information there is the original
query, and the *parsed* query.  By comparing those, you would be able to
detect the synonyms applied at query time.

There may also be a "debug=XXXX" option that would only generate the
parsed query without spending the resources to generate the rest of the
debug info, but I am not familiar with all the debug options.  I know
that debugQuery=true will include that information, but the full debug
might cause your queries to become unacceptably slow.  There is a LOT of
info generated by the generic debugQuery.

Thanks,
Shawn


Re: storing the analyzed value

Posted by Shawn Heisey <ap...@elyograg.org>.
On 4/1/2017 8:51 AM, John Blythe wrote:
> Hi Rick. I should explain further. I'm not looking to have the input stored
> but rather the final product, specifically the synonym that an input may be
> mapped to.
>
> If I have McDonald, McD's, and Mac Donald all mapped to "McDonald's" I'd
> like to be able to not only access which one was sent to solr for search
> (e.g. "McD's") but _also_ the synonym it mapped to: "McDonald's"
>
> Does this make more sense?
>
> Thanks for any continued discussion

Generally speaking, search results contain EXACTLY the information that
was submitted for indexing, assuming that the value was stored.  In
newer versions, docValues can be used instead of stored in order to have
the field appear in results.  Index analysis only affects the search
index, what happens there will never show up in search results.  This is
a fundamental aspect of Solr's operation.

You *can* use a custom UpdateRequestProcessor.  For what you're talking
about, that would probably be a custom processor that you write yourself
and inject into Solr as a plugin.  Update processors are executed on the
indexed data, *before* it reaches the actual indexing part of Solr ...
so stored/docValues data is also affected.

https://cwiki.apache.org/confluence/display/solr/Update+Request+Processors

Thanks,
Shawn


Re: storing the analyzed value

Posted by John Blythe <jo...@curvolabs.com>.
Hi Rick. I should explain further. I'm not looking to have the input stored
but rather the final product, specifically the synonym that an input may be
mapped to.

If I have McDonald, McD's, and Mac Donald all mapped to "McDonald's" I'd
like to be able to not only access which one was sent to solr for search
(e.g. "McD's") but _also_ the synonym it mapped to: "McDonald's"

Does this make more sense?

Thanks for any continued discussion

On Sat, Apr 1, 2017 at 8:50 AM Rick Leir <rl...@leirtech.com> wrote:

> stored="true"  (the default)
>
>
> https://cwiki.apache.org/confluence/display/solr/Field+Type+Definitions+and+Properties
>
> On 2017-03-31 01:55 PM, John Blythe wrote:
> > hey all
> >
> > i'm wanting to store one of my field's analyzed token for retrieval. is
> > there any way to do this? the preliminary googling i'd done had
> discussions
> > from 2007-2010, i didn't notice anything very recent touching on the
> > concept.
> >
> > thanks-
> >
>
> --
-- 
*John Blythe*
Product Manager & Lead Developer

251.605.3071 | john@curvolabs.com
www.curvolabs.com

58 Adams Ave
Evansville, IN 47713