display tokens

[s d <s....@gmail.com>]

How can I retrieve the "analyzed tokens" (e.g. the stemmed values) of a
specific field?

Re: display tokens

[Ryan McKinley <ry...@gmail.com>]

Chris Hostetter wrote:
> : Subject: display tokens
> : 
> : How can I retrieve the "analyzed tokens" (e.g. the stemmed values) of a
> : specific field?
> 
> for a field by name independent of documents?  the LukeRequestHandler can 
> give you the top N terms for a field ... but if you mean "i did a search, 
> i found a document, show me the analyzed tokens for that document in this 
> field" there is no easy way to get that information.
> 
> if you have a stored value for that field you can feed it into the 
> analysis.jsp to see what the analyzed tokens are.
> 

also check out faceting.  This returns the analyzed tokens, not the 
stored fields.

ryan

Re: display tokens

[Chris Hostetter <ho...@fucit.org>]

: Subject: display tokens
: 
: How can I retrieve the "analyzed tokens" (e.g. the stemmed values) of a
: specific field?

for a field by name independent of documents?  the LukeRequestHandler can 
give you the top N terms for a field ... but if you mean "i did a search, 
i found a document, show me the analyzed tokens for that document in this 
field" there is no easy way to get that information.

if you have a stored value for that field you can feed it into the 
analysis.jsp to see what the analyzed tokens are.



-Hoss