How to see results of an analyzer

[Lahiru Samarakoon <la...@gmail.com>]

Dear All,

How can I see (or print) the result of a lucene analyzer.

I believe the analyzing happens when the *writer.addDocument* or *
writer.updateDocument* methods are called.
Can anyone suggest me a method to see the analyzing process as it happens.

If it is not possible in real time, how can i see the analyzed text ?

Thank you,

Best Regards,
Lahiru.

Re: How to see results of an analyzer

[Lahiru Samarakoon <la...@gmail.com>]

Hi Umesh,

Thanks for the descriptive answer.
Using your method I can test my custom analyzers without creating an index.

Best Regards,
Lahiru

Re: How to see results of an analyzer

[Umesh Prasad <um...@gmail.com>]

For small texts, you can call tokenstream of Analyzer directly. That way you
don't have to crate an index.


http://lucene.apache.org/java/3_0_3/api/core/org/apache/lucene/analysis/Analyzer.html
http://lucene.apache.org/java/3_0_3/api/core/org/apache/lucene/analysis/TokenStream.html

Analyzer analyzer =  // create your analyzer.

TokenSteam ts = analyzer.tokenStream(fieldName, new
StringReader(fieldValue));

TermAttribute termAtt = ts.addAttribute(TermAttribute.class);
OffsetAttribute offsetAtt = ts.addAttribute(OffsetAttribute.class);
PositionIncrementAttribute posAtt =
ts.addAttribute(PositionIncrementAttribute.class);

while(ts.incrementToken())
{
  System.out.println(offset + "\t" + termAtt.term()  + "\t" +
posAtt.getPositionIncrement());
}


Hope that helps.



On Wed, Dec 22, 2010 at 9:28 PM, Lahiru Samarakoon <la...@gmail.com>wrote:

> Hi Ian,
>
> Luke works for me.
>
> Thanks,
> Lahiru.
>



-- 
---
Thanks & Regards
Umesh Prasad

Re: How to see results of an analyzer

[Lahiru Samarakoon <la...@gmail.com>]

Hi Ian,

Luke works for me.

Thanks,
Lahiru.

Re: How to see results of an analyzer

[Ian Lea <ia...@gmail.com>]

I don't believe you can see anything in real time.  Luke will show you
pretty much anything about an index or see
http://www.gossamer-threads.com/lists/lucene/java-user/112180 for a
way to feed text into an analyzer and see what comes out.


--
Ian.


On Wed, Dec 22, 2010 at 12:00 PM, Lahiru Samarakoon <la...@gmail.com> wrote:
> Dear All,
>
> How can I see (or print) the result of a lucene analyzer.
>
> I believe the analyzing happens when the *writer.addDocument* or *
> writer.updateDocument* methods are called.
> Can anyone suggest me a method to see the analyzing process as it happens.
>
> If it is not possible in real time, how can i see the analyzed text ?
>
> Thank you,
>
> Best Regards,
> Lahiru.
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org