Finding Issue with NgramAnalyzer in Apache Lucene

[Priyanka Tufchi <pr...@launchship.com>]

Hi all

I am trying to use Apache Lucene  for Ngram Separator.


Reader reader = new StringReader("This is a test string");
NGramTokenizer gramTokenizer = new NGramTokenizer(reader, 1, 3);

CharTermAttribute charTermAttribute =
gramTokenizer.addAttribute(CharTermAttribute.class);
gramTokenizer.reset();

while (gramTokenizer.incrementToken()) {
    String token = charTermAttribute.toString();
    System.out.println(token);
}
gramTokenizer.end();
gramTokenizer.close();

}

This is the code  i used but it is returning character by character , I
want it to return in terms like this ,test , string, this test etc


===================
i tried with shringleFilter also , but it is giving nullpoint exception

*Reader reader = new StringReader("This is a test string");*
*   TokenStream tokenizer = new StandardTokenizer(Version.LUCENE_41,
reader);*
* tokenizer = new ShingleFilter(tokenizer, 2, 3);*
* CharTermAttribute charTermAttribute =
tokenizer.addAttribute(CharTermAttribute.class);*

* while (tokenizer.incrementToken()) {*
*    String token = charTermAttribute.toString();*
*    System.out.println(token);*

* }*

Plz guide


Thanks

-- 
Launchship Technology  respects your privacy. This email is intended only 
for the use of the party to which it is addressed and may contain 
information that is privileged, confidential, or protected by law. If you 
have received this message in error, or do not want to receive any further 
emails from us, please notify us immediately by replying to the message and 
deleting it from your computer.