You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Stanley Xinlei Wang <st...@csail.mit.edu> on 2008/02/26 21:31:37 UTC
Question: Using Shingle Analyzer NGramAnalyzerWrapper in Lucene
Hi,
In Lucene, I'm trying to perform word-level bi-gram query parsing using
NGramAnalyzerWrapper. I'm couldn't get any word pairs in the parsed query
and I was wondering what I should do to make this work. I'm using Lucene
2.2.0
I'm using the files from: https://issues.apache.org/jira/browse/lucene-400
My code is:
NGramAnalyzerWrapper analyzer = new NGramAnalyzerWrapper(new
StandardAnalyzer(), 2);
QueryParser p = new QueryParser("info", analyzer);
Query query = lyricsParser.parse("this is my query");
System.out.println(query.toString());
When I run this, I get the following output, which has no bi-grams:
lyrics:this lyrics:is lyrics:a lyrics:query
Thank you for your help!
Stanley
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Question: Using Shingle Analyzer NGramAnalyzerWrapper in Lucene
Posted by Steven A Rowe <sa...@syr.edu>.
Hi Stanley,
I modernized the files in LUCENE-400 a bit - you can see the details in comments I made on the issue. The results, including all files needed to address the issue, are in the file attached to the issue named "LUCENE-400.patch".
I can tell you aren't using the modernized version because I changed the file names, from NGram* to Shingle*, to avoid conflict with the pre-existing ngram* stuff.
I don't know, however, whether the new patch will apply and build cleanly in the 2.2.0 code base, since it was targetted at the trunk circa version 2.3.
Take a look at ShingleAnalyzerWrapperTest.java in the new patch - like you did, the tests there wrap WhitespaceAnalyser.
If switching to the new code doesn't fix the issue for you, could you add a comment to the issue describing your inputs, and expected and actual outputs? Or, even better, write a unit test to cover your case.
Thanks,
Steve
On 02/26/2008 at 3:39 PM, Stanley Xinlei Wang wrote:
> Sorry slight correction for the code below: I was actually using the
> WhitespaceAnalyzer, not the StandardAnalyzer in constructing the
> NGramAnalyzerWrapper.
>
> On Tue, 26 Feb 2008, Stanley Xinlei Wang wrote:
>
> > Hi,
> >
> > In Lucene, I'm trying to perform word-level bi-gram query parsing using
> > NGramAnalyzerWrapper. I'm couldn't get any word pairs in the parsed
> > query and I was wondering what I should do to make this work. I'm using
> > Lucene 2.2.0
> >
> > I'm using the files from:
> > https://issues.apache.org/jira/browse/lucene-400
> >
> > My code is:
> >
> > NGramAnalyzerWrapper analyzer = new NGramAnalyzerWrapper(new
> > StandardAnalyzer(), 2);
> > QueryParser p = new QueryParser("info", analyzer);
> > Query query = lyricsParser.parse("this is my query");
> > System.out.println(query.toString());
> >
> > When I run this, I get the following output, which has no bi-grams:
> >
> > lyrics:this lyrics:is lyrics:a lyrics:query
> >
> >
> > Thank you for your help!
> >
> > Stanley
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Question: Using Shingle Analyzer NGramAnalyzerWrapper in Lucene
Posted by Stanley Xinlei Wang <st...@csail.mit.edu>.
Sorry slight correction for the code below: I was actually using the
WhitespaceAnalyzer, not the StandardAnalyzer in constructing the
NGramAnalyzerWrapper.
On Tue, 26 Feb 2008, Stanley Xinlei Wang wrote:
> Hi,
>
> In Lucene, I'm trying to perform word-level bi-gram query parsing using
> NGramAnalyzerWrapper. I'm couldn't get any word pairs in the parsed query
> and I was wondering what I should do to make this work. I'm using Lucene
> 2.2.0
>
> I'm using the files from: https://issues.apache.org/jira/browse/lucene-400
>
> My code is:
>
> NGramAnalyzerWrapper analyzer = new NGramAnalyzerWrapper(new
> StandardAnalyzer(), 2);
> QueryParser p = new QueryParser("info", analyzer);
> Query query = lyricsParser.parse("this is my query");
> System.out.println(query.toString());
>
> When I run this, I get the following output, which has no bi-grams:
>
> lyrics:this lyrics:is lyrics:a lyrics:query
>
>
> Thank you for your help!
>
> Stanley
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org