You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Robert Haycock <Ro...@mediasurface.com> on 2006/06/15 13:30:18 UTC
CJKAnalyzer - does it work?
Hi,
I have a very simple example. An IndexWriter (Lucene 1.9.0) with
CJKAnalyzer (latest version as of today). A Chinese friend of mine as
given me a sentence and a word that appears in that sentence, eg:
"C1C2C3C4C5C6C7C8" where the word is "C3C4".
Here's code segment:
IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer(),
true);
Document doc = new Document();
doc.add(new Field("name", " C1C2C3C4C5C6C7C8", Store.YES,
Index.TOKENIZED));
writer.addDocument(doc);
writer.optimize();
writer.close();
IndexSearcher searcher = new IndexSearcher(directory);
Hits hits = searcher.search(new TermQuery(new Term("name", "C3C4")));
This returns no hits. I've also tried with the ChineseAnalyzer too, but
still no good. It still works fine for English though.
Anyone know how to get this working?
Rob.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: CJKAnalyzer - does it work?
Posted by wu fox <fo...@gmail.com>.
Erik is very humorous. Erik 很幽默
>
>
Re: CJKAnalyzer - does it work?
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Jun 15, 2006, at 7:29 PM, Ray Tsang wrote:
> Where did you get that chinese sentence from? That's funny! haha.
我不知道! ;)
Erik
> On 6/15/06, Erik Hatcher <er...@ehatchersolutions.com> wrote:
>> Rob,
>>
>> Your example is, hopefully, not exact since you used "C1..." which I
>> presume was not what you originally tested with.
>>
>> CJKAnalyzer is working fine for me in this example adapted from your
>> code:
>>
>> public void testCJKAnalyzer() throws Exception {
>> RAMDirectory directory = new RAMDirectory();
>> IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer
>> (), true);
>> Document doc = new Document();
>> doc.add(new Field("name", "脱下你的裤子", Field.Store.YES,
>> Field.Index.TOKENIZED));
>> writer.addDocument(doc);
>> writer.optimize();
>> writer.close();
>>
>> IndexSearcher searcher = new IndexSearcher(directory);
>> Hits hits = searcher.search(new TermQuery(new Term("name", "裤
>> 子")));
>> assertEquals(1, hits.length());
>> }
>>
>> Erik
>>
>>
>> On Jun 15, 2006, at 7:30 AM, Robert Haycock wrote:
>>
>> > Hi,
>> >
>> > I have a very simple example. An IndexWriter (Lucene 1.9.0) with
>> > CJKAnalyzer (latest version as of today). A Chinese friend of
>> mine as
>> > given me a sentence and a word that appears in that sentence, eg:
>> >
>> > "C1C2C3C4C5C6C7C8" where the word is "C3C4".
>> >
>> > Here's code segment:
>> >
>> > IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer(),
>> > true);
>> > Document doc = new Document();
>> > doc.add(new Field("name", " C1C2C3C4C5C6C7C8", Store.YES,
>> > Index.TOKENIZED));
>> > writer.addDocument(doc);
>> > writer.optimize();
>> > writer.close();
>> >
>> > IndexSearcher searcher = new IndexSearcher(directory);
>> > Hits hits = searcher.search(new TermQuery(new Term("name",
>> "C3C4")));
>> >
>> >
>> > This returns no hits. I've also tried with the ChineseAnalyzer
>> > too, but
>> > still no good. It still works fine for English though.
>> >
>> > Anyone know how to get this working?
>> >
>> > Rob.
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: CJKAnalyzer - does it work?
Posted by Ray Tsang <sa...@gmail.com>.
Hi Erik,
Where did you get that chinese sentence from? That's funny! haha.
ray,
On 6/15/06, Erik Hatcher <er...@ehatchersolutions.com> wrote:
> Rob,
>
> Your example is, hopefully, not exact since you used "C1..." which I
> presume was not what you originally tested with.
>
> CJKAnalyzer is working fine for me in this example adapted from your
> code:
>
> public void testCJKAnalyzer() throws Exception {
> RAMDirectory directory = new RAMDirectory();
> IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer
> (), true);
> Document doc = new Document();
> doc.add(new Field("name", "脱下你的裤子", Field.Store.YES,
> Field.Index.TOKENIZED));
> writer.addDocument(doc);
> writer.optimize();
> writer.close();
>
> IndexSearcher searcher = new IndexSearcher(directory);
> Hits hits = searcher.search(new TermQuery(new Term("name", "裤
> 子")));
> assertEquals(1, hits.length());
> }
>
> Erik
>
>
> On Jun 15, 2006, at 7:30 AM, Robert Haycock wrote:
>
> > Hi,
> >
> > I have a very simple example. An IndexWriter (Lucene 1.9.0) with
> > CJKAnalyzer (latest version as of today). A Chinese friend of mine as
> > given me a sentence and a word that appears in that sentence, eg:
> >
> > "C1C2C3C4C5C6C7C8" where the word is "C3C4".
> >
> > Here's code segment:
> >
> > IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer(),
> > true);
> > Document doc = new Document();
> > doc.add(new Field("name", " C1C2C3C4C5C6C7C8", Store.YES,
> > Index.TOKENIZED));
> > writer.addDocument(doc);
> > writer.optimize();
> > writer.close();
> >
> > IndexSearcher searcher = new IndexSearcher(directory);
> > Hits hits = searcher.search(new TermQuery(new Term("name", "C3C4")));
> >
> >
> > This returns no hits. I've also tried with the ChineseAnalyzer
> > too, but
> > still no good. It still works fine for English though.
> >
> > Anyone know how to get this working?
> >
> > Rob.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: CJKAnalyzer - does it work?
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
Rob,
Your example is, hopefully, not exact since you used "C1..." which I
presume was not what you originally tested with.
CJKAnalyzer is working fine for me in this example adapted from your
code:
public void testCJKAnalyzer() throws Exception {
RAMDirectory directory = new RAMDirectory();
IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer
(), true);
Document doc = new Document();
doc.add(new Field("name", "脱下你的裤子", Field.Store.YES,
Field.Index.TOKENIZED));
writer.addDocument(doc);
writer.optimize();
writer.close();
IndexSearcher searcher = new IndexSearcher(directory);
Hits hits = searcher.search(new TermQuery(new Term("name", "裤
子")));
assertEquals(1, hits.length());
}
Erik
On Jun 15, 2006, at 7:30 AM, Robert Haycock wrote:
> Hi,
>
> I have a very simple example. An IndexWriter (Lucene 1.9.0) with
> CJKAnalyzer (latest version as of today). A Chinese friend of mine as
> given me a sentence and a word that appears in that sentence, eg:
>
> "C1C2C3C4C5C6C7C8" where the word is "C3C4".
>
> Here's code segment:
>
> IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer(),
> true);
> Document doc = new Document();
> doc.add(new Field("name", " C1C2C3C4C5C6C7C8", Store.YES,
> Index.TOKENIZED));
> writer.addDocument(doc);
> writer.optimize();
> writer.close();
>
> IndexSearcher searcher = new IndexSearcher(directory);
> Hits hits = searcher.search(new TermQuery(new Term("name", "C3C4")));
>
>
> This returns no hits. I've also tried with the ChineseAnalyzer
> too, but
> still no good. It still works fine for English though.
>
> Anyone know how to get this working?
>
> Rob.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org