You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Robert Haycock <Ro...@mediasurface.com> on 2006/06/15 13:30:18 UTC

CJKAnalyzer - does it work?

Hi,

I have a very simple example.  An IndexWriter (Lucene 1.9.0) with
CJKAnalyzer (latest version as of today).  A Chinese friend of mine as
given me a sentence and a word that appears in that sentence, eg:

"C1C2C3C4C5C6C7C8" where the word is "C3C4".

Here's code segment:

IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer(),
true);
Document doc = new Document();
doc.add(new Field("name", " C1C2C3C4C5C6C7C8", Store.YES,
Index.TOKENIZED));
writer.addDocument(doc);
writer.optimize();
writer.close();

IndexSearcher searcher = new IndexSearcher(directory);
Hits hits = searcher.search(new TermQuery(new Term("name", "C3C4")));


This returns no hits.  I've also tried with the ChineseAnalyzer too, but
still no good.  It still works fine for English though.

Anyone know how to get this working?

Rob.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: CJKAnalyzer - does it work?

Posted by wu fox <fo...@gmail.com>.
 Erik is very humorous. Erik 很幽默

>
>

Re: CJKAnalyzer - does it work?

Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Jun 15, 2006, at 7:29 PM, Ray Tsang wrote:
> Where did you get that chinese sentence from?  That's funny! haha.

我不知道! ;)

	Erik


> On 6/15/06, Erik Hatcher <er...@ehatchersolutions.com> wrote:
>> Rob,
>>
>> Your example is, hopefully, not exact since you used "C1..." which I
>> presume was not what you originally tested with.
>>
>> CJKAnalyzer is working fine for me in this example adapted from your
>> code:
>>
>>    public void testCJKAnalyzer() throws Exception {
>>      RAMDirectory directory = new RAMDirectory();
>>      IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer
>> (), true);
>>      Document doc = new Document();
>>      doc.add(new Field("name", "脱下你的裤子", Field.Store.YES,
>>          Field.Index.TOKENIZED));
>>      writer.addDocument(doc);
>>      writer.optimize();
>>      writer.close();
>>
>>      IndexSearcher searcher = new IndexSearcher(directory);
>>      Hits hits = searcher.search(new TermQuery(new Term("name", "裤
>> 子")));
>>      assertEquals(1, hits.length());
>>    }
>>
>> Erik
>>
>>
>> On Jun 15, 2006, at 7:30 AM, Robert Haycock wrote:
>>
>> > Hi,
>> >
>> > I have a very simple example.  An IndexWriter (Lucene 1.9.0) with
>> > CJKAnalyzer (latest version as of today).  A Chinese friend of  
>> mine as
>> > given me a sentence and a word that appears in that sentence, eg:
>> >
>> > "C1C2C3C4C5C6C7C8" where the word is "C3C4".
>> >
>> > Here's code segment:
>> >
>> > IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer(),
>> > true);
>> > Document doc = new Document();
>> > doc.add(new Field("name", " C1C2C3C4C5C6C7C8", Store.YES,
>> > Index.TOKENIZED));
>> > writer.addDocument(doc);
>> > writer.optimize();
>> > writer.close();
>> >
>> > IndexSearcher searcher = new IndexSearcher(directory);
>> > Hits hits = searcher.search(new TermQuery(new Term("name",  
>> "C3C4")));
>> >
>> >
>> > This returns no hits.  I've also tried with the ChineseAnalyzer
>> > too, but
>> > still no good.  It still works fine for English though.
>> >
>> > Anyone know how to get this working?
>> >
>> > Rob.
>> >
>> >  
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: CJKAnalyzer - does it work?

Posted by Ray Tsang <sa...@gmail.com>.
Hi Erik,

Where did you get that chinese sentence from?  That's funny! haha.

ray,

On 6/15/06, Erik Hatcher <er...@ehatchersolutions.com> wrote:
> Rob,
>
> Your example is, hopefully, not exact since you used "C1..." which I
> presume was not what you originally tested with.
>
> CJKAnalyzer is working fine for me in this example adapted from your
> code:
>
>    public void testCJKAnalyzer() throws Exception {
>      RAMDirectory directory = new RAMDirectory();
>      IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer
> (), true);
>      Document doc = new Document();
>      doc.add(new Field("name", "脱下你的裤子", Field.Store.YES,
>          Field.Index.TOKENIZED));
>      writer.addDocument(doc);
>      writer.optimize();
>      writer.close();
>
>      IndexSearcher searcher = new IndexSearcher(directory);
>      Hits hits = searcher.search(new TermQuery(new Term("name", "裤
> 子")));
>      assertEquals(1, hits.length());
>    }
>
> Erik
>
>
> On Jun 15, 2006, at 7:30 AM, Robert Haycock wrote:
>
> > Hi,
> >
> > I have a very simple example.  An IndexWriter (Lucene 1.9.0) with
> > CJKAnalyzer (latest version as of today).  A Chinese friend of mine as
> > given me a sentence and a word that appears in that sentence, eg:
> >
> > "C1C2C3C4C5C6C7C8" where the word is "C3C4".
> >
> > Here's code segment:
> >
> > IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer(),
> > true);
> > Document doc = new Document();
> > doc.add(new Field("name", " C1C2C3C4C5C6C7C8", Store.YES,
> > Index.TOKENIZED));
> > writer.addDocument(doc);
> > writer.optimize();
> > writer.close();
> >
> > IndexSearcher searcher = new IndexSearcher(directory);
> > Hits hits = searcher.search(new TermQuery(new Term("name", "C3C4")));
> >
> >
> > This returns no hits.  I've also tried with the ChineseAnalyzer
> > too, but
> > still no good.  It still works fine for English though.
> >
> > Anyone know how to get this working?
> >
> > Rob.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Re: CJKAnalyzer - does it work?

Posted by Erik Hatcher <er...@ehatchersolutions.com>.
Rob,

Your example is, hopefully, not exact since you used "C1..." which I  
presume was not what you originally tested with.

CJKAnalyzer is working fine for me in this example adapted from your  
code:

   public void testCJKAnalyzer() throws Exception {
     RAMDirectory directory = new RAMDirectory();
     IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer 
(), true);
     Document doc = new Document();
     doc.add(new Field("name", "脱下你的裤子", Field.Store.YES,
         Field.Index.TOKENIZED));
     writer.addDocument(doc);
     writer.optimize();
     writer.close();

     IndexSearcher searcher = new IndexSearcher(directory);
     Hits hits = searcher.search(new TermQuery(new Term("name", "裤 
子")));
     assertEquals(1, hits.length());
   }

Erik


On Jun 15, 2006, at 7:30 AM, Robert Haycock wrote:

> Hi,
>
> I have a very simple example.  An IndexWriter (Lucene 1.9.0) with
> CJKAnalyzer (latest version as of today).  A Chinese friend of mine as
> given me a sentence and a word that appears in that sentence, eg:
>
> "C1C2C3C4C5C6C7C8" where the word is "C3C4".
>
> Here's code segment:
>
> IndexWriter writer = new IndexWriter(directory, new CJKAnalyzer(),
> true);
> Document doc = new Document();
> doc.add(new Field("name", " C1C2C3C4C5C6C7C8", Store.YES,
> Index.TOKENIZED));
> writer.addDocument(doc);
> writer.optimize();
> writer.close();
>
> IndexSearcher searcher = new IndexSearcher(directory);
> Hits hits = searcher.search(new TermQuery(new Term("name", "C3C4")));
>
>
> This returns no hits.  I've also tried with the ChineseAnalyzer  
> too, but
> still no good.  It still works fine for English though.
>
> Anyone know how to get this working?
>
> Rob.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org