You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by TAKAHASHI hideaki <my...@gmail.com> on 2007/12/17 07:30:44 UTC

KeywordTokenizer isn't reusable

Hi, all

I found KeywordAnalyzer/KeywordTokenizer on trunk has a problem.

These have a condition(tokenStreams in Analyzer and done in KeywordTokenizer),
but these don't reset the condition. So KeywordAnalyzer can't analyze
a field more then twice.

I already created a patch for this problem.
Can I send this patch?

Thanks,
Hideaki

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: KeywordTokenizer isn't reusable

Posted by Michael McCandless <lu...@mikemccandless.com>.
Awesome, thanks!  I'll commit this.

Mike

TAKAHASHI hideaki wrote:

> Hi,
> Here is the patch for KeywordAnalyzer, KeywordTokenizer,  
> TestKeywordAnalyzer.
>
> Thanks,
> Hideaki,
>
> On Dec 17, 2007 6:49 PM, Michael McCandless  
> <lu...@mikemccandless.com> wrote:
>>
>> Yes please do!  Thanks.
>>
>> Mike
>>
>>
>> TAKAHASHI hideaki wrote:
>>
>>> Hi, all
>>>
>>> I found KeywordAnalyzer/KeywordTokenizer on trunk has a problem.
>>>
>>> These have a condition(tokenStreams in Analyzer and done in
>>> KeywordTokenizer),
>>> but these don't reset the condition. So KeywordAnalyzer can't  
>>> analyze
>>> a field more then twice.
>>>
>>> I already created a patch for this problem.
>>> Can I send this patch?
>>>
>>> Thanks,
>>> Hideaki
>>>
>>> -------------------------------------------------------------------- 
>>> -
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
>
>
> -- 
> 高橋 秀明 
> <KeywordAnalyzer.patch>----------------------------------------------- 
> ----------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: KeywordTokenizer isn't reusable

Posted by TAKAHASHI hideaki <my...@gmail.com>.
Hi,
Here is the patch for KeywordAnalyzer, KeywordTokenizer, TestKeywordAnalyzer.

Thanks,
Hideaki,

On Dec 17, 2007 6:49 PM, Michael McCandless <lu...@mikemccandless.com> wrote:
>
> Yes please do!  Thanks.
>
> Mike
>
>
> TAKAHASHI hideaki wrote:
>
> > Hi, all
> >
> > I found KeywordAnalyzer/KeywordTokenizer on trunk has a problem.
> >
> > These have a condition(tokenStreams in Analyzer and done in
> > KeywordTokenizer),
> > but these don't reset the condition. So KeywordAnalyzer can't analyze
> > a field more then twice.
> >
> > I already created a patch for this problem.
> > Can I send this patch?
> >
> > Thanks,
> > Hideaki
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-dev-help@lucene.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>



-- 
高橋 秀明

Re: KeywordTokenizer isn't reusable

Posted by Michael McCandless <lu...@mikemccandless.com>.
Yes please do!  Thanks.

Mike

TAKAHASHI hideaki wrote:

> Hi, all
>
> I found KeywordAnalyzer/KeywordTokenizer on trunk has a problem.
>
> These have a condition(tokenStreams in Analyzer and done in  
> KeywordTokenizer),
> but these don't reset the condition. So KeywordAnalyzer can't analyze
> a field more then twice.
>
> I already created a patch for this problem.
> Can I send this patch?
>
> Thanks,
> Hideaki
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org