[jira] Closed: (LUCENE-324) org.apache.lucene.analysis.cn.ChineseTokenizer missing offset decrement

["Erik Hatcher (JIRA)" <ji...@apache.org>]

     [ http://issues.apache.org/jira/browse/LUCENE-324?page=all ]
     
Erik Hatcher closed LUCENE-324:
-------------------------------

    Assign To:     (was: Erik Hatcher)

> org.apache.lucene.analysis.cn.ChineseTokenizer missing offset decrement
> -----------------------------------------------------------------------
>
>          Key: LUCENE-324
>          URL: http://issues.apache.org/jira/browse/LUCENE-324
>      Project: Lucene - Java
>         Type: Bug
>   Components: Analysis
>     Versions: unspecified
>  Environment: Operating System: All
> Platform: All
>     Reporter: Ray Tsang
>     Priority: Trivial
>      Fix For: 1.9
>  Attachments: ChineseTokenizerTest.java, chinese_tokenizer-missing_offset.patch
>
> Apparently, in ChineseTokenizer, offset should be decremented like bufferIndex
> when Character is OTHER_LETTER.  This directly affects startOffset and endOffset
> values.
> This is critical to have Highlighter working correctly because Highlighter marks
> matching text based on these offset values.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org