You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by "Greg Holmberg (JIRA)" <ui...@incubator.apache.org> on 2007/01/04 18:35:27 UTC
[jira] Commented: (UIMA-151) Add project for uima whitespace
tokenizer implementation
[ https://issues.apache.org/jira/browse/UIMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462262 ]
Greg Holmberg commented on UIMA-151:
------------------------------------
May I suggest using the ICU Unicode package: http://icu.sourceforge.net
It already has a whitespace tokenizer for Java that understands all the Unicode conventions (apparently these are not trivial). See the com.ibm.icu.text.BreakIterator and com.ibm.icu.util.StringTokenizer classes. They also have a nice article, "Text Boundary Analysis in Java", here: http://icu.sourceforge.net/docs/papers/text_boundary_analysis_in_java
> Add project for uima whitespace tokenizer implementation
> --------------------------------------------------------
>
> Key: UIMA-151
> URL: https://issues.apache.org/jira/browse/UIMA-151
> Project: UIMA
> Issue Type: New Feature
> Components: Sandbox
> Affects Versions: 2.1
> Reporter: Michael Baessler
> Assigned To: Michael Baessler
> Fix For: 2.1
>
>
> Add project for uima whitespace tokenizer implementation
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira