You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by "Michael Baessler (JIRA)" <ui...@incubator.apache.org> on 2007/01/03 10:07:27 UTC
[jira] Created: (UIMA-151) Add project for uima whitespace
tokenizer implementation
Add project for uima whitespace tokenizer implementation
--------------------------------------------------------
Key: UIMA-151
URL: https://issues.apache.org/jira/browse/UIMA-151
Project: UIMA
Issue Type: New Feature
Components: Sandbox
Affects Versions: 2.1
Reporter: Michael Baessler
Assigned To: Michael Baessler
Fix For: 2.1
Add project for uima whitespace tokenizer implementation
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (UIMA-151) Add project for uima whitespace tokenizer
implementation
Posted by "Michael Baessler (JIRA)" <ui...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/UIMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Baessler closed UIMA-151.
---------------------------------
Resolution: Fixed
> Add project for uima whitespace tokenizer implementation
> --------------------------------------------------------
>
> Key: UIMA-151
> URL: https://issues.apache.org/jira/browse/UIMA-151
> Project: UIMA
> Issue Type: New Feature
> Components: Sandbox
> Affects Versions: 2.1
> Reporter: Michael Baessler
> Assigned To: Michael Baessler
> Fix For: 2.1
>
>
> Add project for uima whitespace tokenizer implementation
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (UIMA-151) Add project for uima whitespace
tokenizer implementation
Posted by "Michael Baessler (JIRA)" <ui...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/UIMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462477 ]
Michael Baessler commented on UIMA-151:
---------------------------------------
Thanks for looking into that topic.
I know the ICU and the BreakIterator stuff but I was not familiar with all the details mentioned in the "Text Boundary Analysis in Java" article. Thanks for that!
My intention when implementing the whitespace tokenizer annotator was to keep things as simple as possible and have no dependencies to other libraries. So that everyone can easily understand the annotator code.
But you are right, it will be good if we could have another implementation using the ICU. We would appreciate if you would like to develop such an annotator in the UIMA sandbox. We are very interested in such a component.
If you are interested please let us know.
Thanks Michael
> Add project for uima whitespace tokenizer implementation
> --------------------------------------------------------
>
> Key: UIMA-151
> URL: https://issues.apache.org/jira/browse/UIMA-151
> Project: UIMA
> Issue Type: New Feature
> Components: Sandbox
> Affects Versions: 2.1
> Reporter: Michael Baessler
> Assigned To: Michael Baessler
> Fix For: 2.1
>
>
> Add project for uima whitespace tokenizer implementation
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (UIMA-151) Add project for uima whitespace
tokenizer implementation
Posted by "Greg Holmberg (JIRA)" <ui...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/UIMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462262 ]
Greg Holmberg commented on UIMA-151:
------------------------------------
May I suggest using the ICU Unicode package: http://icu.sourceforge.net
It already has a whitespace tokenizer for Java that understands all the Unicode conventions (apparently these are not trivial). See the com.ibm.icu.text.BreakIterator and com.ibm.icu.util.StringTokenizer classes. They also have a nice article, "Text Boundary Analysis in Java", here: http://icu.sourceforge.net/docs/papers/text_boundary_analysis_in_java
> Add project for uima whitespace tokenizer implementation
> --------------------------------------------------------
>
> Key: UIMA-151
> URL: https://issues.apache.org/jira/browse/UIMA-151
> Project: UIMA
> Issue Type: New Feature
> Components: Sandbox
> Affects Versions: 2.1
> Reporter: Michael Baessler
> Assigned To: Michael Baessler
> Fix For: 2.1
>
>
> Add project for uima whitespace tokenizer implementation
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (UIMA-151) Add project for uima whitespace
tokenizer implementation
Posted by "Michael Baessler (JIRA)" <ui...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/UIMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Baessler updated UIMA-151:
----------------------------------
Component/s: (was: Sandbox)
Sandbox-WhitespaceTokenizer
> Add project for uima whitespace tokenizer implementation
> --------------------------------------------------------
>
> Key: UIMA-151
> URL: https://issues.apache.org/jira/browse/UIMA-151
> Project: UIMA
> Issue Type: New Feature
> Components: Sandbox-WhitespaceTokenizer
> Reporter: Michael Baessler
> Assignee: Michael Baessler
>
> Add project for uima whitespace tokenizer implementation
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.