You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@opennlp.apache.org by "Ajay Bhat (JIRA)" <ji...@apache.org> on 2014/05/03 19:17:15 UTC

[jira] [Commented] (OPENNLP-327) Doccats bag of word feature generator should not use numbers as features

    [ https://issues.apache.org/jira/browse/OPENNLP-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988754#comment-13988754 ] 

Ajay Bhat commented on OPENNLP-327:
-----------------------------------

How should numbers be dealt with? Should they be replaced with whitespaces?

> Doccats bag of word feature generator should not use numbers as features
> ------------------------------------------------------------------------
>
>                 Key: OPENNLP-327
>                 URL: https://issues.apache.org/jira/browse/OPENNLP-327
>             Project: OpenNLP
>          Issue Type: Improvement
>          Components: Doccat
>            Reporter: Joern Kottmann
>            Assignee: Joern Kottmann
>            Priority: Minor
>
> It turned out that Doccats bag of word feature generator can be very sensitive to numbers when used for language identification. Therefore numbers should not be included in the bag of words.



--
This message was sent by Atlassian JIRA
(v6.2#6252)