You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@opennlp.apache.org by "Jeff Zemerick (Jira)" <ji...@apache.org> on 2022/09/17 19:30:00 UTC

[jira] [Created] (OPENNLP-1387) Fix alphaNumOpt in tokenizer example

Jeff Zemerick created OPENNLP-1387:
--------------------------------------

             Summary: Fix alphaNumOpt in tokenizer example
                 Key: OPENNLP-1387
                 URL: https://issues.apache.org/jira/browse/OPENNLP-1387
             Project: OpenNLP
          Issue Type: Task
          Components: Documentation, Tokenizer
    Affects Versions: 2.0.0, 1.9.4
            Reporter: Jeff Zemerick


The example command for the tokenizer in the documentation is:

opennlp TokenizerTrainer -model en-token.bin -alphaNumOpt -lang en -data en-token.train -encoding UTF-8

This is an invalid command because the alphaNumOpt parameter must have a value. Running this command as-is will give the error "Number of parameters must be always be even." 

The documentation for alphaNumOpt specifies it must have a value:
 -alphaNumOpt isAlphaNumOpt
                Optimization flag to skip alpha numeric tokens for further tokenization



--
This message was sent by Atlassian Jira
(v8.20.10#820010)