[jira] [Created] (OPENNLP-1115) Document Categorizer all events dropped

["Alessandro Depase (JIRA)" <ji...@apache.org>]

Alessandro Depase created OPENNLP-1115:
------------------------------------------

             Summary: Document Categorizer all events dropped
                 Key: OPENNLP-1115
                 URL: https://issues.apache.org/jira/browse/OPENNLP-1115
             Project: OpenNLP
          Issue Type: Question
          Components: Doccat
    Affects Versions: 1.7.2
            Reporter: Alessandro Depase
         Attachments: Train1.train

Hi all,
I'm trying to perform my first (newbie) document categorization using italian language.
I'm using the attached train file and i got this output:

{{$ ./opennlp.bat DoccatTrainer -model it-doccat.bin -lang it -data "C:\Users\adepase\MPSProjects\MrJEditor\languages\MrJEditor\sandbox\source_gen\MrJEditor\sandbox\Train1.train" -encoding UTF-8
Indexing events using cutoff of 5

        Computing event counts...  done. 12 events
        Indexing...  Dropped event Ok:[bow=ok]
Dropped event Ok:[bow=tutto, bow=bene]
Dropped event Ok:[bow=decisamente, bow=non, bow=male]
Dropped event Ok:[bow=fantastica, bow=scelta]
Dropped event Ok:[bow=non, bow=pensavo, bow=di, bow=poter, bow=essere, bow=così, bow=contento]
Dropped event Ok:[bow=certamente, bow=un'ottimo, bow=risultato]
Dropped event no:[bow=non, bow=va, bow=affatto, bow=bene]
Dropped event no:[bow=per, bow=nulla]
Dropped event no:[bow=niente, bow=affatto, bow=divertente]
Dropped event no:[bow=va, bow=malissimo]
Dropped event no:[bow=va, bow=decisamente, bow=male]
Dropped event no:[bow=sono, bow=molto, bow=triste]
done.
Sorting and merging events...

ERROR: Not enough training data
The provided training data is not sufficient to create enough events to train a model.
To resolve this error use more training data, if this doesn't help there might
be some fundamental problem with the training data itself.}}

I already found a couple of other similar issues, just saying that there are not enough lines (but I have 6 lines for each category and a cutoff of 5) or that without at least 100 lines the categorization quality is not sufficient (ok, but that's just a quality matter, it should work, with bad results, but it should work). The reason for insufficient data is that all the lines are dropped.
But why? What did I miss? I cannot find useful documentation...

Thank you in advance
Kind Regards
    Alessandro



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)