You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@opennlp.apache.org by Cohan Sujay Carlos <co...@aiaioo.com> on 2015/07/14 17:05:49 UTC
Re: Are there plans to offer a Naive Bayesian Classifier in OpenNLP ?
The Naive Bayes classifier is ready!
I have created a patch (over OpenNLP 1.6.0 rc6).
The patch is attached to the issue we opened (issue 777) on Jira for this
feature:
https://issues.apache.org/jira/browse/OPENNLP-777
Here are a couple of scripts you can use to exercise the Naive Bayes
classifier.
You can train a model as follows:
public class D1TopicClassifierTrainingDemoNB {
public static void main(String[] args) {
DoccatModel model = null;
InputStream dataIn = null;
try {
dataIn = new FileInputStream("corpora/topics/topics.train");
ObjectStream<String> lineStream =
new PlainTextByLineStream(dataIn, "UTF-8");
ObjectStream<DocumentSample> sampleStream = new
DocumentSampleStream(lineStream);
model = DocumentCategorizerNB.train("en", sampleStream);
}
catch (IOException e) {
// Failed to read or parse training data, training failed
e.printStackTrace();
}
finally {
if (dataIn != null) {
try {
dataIn.close();
}
catch (IOException e) {
// Not an issue, training already finished.
// The exception should be logged and investigated
// if part of a production system.
e.printStackTrace();
}
}
}
String modelFile = "models/topics_nb.bin";
OutputStream modelOut = null;
try {
modelOut = new BufferedOutputStream(new FileOutputStream(modelFile));
model.serialize(modelOut);
}
catch (IOException e) {
// Failed to save model
e.printStackTrace();
}
finally {
if (modelOut != null) {
try {
modelOut.close();
}
catch (IOException e) {
// Failed to correctly save model.
// Written model might be invalid.
e.printStackTrace();
}
}
}
}
}
The training data is also attached ('topics.train') to the Jira issue.
This training data has to be placed in a directory named '*corpora/topics*'
under the current directory.
This will create a model file named 'topics_nb.bin' in the '*models' *folder
also under the current directory.
The following script let's you run some classification tasks using the
above model:
public class D1TopicClassifierUsageDemoNB {
public static void main(String[] args) {
String paragraph = "Rumours before the game suggested the Portuguese would
be out at the end of the season if Inter failed to progress but in the end
there was little to worry about as goals from Samuel Eto'o and Mario
Balotelli ensured a comfortable night.";
// always start with a model, a model is learned from training data
InputStream is = null;
try {
is = new FileInputStream("models/topics_nb.bin");
DoccatModel model = new DoccatModel(is);
AbstractModel internalModel = (AbstractModel)model.getMaxentModel();
System.out.println("ModelType: "+internalModel.getModelType());
System.out.println("Model Outcomes: ");
Object[] data = internalModel.getDataStructures();
for (String val : (String[])internalModel.getDataStructures()[2]) {
System.out.println(val);
}
IndexHashTable<String> pmap = (IndexHashTable<String>) data[1];
//String[] PRED_LABELS = new String[pmap.size()];
//pmap.toArray(PRED_LABELS);
//Context[] contexts = (Context[])data[0];
//System.out.println("Pred labels: ");
//for (String label : PRED_LABELS) {
// System.out.println(label + " " + pmap.get(label) + " " +
contexts[pmap.get(label)].getOutcomes().length + " " +
contexts[pmap.get(label)].getOutcomes()[0] + " " +
contexts[pmap.get(label)].getParameters()[0]);
//}
System.out.println("Running the classifier: ");
DocumentCategorizerNB categorizer = new DocumentCategorizerNB(model);
double[] results = categorizer.categorize(paragraph);
String bestResult = categorizer.getBestCategory(results);
System.out.println(bestResult);
results = categorizer.categorize("government");
bestResult = categorizer.getBestCategory(results);
System.out.println(bestResult);
results = categorizer.categorize("");
bestResult = categorizer.getBestCategory(results);
System.out.println(bestResult);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (InvalidFormatException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (is != null) {
try {
is.close();
}
catch (IOException e) {
}
}
}
}
}
The outcome looks mathematically correct.
But I haven't done any exhaustive testing.
If you can take a look at the organization of this code, and ok the same,
I'll proceed with creating the verification scripts and test-cases for it.
Warm regards,
Cohan Sujay Carlos
CEO, Aiaioo Labs
+91-77605-80015 +91-80-4125-0730
On Tue, May 19, 2015 at 7:21 PM, Cohan Sujay Carlos <co...@aiaioo.com>
wrote:
> Tommaso,
>
> I have created the Jira issue:
> https://issues.apache.org/jira/browse/OPENNLP-777
>
> The details of the Java version compatibility and the classifier's
> internals are as follows:
>
> "Implementation details: We have a production-hardened piece of Java code
> for a multinomial Naive Bayesian classifier (with default Laplace
> smoothing) that we'd like to contribute. The code is Java 1.5 compatible.
> I'd have to write an adapter to make the interface compatible with the ME
> classifier in OpenNLP. I expect the patch to be available 1 to 3 weeks from
> now."
>
> This is the default configuration but the code is well-refactored and you
> can actually plug in any smoothing algorithm and any feature set. It also
> has some support for succinct memory models, and I later plan to add a
> multivariate bernoulli implementation as well (I wanted to start with the
> multinomial version because the advantages of the multinomial model will
> make it the better performer for most NLP projects).
>
> I could not figure out how to assign the issue to myself. The patch will
> be available 1 to 3 weeks from now.
>
> Thanks and regards,
>
> Cohan Sujay Carlos
>
>
> On Tue, May 19, 2015 at 5:26 PM, Tommaso Teofili <
> tommaso.teofili@gmail.com> wrote:
>
>> Hi Cohan,
>>
>> I think that'd be a very valuable contribution, as NB is one of the
>> foundation algorithms, often used as basis for comparisons.
>> It would be good if you could create a Jira issue and provide more details
>> about the implementation and, eventually, a patch.
>>
>> Thanks and regards,
>> Tommaso
>>
>> 2015-05-19 9:57 GMT+02:00 Cohan Sujay Carlos <co...@gmail.com>:
>>
>> > I have a question for the OpenNLP project team.
>> >
>> > I was wondering if there is a Naive Bayesian classifier implementation
>> in
>> > OpenNLP that I've not come across, or if there are plans to implement
>> one.
>> >
>> > If it is the latter, I should love to contribute an implementation.
>> >
>> > There is an ME classifier already available in OpenNLP, of course, but I
>> > felt that there was an unmet need for a Naive Bayesian (NB) classifier
>> > implementation to be offered as well.
>> >
>> > An NB classifier could be bootstrapped up with partially labelled
>> training
>> > data as explained in the Nigam, McCallum, et al paper of 2000 "Text
>> > Classification from Labeled and Unlabeled Documents using EM".
>> >
>> > So, if there isn't an NB code base out there already, I'd be happy to
>> > contribute a very solid implementation that we've used in production
>> for a
>> > good 5 years.
>> >
>> > I'd have to adapt it to load the same training data format as the ME
>> > classifier, but I guess that shouldn't be very difficult to do.
>> >
>> > I was wondering if there was some interest in adding an NB
>> implementation
>> > and I'd love to know who could I coordinate with if there is?
>> >
>> > Cohan Sujay Carlos
>> > CEO, Aiaioo Labs, India
>> > +91-77605-80015 +91-80-4125-0730
>> >
>>
>
>
Re: Are there plans to offer a Naive Bayesian Classifier in OpenNLP ?
Posted by Cohan Sujay Carlos <co...@aiaioo.com>.
Thanks Tommaso.
The latest patch is attached to the Jira issue as well (and I see I needn't
have attached it in these email messages).
(I have also emailed a copy of the completed ICLA to Joern).
Cohan
On Mon, Jul 20, 2015 at 1:44 PM, Tommaso Teofili <to...@gmail.com>
wrote:
> Hi Cohan,
>
> for future reference it'd be better if you could attach the patch to the
> Jira issue, I can have a look at your patch and comment later this week.
> Usually we mark an issue as fixed as soon as the relevant code is committed.
>
> Regards,
> Tommaso
>
>
> 2015-07-18 15:36 GMT+02:00 Cohan Sujay Carlos <co...@aiaioo.com>:
>
>> Yet another patch (ouch, yeah, like they say, "in product development,
>> you're never done, till you're done!").
>>
>> Reason: Error in one of the test-cases. The test-case
>> DocumentCategorizerNBTest is now correct and runs fine.
>>
>> Cohan Sujay Carlos
>> CEO, Aiaioo Labs
>> +91-77605-80015
>>
>> On Sat, Jul 18, 2015 at 6:51 PM, Cohan Sujay Carlos <co...@aiaioo.com>
>> wrote:
>>
>>> A small update to the patch (I removed a superfluous piece of code).
>>>
>>> In the earlier path, I had used a subclass of
>>> opennlp.tools.doccat.DoccatModel called opennlp.tools.doccat.DoccatModelNB
>>> that was functionally identical. I removed that subclass since it
>>> wasn't essential (DoccatModel does the trick just fine).
>>>
>>> Is there anything else I need to do?
>>>
>>> Is someone on the dev team going to be responsible for incorporating the
>>> patch into the codebase?
>>>
>>> Can I mark this Jira issue fixed (for target 1.6.1?).
>>>
>>> Cohan Sujay Carlos
>>> CEO, Aiaioo Labs
>>> +91-77605-80015
>>>
>>>
>>> On Sat, Jul 18, 2015 at 6:02 PM, Cohan Sujay Carlos <co...@aiaioo.com>
>>> wrote:
>>>
>>>> I have gone ahead and written the test-cases and verified that the
>>>> Naive Bayes Classifier works correctly.
>>>>
>>>> Here is the latest patch (attached) with the test-cases and everything.
>>>>
>>>> In implementing the Naive Bayes classifier, we tried to *ensure
>>>> minimal disruption* to existing code.
>>>>
>>>> The *only* changes to existing code are as follows:
>>>>
>>>> 1. The opennlp.tools.ml.model.AbstractModel class has been changed to
>>>> include a new model type:
>>>>
>>>> line 35: *public enum ModelType *
>>>> *{Maxent,Perceptron,MaxentQn,NaiveBayes};*
>>>>
>>>> 2. The opennlp.tools.ml.model.GenericModelReader class has been
>>>> changed in one place:
>>>>
>>>> line 53:
>>>> *else if (modelType.equals("NaiveBayes")) **{ delegateModelReader =
>>>> new NaiveBayesModelReader(this.dataReader); }*
>>>>
>>>> 3. The opennlp.tools.ml.model.GenericModelWriter class has been
>>>> changed in two places:
>>>>
>>>> line 79:
>>>> *if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter
>>>> = new BinaryNaiveBayesModelWriter(model,dos); }*
>>>>
>>>> line 91:
>>>> *if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter
>>>> = new PlainTextNaiveBayesModelWriter(model,bw); }*
>>>>
>>>> 4. The initializer of the opennlp.tools.ml.TrainerFactory class has
>>>> been changed in one place to add the Naive Bayes trainer:
>>>>
>>>> line 51:
>>>> *_trainers.put(NaiveBayesTrainer.NAIVE_BAYES_VALUE,
>>>> NaiveBayesTrainer.class);*
>>>>
>>>> That was it!
>>>>
>>>> We didn't change anything else in the existing OpenNLP code.
>>>>
>>>> All the new code for the Naive Bayesian classifier sits in the package
>>>> opennlp.tools.ml.naivebayes - just above the perceptron
>>>>
>>>> The code for the document categorizer using the Naive Bayesian
>>>> classifier can be found in opennlp.tools.doccat (we didn't have to
>>>> change any existing code). The new doccat is called
>>>> opennlp.tools.doccat.DocumentCategorizerNB (reflecting the name of the
>>>> maxent document categorizer, which is DocumentCategorizerME).
>>>>
>>>> Proof of correctness!
>>>>
>>>> I have included two testcases:
>>>>
>>>> 1. A test to validate the document categorizer - under the tests
>>>> folder, you will find opennlp.tools.doccat.DocumentCategorizerNBTest -
>>>> which runs the same tests that were run on the ME document categorizer, but
>>>> on the Naive Bayes categorizer instead (all tests passed).
>>>>
>>>> 2. A test to check the mathematical correctness of the Naive Bayes
>>>> implementation can be found in
>>>> opennlp.tools.ml.naivebayes.NaiveBayesCorrectnessTest.
>>>>
>>>> So, the inclusion of this code will minimally impact any existing code.
>>>>
>>>> And the code in this patch contains a multinomial Naive Bayesian
>>>> classifier that is verifiably correct.
>>>>
>>>> Is there anything else I have to do to have this patch pulled into the
>>>> OpenNLP code base (for say 1.7.0)?
>>>>
>>>> Cohan Sujay Carlos
>>>> CEO, Aiaioo Labs
>>>> +91-77605-80015
>>>>
>>>> On Tue, May 19, 2015 at 7:21 PM, Cohan Sujay Carlos <co...@aiaioo.com>
>>>>> wrote:
>>>>>
>>>>>> Tommaso,
>>>>>>
>>>>>> I have created the Jira issue:
>>>>>> https://issues.apache.org/jira/browse/OPENNLP-777
>>>>>>
>>>>>>
>
Re: Are there plans to offer a Naive Bayesian Classifier in OpenNLP ?
Posted by Tommaso Teofili <to...@gmail.com>.
Hi Cohan,
for future reference it'd be better if you could attach the patch to the
Jira issue, I can have a look at your patch and comment later this week.
Usually we mark an issue as fixed as soon as the relevant code is committed.
Regards,
Tommaso
2015-07-18 15:36 GMT+02:00 Cohan Sujay Carlos <co...@aiaioo.com>:
> Yet another patch (ouch, yeah, like they say, "in product development,
> you're never done, till you're done!").
>
> Reason: Error in one of the test-cases. The test-case
> DocumentCategorizerNBTest is now correct and runs fine.
>
> Cohan Sujay Carlos
> CEO, Aiaioo Labs
> +91-77605-80015
>
> On Sat, Jul 18, 2015 at 6:51 PM, Cohan Sujay Carlos <co...@aiaioo.com>
> wrote:
>
>> A small update to the patch (I removed a superfluous piece of code).
>>
>> In the earlier path, I had used a subclass of
>> opennlp.tools.doccat.DoccatModel called opennlp.tools.doccat.DoccatModelNB
>> that was functionally identical. I removed that subclass since it
>> wasn't essential (DoccatModel does the trick just fine).
>>
>> Is there anything else I need to do?
>>
>> Is someone on the dev team going to be responsible for incorporating the
>> patch into the codebase?
>>
>> Can I mark this Jira issue fixed (for target 1.6.1?).
>>
>> Cohan Sujay Carlos
>> CEO, Aiaioo Labs
>> +91-77605-80015
>>
>>
>> On Sat, Jul 18, 2015 at 6:02 PM, Cohan Sujay Carlos <co...@aiaioo.com>
>> wrote:
>>
>>> I have gone ahead and written the test-cases and verified that the Naive
>>> Bayes Classifier works correctly.
>>>
>>> Here is the latest patch (attached) with the test-cases and everything.
>>>
>>> In implementing the Naive Bayes classifier, we tried to *ensure minimal
>>> disruption* to existing code.
>>>
>>> The *only* changes to existing code are as follows:
>>>
>>> 1. The opennlp.tools.ml.model.AbstractModel class has been changed to
>>> include a new model type:
>>>
>>> line 35: *public enum ModelType *
>>> *{Maxent,Perceptron,MaxentQn,NaiveBayes};*
>>>
>>> 2. The opennlp.tools.ml.model.GenericModelReader class has been changed
>>> in one place:
>>>
>>> line 53:
>>> *else if (modelType.equals("NaiveBayes")) **{ delegateModelReader = new
>>> NaiveBayesModelReader(this.dataReader); }*
>>>
>>> 3. The opennlp.tools.ml.model.GenericModelWriter class has been changed
>>> in two places:
>>>
>>> line 79:
>>> *if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter =
>>> new BinaryNaiveBayesModelWriter(model,dos); }*
>>>
>>> line 91:
>>> *if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter =
>>> new PlainTextNaiveBayesModelWriter(model,bw); }*
>>>
>>> 4. The initializer of the opennlp.tools.ml.TrainerFactory class has
>>> been changed in one place to add the Naive Bayes trainer:
>>>
>>> line 51:
>>> *_trainers.put(NaiveBayesTrainer.NAIVE_BAYES_VALUE,
>>> NaiveBayesTrainer.class);*
>>>
>>> That was it!
>>>
>>> We didn't change anything else in the existing OpenNLP code.
>>>
>>> All the new code for the Naive Bayesian classifier sits in the package
>>> opennlp.tools.ml.naivebayes - just above the perceptron
>>>
>>> The code for the document categorizer using the Naive Bayesian
>>> classifier can be found in opennlp.tools.doccat (we didn't have to
>>> change any existing code). The new doccat is called
>>> opennlp.tools.doccat.DocumentCategorizerNB (reflecting the name of the
>>> maxent document categorizer, which is DocumentCategorizerME).
>>>
>>> Proof of correctness!
>>>
>>> I have included two testcases:
>>>
>>> 1. A test to validate the document categorizer - under the tests folder,
>>> you will find opennlp.tools.doccat.DocumentCategorizerNBTest - which
>>> runs the same tests that were run on the ME document categorizer, but on
>>> the Naive Bayes categorizer instead (all tests passed).
>>>
>>> 2. A test to check the mathematical correctness of the Naive Bayes
>>> implementation can be found in
>>> opennlp.tools.ml.naivebayes.NaiveBayesCorrectnessTest.
>>>
>>> So, the inclusion of this code will minimally impact any existing code.
>>>
>>> And the code in this patch contains a multinomial Naive Bayesian
>>> classifier that is verifiably correct.
>>>
>>> Is there anything else I have to do to have this patch pulled into the
>>> OpenNLP code base (for say 1.7.0)?
>>>
>>> Cohan Sujay Carlos
>>> CEO, Aiaioo Labs
>>> +91-77605-80015
>>>
>>> On Tue, May 19, 2015 at 7:21 PM, Cohan Sujay Carlos <co...@aiaioo.com>
>>>> wrote:
>>>>
>>>>> Tommaso,
>>>>>
>>>>> I have created the Jira issue:
>>>>> https://issues.apache.org/jira/browse/OPENNLP-777
>>>>>
>>>>>
Re: Are there plans to offer a Naive Bayesian Classifier in OpenNLP ?
Posted by Cohan Sujay Carlos <co...@aiaioo.com>.
Yet another patch (ouch, yeah, like they say, "in product development,
you're never done, till you're done!").
Reason: Error in one of the test-cases. The test-case
DocumentCategorizerNBTest is now correct and runs fine.
Cohan Sujay Carlos
CEO, Aiaioo Labs
+91-77605-80015
On Sat, Jul 18, 2015 at 6:51 PM, Cohan Sujay Carlos <co...@aiaioo.com>
wrote:
> A small update to the patch (I removed a superfluous piece of code).
>
> In the earlier path, I had used a subclass of
> opennlp.tools.doccat.DoccatModel called opennlp.tools.doccat.DoccatModelNB
> that was functionally identical. I removed that subclass since it wasn't
> essential (DoccatModel does the trick just fine).
>
> Is there anything else I need to do?
>
> Is someone on the dev team going to be responsible for incorporating the
> patch into the codebase?
>
> Can I mark this Jira issue fixed (for target 1.6.1?).
>
> Cohan Sujay Carlos
> CEO, Aiaioo Labs
> +91-77605-80015
>
>
> On Sat, Jul 18, 2015 at 6:02 PM, Cohan Sujay Carlos <co...@aiaioo.com>
> wrote:
>
>> I have gone ahead and written the test-cases and verified that the Naive
>> Bayes Classifier works correctly.
>>
>> Here is the latest patch (attached) with the test-cases and everything.
>>
>> In implementing the Naive Bayes classifier, we tried to *ensure minimal
>> disruption* to existing code.
>>
>> The *only* changes to existing code are as follows:
>>
>> 1. The opennlp.tools.ml.model.AbstractModel class has been changed to
>> include a new model type:
>>
>> line 35: *public enum ModelType *
>> *{Maxent,Perceptron,MaxentQn,NaiveBayes};*
>>
>> 2. The opennlp.tools.ml.model.GenericModelReader class has been changed
>> in one place:
>>
>> line 53:
>> *else if (modelType.equals("NaiveBayes")) **{ delegateModelReader = new
>> NaiveBayesModelReader(this.dataReader); }*
>>
>> 3. The opennlp.tools.ml.model.GenericModelWriter class has been changed
>> in two places:
>>
>> line 79:
>> *if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter =
>> new BinaryNaiveBayesModelWriter(model,dos); }*
>>
>> line 91:
>> *if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter =
>> new PlainTextNaiveBayesModelWriter(model,bw); }*
>>
>> 4. The initializer of the opennlp.tools.ml.TrainerFactory class has been
>> changed in one place to add the Naive Bayes trainer:
>>
>> line 51:
>> *_trainers.put(NaiveBayesTrainer.NAIVE_BAYES_VALUE,
>> NaiveBayesTrainer.class);*
>>
>> That was it!
>>
>> We didn't change anything else in the existing OpenNLP code.
>>
>> All the new code for the Naive Bayesian classifier sits in the package
>> opennlp.tools.ml.naivebayes - just above the perceptron
>>
>> The code for the document categorizer using the Naive Bayesian classifier
>> can be found in opennlp.tools.doccat (we didn't have to change any
>> existing code). The new doccat is called
>> opennlp.tools.doccat.DocumentCategorizerNB (reflecting the name of the
>> maxent document categorizer, which is DocumentCategorizerME).
>>
>> Proof of correctness!
>>
>> I have included two testcases:
>>
>> 1. A test to validate the document categorizer - under the tests folder,
>> you will find opennlp.tools.doccat.DocumentCategorizerNBTest - which
>> runs the same tests that were run on the ME document categorizer, but on
>> the Naive Bayes categorizer instead (all tests passed).
>>
>> 2. A test to check the mathematical correctness of the Naive Bayes
>> implementation can be found in
>> opennlp.tools.ml.naivebayes.NaiveBayesCorrectnessTest.
>>
>> So, the inclusion of this code will minimally impact any existing code.
>>
>> And the code in this patch contains a multinomial Naive Bayesian
>> classifier that is verifiably correct.
>>
>> Is there anything else I have to do to have this patch pulled into the
>> OpenNLP code base (for say 1.7.0)?
>>
>> Cohan Sujay Carlos
>> CEO, Aiaioo Labs
>> +91-77605-80015
>>
>> On Tue, May 19, 2015 at 7:21 PM, Cohan Sujay Carlos <co...@aiaioo.com>
>>> wrote:
>>>
>>>> Tommaso,
>>>>
>>>> I have created the Jira issue:
>>>> https://issues.apache.org/jira/browse/OPENNLP-777
>>>>
>>>>
Re: Are there plans to offer a Naive Bayesian Classifier in OpenNLP ?
Posted by Cohan Sujay Carlos <co...@aiaioo.com>.
A small update to the patch (I removed a superfluous piece of code).
In the earlier path, I had used a subclass of
opennlp.tools.doccat.DoccatModel called opennlp.tools.doccat.DoccatModelNB
that was functionally identical. I removed that subclass since it wasn't
essential (DoccatModel does the trick just fine).
Is there anything else I need to do?
Is someone on the dev team going to be responsible for incorporating the
patch into the codebase?
Can I mark this Jira issue fixed (for target 1.6.1?).
Cohan Sujay Carlos
CEO, Aiaioo Labs
+91-77605-80015
On Sat, Jul 18, 2015 at 6:02 PM, Cohan Sujay Carlos <co...@aiaioo.com>
wrote:
> I have gone ahead and written the test-cases and verified that the Naive
> Bayes Classifier works correctly.
>
> Here is the latest patch (attached) with the test-cases and everything.
>
> In implementing the Naive Bayes classifier, we tried to *ensure minimal
> disruption* to existing code.
>
> The *only* changes to existing code are as follows:
>
> 1. The opennlp.tools.ml.model.AbstractModel class has been changed to
> include a new model type:
>
> line 35: *public enum ModelType *
> *{Maxent,Perceptron,MaxentQn,NaiveBayes};*
>
> 2. The opennlp.tools.ml.model.GenericModelReader class has been changed
> in one place:
>
> line 53:
> *else if (modelType.equals("NaiveBayes")) **{ delegateModelReader = new
> NaiveBayesModelReader(this.dataReader); }*
>
> 3. The opennlp.tools.ml.model.GenericModelWriter class has been changed
> in two places:
>
> line 79:
> *if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter =
> new BinaryNaiveBayesModelWriter(model,dos); }*
>
> line 91:
> *if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter =
> new PlainTextNaiveBayesModelWriter(model,bw); }*
>
> 4. The initializer of the opennlp.tools.ml.TrainerFactory class has been
> changed in one place to add the Naive Bayes trainer:
>
> line 51:
> *_trainers.put(NaiveBayesTrainer.NAIVE_BAYES_VALUE,
> NaiveBayesTrainer.class);*
>
> That was it!
>
> We didn't change anything else in the existing OpenNLP code.
>
> All the new code for the Naive Bayesian classifier sits in the package
> opennlp.tools.ml.naivebayes - just above the perceptron
>
> The code for the document categorizer using the Naive Bayesian classifier
> can be found in opennlp.tools.doccat (we didn't have to change any
> existing code). The new doccat is called
> opennlp.tools.doccat.DocumentCategorizerNB (reflecting the name of the
> maxent document categorizer, which is DocumentCategorizerME).
>
> Proof of correctness!
>
> I have included two testcases:
>
> 1. A test to validate the document categorizer - under the tests folder,
> you will find opennlp.tools.doccat.DocumentCategorizerNBTest - which runs
> the same tests that were run on the ME document categorizer, but on the
> Naive Bayes categorizer instead (all tests passed).
>
> 2. A test to check the mathematical correctness of the Naive Bayes
> implementation can be found in
> opennlp.tools.ml.naivebayes.NaiveBayesCorrectnessTest.
>
> So, the inclusion of this code will minimally impact any existing code.
>
> And the code in this patch contains a multinomial Naive Bayesian
> classifier that is verifiably correct.
>
> Is there anything else I have to do to have this patch pulled into the
> OpenNLP code base (for say 1.7.0)?
>
> Cohan Sujay Carlos
> CEO, Aiaioo Labs
> +91-77605-80015
>
> On Tue, May 19, 2015 at 7:21 PM, Cohan Sujay Carlos <co...@aiaioo.com>
>> wrote:
>>
>>> Tommaso,
>>>
>>> I have created the Jira issue:
>>> https://issues.apache.org/jira/browse/OPENNLP-777
>>>
>>>
Re: Are there plans to offer a Naive Bayesian Classifier in OpenNLP ?
Posted by Cohan Sujay Carlos <co...@aiaioo.com>.
I have gone ahead and written the test-cases and verified that the Naive
Bayes Classifier works correctly.
Here is the latest patch (attached) with the test-cases and everything.
In implementing the Naive Bayes classifier, we tried to *ensure minimal
disruption* to existing code.
The *only* changes to existing code are as follows:
1. The opennlp.tools.ml.model.AbstractModel class has been changed to
include a new model type:
line 35: *public enum ModelType **{Maxent,Perceptron,MaxentQn,NaiveBayes};*
2. The opennlp.tools.ml.model.GenericModelReader class has been changed in
one place:
line 53:
*else if (modelType.equals("NaiveBayes")) **{ delegateModelReader = new
NaiveBayesModelReader(this.dataReader); }*
3. The opennlp.tools.ml.model.GenericModelWriter class has been changed in
two places:
line 79:
*if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter = new
BinaryNaiveBayesModelWriter(model,dos); }*
line 91:
*if (model.getModelType() == ModelType.NaiveBayes) **{ delegateWriter = new
PlainTextNaiveBayesModelWriter(model,bw); }*
4. The initializer of the opennlp.tools.ml.TrainerFactory class has been
changed in one place to add the Naive Bayes trainer:
line 51:
*_trainers.put(NaiveBayesTrainer.NAIVE_BAYES_VALUE,
NaiveBayesTrainer.class);*
That was it!
We didn't change anything else in the existing OpenNLP code.
All the new code for the Naive Bayesian classifier sits in the package
opennlp.tools.ml.naivebayes - just above the perceptron
The code for the document categorizer using the Naive Bayesian classifier
can be found in opennlp.tools.doccat (we didn't have to change any existing
code). The new doccat is called opennlp.tools.doccat.DocumentCategorizerNB
(reflecting the name of the maxent document categorizer, which is
DocumentCategorizerME).
Proof of correctness!
I have included two testcases:
1. A test to validate the document categorizer - under the tests folder,
you will find opennlp.tools.doccat.DocumentCategorizerNBTest - which runs
the same tests that were run on the ME document categorizer, but on the
Naive Bayes categorizer instead (all tests passed).
2. A test to check the mathematical correctness of the Naive Bayes
implementation can be found in
opennlp.tools.ml.naivebayes.NaiveBayesCorrectnessTest.
So, the inclusion of this code will minimally impact any existing code.
And the code in this patch contains a multinomial Naive Bayesian classifier
that is verifiably correct.
Is there anything else I have to do to have this patch pulled into the
OpenNLP code base (for say 1.7.0)?
Cohan Sujay Carlos
CEO, Aiaioo Labs
+91-77605-80015
On Tue, May 19, 2015 at 7:21 PM, Cohan Sujay Carlos <co...@aiaioo.com>
> wrote:
>
>> Tommaso,
>>
>> I have created the Jira issue:
>> https://issues.apache.org/jira/browse/OPENNLP-777
>>
>>