You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Robin Anil (JIRA)" <ji...@apache.org> on 2012/06/04 16:10:23 UTC
[jira] [Resolved] (MAHOUT-955) Bayes classification result are
unstable after classifying non-existing features
[ https://issues.apache.org/jira/browse/MAHOUT-955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Anil resolved MAHOUT-955.
-------------------------------
Resolution: Fixed
There is a new naivebayes implementation in 0.7 which should be way for scalable and stable.
> Bayes classification result are unstable after classifying non-existing features
> --------------------------------------------------------------------------------
>
> Key: MAHOUT-955
> URL: https://issues.apache.org/jira/browse/MAHOUT-955
> Project: Mahout
> Issue Type: Bug
> Components: Classification
> Affects Versions: 0.5
> Environment: JRE 7
> Reporter: MichaĆ B
> Assignee: Robin Anil
> Priority: Critical
> Labels: Bayes,, InMemoryDatastore,, vocabCount
>
> Bayes classification results are unstable, and change during runtime!
> Sample test:
> MyClassifier classifier = new MyClassifier(new BayesAlgorithm(),params); //Custom simple wrapper for classifier
> ClassifierResult category = classifier.classify("existing");
> double resultA = category.getScore();
> category = classifier.classify("nonexisting");
> category = classifier.classify("existing");
> double resultB = category.getScore();
> Assert.assertEquals(resultA,resultB,0.0); // FAIL!!!
> Test like the one above will fail. Because nonexisting tokens are added to InMemoryBayesDatastore->featureDictionary therefore datastore.getWeight("sumWeight", "vocabCount") change after classification of unknown feature. Moreover, the featureDictionary fills with not wanted strings using heapspace.
> More on this here
> http://www.lucidimagination.com/search/document/7dabe3efec8d136d/issues_with_memory_use_and_inconsistent_or_state_influenced_results_when_using_cbayesalgorit#8853165db260bf75
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira