You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Jens Grivolla (JIRA)" <ji...@apache.org> on 2009/11/05 19:01:33 UTC
[jira] Created: (MAHOUT-196) bounded values for
RecommenderEvaluator
bounded values for RecommenderEvaluator
---------------------------------------
Key: MAHOUT-196
URL: https://issues.apache.org/jira/browse/MAHOUT-196
Project: Mahout
Issue Type: Improvement
Components: Collaborative Filtering
Reporter: Jens Grivolla
Priority: Minor
When evaluating a recommender using RMSRecommenderEvaluator (or some others) on e.g. Netflix data, a recommender gets heavily penalized for predicting values below 1 or above 5 (that are known to be out of the permitted bounds).
It seems to me that it makes no sense to change the recommender to avoid those predictions, since an estimated 6 probably has a greater chance to be highly rated than a predicted 5.1. I therefore propose to allow truncating predictions to those "legal" values directly in the evaluator and leave the recommenders unchanged, since it is more of a post-processing step than part of the recommender itself.
I added those boundaries to the constructor of RMSRecommenderEvaluator and limit estimatedPreference to the allowed range before calculating "realPref.getValue() - estimatedPreference" and seem to get slightly better scores.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAHOUT-196) bounded values for
RecommenderEvaluator
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved MAHOUT-196.
------------------------------
Resolution: Fixed
Fix Version/s: 0.3
Assignee: Sean Owen
I committed a variant on your suggestion. Since these are optional parameters to the object, I implemented them as bean properties rather than constructor args (which I use for "necessary" parameters, though admittedly inconsistently). You can now set a max/min preference value for use in the evaluation.
> bounded values for RecommenderEvaluator
> ---------------------------------------
>
> Key: MAHOUT-196
> URL: https://issues.apache.org/jira/browse/MAHOUT-196
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Reporter: Jens Grivolla
> Assignee: Sean Owen
> Priority: Minor
> Fix For: 0.3
>
>
> When evaluating a recommender using RMSRecommenderEvaluator (or some others) on e.g. Netflix data, a recommender gets heavily penalized for predicting values below 1 or above 5 (that are known to be out of the permitted bounds).
> It seems to me that it makes no sense to change the recommender to avoid those predictions, since an estimated 6 probably has a greater chance to be highly rated than a predicted 5.1. I therefore propose to allow truncating predictions to those "legal" values directly in the evaluator and leave the recommenders unchanged, since it is more of a post-processing step than part of the recommender itself.
> I added those boundaries to the constructor of RMSRecommenderEvaluator and limit estimatedPreference to the allowed range before calculating "realPref.getValue() - estimatedPreference" and seem to get slightly better scores.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-196) bounded values for
RecommenderEvaluator
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774025#action_12774025 ]
Sean Owen commented on MAHOUT-196:
----------------------------------
It's an interesting question, yeah. One approach would be to cap this in the recommender, which makes some sense. Why would I ever estimate a movie was rated 6 stars? the only catch is then you lose some ordering information that the estimates provide. A 5.5 star movie should still be recommended before 5.4.
Let me think about a way to incorporate this. I imagine it is indeed just a matter of exposing some way to express limits.
> bounded values for RecommenderEvaluator
> ---------------------------------------
>
> Key: MAHOUT-196
> URL: https://issues.apache.org/jira/browse/MAHOUT-196
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Reporter: Jens Grivolla
> Priority: Minor
>
> When evaluating a recommender using RMSRecommenderEvaluator (or some others) on e.g. Netflix data, a recommender gets heavily penalized for predicting values below 1 or above 5 (that are known to be out of the permitted bounds).
> It seems to me that it makes no sense to change the recommender to avoid those predictions, since an estimated 6 probably has a greater chance to be highly rated than a predicted 5.1. I therefore propose to allow truncating predictions to those "legal" values directly in the evaluator and leave the recommenders unchanged, since it is more of a post-processing step than part of the recommender itself.
> I added those boundaries to the constructor of RMSRecommenderEvaluator and limit estimatedPreference to the allowed range before calculating "realPref.getValue() - estimatedPreference" and seem to get slightly better scores.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.