[jira] [Commented] (MAHOUT-925) Evaluate the reach of recommender algorithms

["Sean Owen (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAHOUT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167573#comment-13167573 ] 

Sean Owen commented on MAHOUT-925:
----------------------------------

This is fine, though, don't you want to count like so?

    if (numRecommendedItems > 0) {
      reach++;
    }

Otherwise it seems like you're just counting all users, except the ones that the *test framework* couldn't test due to sampling size, which is something else and something you want to ignore in general. (So actually I think I'd count both reach and the number of users that the test framework succeeded for). Is that about right for you?
                
> Evaluate the reach of recommender algorithms
> --------------------------------------------
>
>                 Key: MAHOUT-925
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-925
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.5
>            Reporter: Anatoliy Kats
>            Assignee: Sean Owen
>            Priority: Minor
>         Attachments: MAHOUT-925.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> The evaluation of a CF algorithm should include reach, the proportion of users for whom a recommendation could be made.  An algorithm usually has a cutoff value on the confidence of the recommender, and if it is not high enough, no recommendation is made.  The number of requested recommendations, or this parameter could be varied as part of the evaluation.  The proposed patch adds this.
> My build with this patch breaks testMapper(org.apache.mahout.classifier.df.mapreduce.partial.Step1MapperTest): org.apache.mahout.classifier.df.node.Leaf.<init>(I)V .  The test seems unrelated to the patch, so I am assuming this is broken in the trunk head as well.  Unfortunately I am under a deadline, and I do not have time to write tests for the patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira