[jira] Commented: (MAHOUT-247) GenericUserBasedRecommender.recommend causes connection leak when called for user with no preferences

["Sean Owen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAHOUT-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800466#action_12800466 ] 

Sean Owen commented on MAHOUT-247:
----------------------------------

This is actually very messy to really fix -- LongPrimitiveIterator is used in a lot of places, and to make sure it's "drained" in all exception cases would make quite a mess of the code.

I did add a finalize() method to the implementations which are based on a JDBC connection, to close the connection. This doesn't really fix anything but at least may get the connection closed via finalization somewhat faster.

While I think it's hard to actually guarantee closing the connection immediately on all exception paths, it's relatively easy to apply fixes that would avoid the cases that would actually probably cause problems, such as the code you cite above.

I think my best bet at a fix here is to apply your change and related changes in key parts of the code and leave it at that.

> GenericUserBasedRecommender.recommend causes connection leak when called for user with no preferences
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-247
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-247
>             Project: Mahout
>          Issue Type: Bug
>          Components: Collaborative Filtering
>    Affects Versions: 0.2
>         Environment: Reproducable on Win32 and Ubuntu
>            Reporter: Tolga Oral
>            Assignee: Sean Owen
>            Priority: Minor
>             Fix For: 0.3
>
>
> 			UserSimilarity userSimilarity = new TanimotoCoefficientSimilarity(getBooleanPrefDataModel());
> 			UserNeighborhood neighborhood = new NearestNUserNeighborhood(3, userSimilarity, getBooleanPrefDataModel());
> 			Recommender recommender = new GenericBooleanPrefUserBasedRecommender(getBooleanPrefDataModel(), neighborhood, userSimilarity);
>                         recommender.recommend(userwithnopreferencesdata, 10);
> code properly throws NoSuchUserException however one of the connections is hang on LongPrimitiveIterator backed by org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel$ResultSetIDIterator as Exception is thrown before TopItems.getTopUsers finishes the while loop 
> public static long[] getTopUsers(int howMany,
>                                    LongPrimitiveIterator allUserIDs,
>                                    Rescorer<Long> rescorer,
>                                    Estimator<Long> estimator) throws TasteException {
>     Queue<SimilarUser> topUsers = new PriorityQueue<SimilarUser>(howMany + 1, Collections.reverseOrder());
>     boolean full = false;
>     double lowestTopValue = Double.NEGATIVE_INFINITY;
> //HERE IS THE ITERATOR
>     while (allUserIDs.hasNext()) {
>       long userID = allUserIDs.next();
>       if (rescorer != null && rescorer.isFiltered(userID)) {
>         continue;
>       }
> //EXCEPTION THROWN HERE CAUSES THE CONNECTION LEAK
>       double similarity = estimator.estimate(userID);
>       double rescoredSimilarity = rescorer == null ? similarity : rescorer.rescore(userID, similarity);
>       if (!Double.isNaN(rescoredSimilarity) && (!full || rescoredSimilarity > lowestTopValue)) {
>         topUsers.add(new SimilarUser(userID, similarity));
>         if (full) {
>           topUsers.poll();
>         } else if (topUsers.size() > howMany) {
>           full = true;
>           topUsers.poll();
>         }
>         lowestTopValue = topUsers.peek().getSimilarity();
>       }
>     }
>     if (topUsers.isEmpty()) {
>       return NO_IDS;
>     }
>     List<SimilarUser> sorted = new ArrayList<SimilarUser>(topUsers.size());
>     sorted.addAll(topUsers);
>     Collections.sort(sorted);
>     long[] result = new long[sorted.size()];
>     int i = 0;
>     for (SimilarUser similarUser : sorted) {
>       result[i++] = similarUser.getUserID();
>     }
>     return result;
>   }
> ============================================================================================================
> I currently fixed it in our application by checking first to see if user has preferences for the given dataset (user might exists and have preferences for a different dataset).
> However this edge case does not cause issues in some other recommenders as long as we handle the NoSuchUserException.
> Easy solution is to use AbstractJDBCDataModel$ResultSetIDIterator always with try/catch/finally and release the connection.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.