You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Han Hui Wen (JIRA)" <ji...@apache.org> on 2010/08/30 09:35:54 UTC
[jira] Issue Comment Edited: (MAHOUT-484) The RecommenderJob exit
,some sub-jobs can not be run.
[ https://issues.apache.org/jira/browse/MAHOUT-484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904097#action_12904097 ]
Han Hui Wen edited comment on MAHOUT-484 at 8/30/10 3:34 AM:
--------------------------------------------------------------
The reason is as following:
1) I run the recommenderJob liking this way:
hadoop jar ../../singlejar/mahout-core-0.4-SNAPSHOT.job org.apache.mahout.cf.taste.hadoop.item.RecommenderJob -Dmapred.job.name=RECOMMENDATION_tap_computed -Dmapred.reduce.tasks=80 -Dmapred.input.dir=/in -Dmapred.output.dir=/out -Dmapred.output.compress=false --tempDir /temp --itemsFile /userInvalidItemsFile --userInvalidItemsFile /itemsFile --numRecommendations 10 --booleanData false --similarityClassname SIMILARITY_TANIMOTO_COEFFICIENT --maxPrefsPerUser 10 --maxSimilaritiesPerItem 100 --maxCooccurrencesPerItem 200
2) When hadoop run the RecommenderJob ,it will open the jar file mahout-core-0.4-SNAPSHOT.job and find all the entry of the jar file.
2) when run job RowSimilarityJob ,it will open the jar file mahout-core-0.4-SNAPSHOT.job again ,but mahout-core-0.4-SNAPSHOT.job has already opened.
so cause the problem.
the detail can see the log file https://issues.apache.org/jira/secure/attachment/12452530/hs_err_pid7384.log.
This issue is happened in many version JDK.
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6390352 (1.4)
http://forums.sun.com/thread.jspa?threadID=762731 (1.4,1.5)
http://forums.sun.com/thread.jspa?forumID=546&threadID=5423931
JDK seems can not avoid this issue. so the sate way is converting nested call to flat call.
was (Author: huiwenhan):
The reason is as following:
1) I run the recommenderJob liking this way:
hadoop jar ../../singlejar/mahout-core-0.4-SNAPSHOT.job org.apache.mahout.cf.taste.hadoop.item.RecommenderJob -Dmapred.job.name=RECOMMENDATION_tap_computed -Dmapred.reduce.tasks=80 -Dmapred.input.dir=/in -Dmapred.output.dir=/out -Dmapred.output.compress=false --tempDir /temp --itemsFile /userInvalidItemsFile --userInvalidItemsFile /itemsFile --numRecommendations 10 --booleanData false --similarityClassname SIMILARITY_TANIMOTO_COEFFICIENT --maxPrefsPerUser 10 --maxSimilaritiesPerItem 100 --maxCooccurrencesPerItem 200
2) When hadoop run the RecommenderJob ,it will open the jar file mahout-core-0.4-SNAPSHOT.job and find all the entry of the jar file.
2) when run job RowSimilarityJob ,it will open the jar file mahout-core-0.4-SNAPSHOT.job again ,but mahout-core-0.4-SNAPSHOT.job has already opened.
so cause the problem.
the detail can see the log file https://issues.apache.org/jira/secure/attachment/12452530/hs_err_pid7384.log.
This issue is happened in many version JDK.
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6390352 (1.4)
http://forums.sun.com/thread.jspa?threadID=762731 (1.4,1.5)
JDK seems can not avoid this issue. so the sate way is converting nested call to flat call.
> The RecommenderJob exit ,some sub-jobs can not be run.
> ------------------------------------------------------
>
> Key: MAHOUT-484
> URL: https://issues.apache.org/jira/browse/MAHOUT-484
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Han Hui Wen
> Attachments: hs_err_pid7384.log, patch-20100820.txt, patch-20100824_2.txt, screenshot-1.jpg, screenshot-2.jpg
>
>
> I have done a few test today,
> The RecommenderJob exit in middle.
> The first time it exited when it finished RowSimilarityJob-CooccurrencesMapper-SimilarityReducer
> the second time it exited when it finished RECOMMENDATION_dogear_bookmark-Mapper-EntriesToVectorsReducer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.