You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Hui Wen Han (JIRA)" <ji...@apache.org> on 2010/04/02 08:07:27 UTC
[jira] Created: (MAHOUT-358) the score field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
the score field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
-----------------------------------------------------------------------------------------------
Key: MAHOUT-358
URL: https://issues.apache.org/jira/browse/MAHOUT-358
Project: Mahout
Issue Type: Test
Components: Collaborative Filtering
Affects Versions: 0.4
Reporter: Hui Wen Han
the value has negative value ,also has decimal fraction (it should is integer )
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (MAHOUT-358) the pref value field of
output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has
negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854438#action_12854438 ]
Hui Wen Han edited comment on MAHOUT-358 at 4/7/10 11:58 AM:
-------------------------------------------------------------
input file https://issues.apache.org/jira/secure/attachment/12441022/2010_input_steer_item_in_prefs-1.dat
was (Author: huiwenhan):
input file
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854387#action_12854387 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
it seems that we got the dirty data here;
recommendations.addAll(topItems);
I want to get 10 recommendation items,but it exceeds 10 sometime and data is strange.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854432#action_12854432 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
The output is helpful. I am more confused, we may be talking about different things. userID could well be negative. It's a long. What is your input like? how are you mapping user IDs to longs? That is not a problem per se, but I would like to understand what your user IDs are like.
Earlier your output showed negative *values*, not *IDs*, and that is something I can't figure out. But we don't see that here.
Actually, this output seems correct to me.
Which are you talking about Sebastian?
I can't quite understand why you would see more than 10 recommendations since as you see, the queue is kept to this size in the code above. I can try one or two more changes that try to make the code even more straightforward (e.g. moving the ordering logic in a Comparator, not reusing Writables in case something strange goes on there.)
Is the change to Text really a solution -- I am not yet clear we're talking about the same issue. I can't see how this changes behavior. We're already seeing that RecommendedItemsWrtiable.toString() is being invoked. I do not see how String.valueOf(long) could return the 'right' value, but StringBuffer.append(long) the 'wrong' value. They're calling into the same java.lang code in the end.
Very very confused here, esp since I cannot reproduce this.
Could it be a Hadoop version issue? I'm on 0.20.2.
I will commit any wild-guess changes I can here in a minute in an effort to shed any light.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853982#action_12853982 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
That is truly strange then. The user vectors have nonnegative values, and so does the co-occurrence matrix. Their product can't have negative values. Something is going wrong somewhere in there.
I can't reproduce to check without your data, though if you're in a position to debug, you'll have to see why user vectors or the matrix has negative values. I have looked at the code many times and do not see a way this can happen.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (MAHOUT-358) the pref value field of
output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has
negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854453#action_12854453 ]
Hui Wen Han edited comment on MAHOUT-358 at 4/7/10 1:58 PM:
------------------------------------------------------------
please see the screen shot ,
https://issues.apache.org/jira/secure/attachment/12441026/screenshot-3.jpg
the value of the final output has negative .
it may been caused by hadoop ,
maybe in the intermediate phase ,it read and write the float value using write and readFields mothod,
it use toString method just in the final output .
was (Author: huiwenhan):
please see the screen shot ,
https://issues.apache.org/jira/secure/attachment/12441026/screenshot-3.jpg
the value of the final output has negative .
it may been caused by hadoop ,
maybe it the intermediate phase ,it read and write the float value using write and readFields mothod,
it use toString method just in the final output .
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854516#action_12854516 ]
Sebastian Schelter commented on MAHOUT-358:
-------------------------------------------
I updated to revision 931566 and everything looks fine now!
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854473#action_12854473 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
Yes, looks like the same kind of output you posted earlier. Are you using my latest changes?
Sebastian I'm also very puzzled. It does not seem to be sorted by number, when it is sorted by number in the code. Again try my very latest change, which I have some reason to suspect would address this. Still pretty confused.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854317#action_12854317 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
http://java.sun.com/javase/6/docs/api/java/math/BigDecimal.html#BigDecimal(double)
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854580#action_12854580 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
what's the reason ? cause by PriorityQueue?
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Assignee: Sean Owen
> Fix For: 0.4
>
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852785#action_12852785 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
Having decimal values is normal. Negative values are bizarre though. The co-occurrence matrix and user vector should have nonnegative values, so I can't see how it gets a negative element from multiplications and additions. I don't see any obvious problems with the logic.
I wonder if you can debug, look at the cooccurrence matrix and/or user vectors? make sure neither has negative values?
I'm trying to imagine wild scenarios, like overflowing an int in the cooccurrence matrix? but you don't have billions of users...
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854482#action_12854482 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
hi,Sean
maybe we need order it in the reduce job and need a reduce implementation .
maybe map/reduce ruffle the data in the IdentityReducer phase.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854568#action_12854568 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
it's very good now,thanks Sean! :)
only one little issue ,can ignore line that has no recommendations ?
-2146704409350804772 [-5143306609724270245:0.0,-6887672525304767274:0.0,-7869337437647864448:0.0,1143520528292887196:0.0,4565256005884091248:0.0,-403573745216718449:0.0]
-2140684086652787668 [5066321748533664330:3.33,-6196380629601347193:2.22,1388936500341917293:2.22,-5143306609724270245:2.22,-6887672525304767274:2.22,-403573745216718449:2.22,4000074642144994975:0.0,1471545749001003492:0.0,8619821060300841906:0.0,-8796495370659715938:0.0]
-2087578807957211564 [3693944382462874249:0.0]
-2084997234534227590 []
-1994184181161252808 []
-1956807977260853620 []
-1905139778468734800 []
-1897443610249479948 []
-1861747039795463904 []
-1783644832320395644 []
-1759587438285537664 []
-1598157027497721376 []
-1524111942416072322 []
-1508995658783890206 []
-1478238285808155750 []
-1473372051895027798 []
-1360993164612489840 []
-1170376071408489552 [-8608197272004918866:0.0,-8235449048613678861:0.0]
-1106281807063652538 []
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854383#action_12854383 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
I have some finding :
I debug as following:
code:
public void map(LongWritable userID,
VectorWritable vectorWritable,
OutputCollector<LongWritable,RecommendedItemsWritable> output,
Reporter reporter) throws IOException {
if ((usersToRecommendFor != null) && !usersToRecommendFor.contains(userID.get())) {
return;
}
System.out.println("user_id:start-----------"+userID.get());
Vector userVector = vectorWritable.get();
Iterator<Vector.Element> userVectorIterator = userVector.iterateNonZero();
Vector recommendationVector = new RandomAccessSparseVector(Integer.MAX_VALUE, 1000);
while (userVectorIterator.hasNext()) {
Vector.Element element = userVectorIterator.next();
int index = element.index();
Vector columnVector;
try {
columnVector = cooccurrenceColumnCache.get(new IntWritable(index));
} catch (TasteException te) {
if (te.getCause() instanceof IOException) {
throw (IOException) te.getCause();
} else {
throw new IOException(te.getCause());
}
}
if (columnVector != null) {
Iterator<Vector.Element> columnVectorIterator = columnVector.iterateNonZero(); //debug here
while (columnVectorIterator.hasNext()) {
Vector.Element element1 = columnVectorIterator.next();
int index1 = element1.index();
double value1 = element1.get();
System.out.println("before index["+index1+"]:"+value1);
}
if (booleanData) { // because 'value' is 1.0
columnVector.addTo(recommendationVector);
} else {
double value = element.get();
System.out.println(value);
columnVector.times(value).addTo(recommendationVector);
}
}
}
Iterator<Vector.Element> recVectorIterator = recommendationVector.iterateNonZero(); //debug here
while (recVectorIterator.hasNext()) {
Vector.Element element2 = recVectorIterator.next();
int index2 = element2.index();
double value2 = element2.get();
System.out.println("after inex["+index2+"]:"+value2);
}
Queue<RecommendedItem> topItems = new PriorityQueue<RecommendedItem>(recommendationsPerUser + 1,
Collections.reverseOrder());
Iterator<Vector.Element> recommendationVectorIterator = recommendationVector.iterateNonZero();
LongWritable itemID = new LongWritable();
while (recommendationVectorIterator.hasNext()) {
Vector.Element element = recommendationVectorIterator.next();
int index = element.index();
if (userVector.get(index) == 0.0) {
if (topItems.size() < recommendationsPerUser) {
indexItemIDMap.get(new IntWritable(index), itemID);
topItems.add(new GenericRecommendedItem(itemID.get(), (float) element.get()));
} else if (element.get() > topItems.peek().getValue()) {
indexItemIDMap.get(new IntWritable(index), itemID);
topItems.add(new GenericRecommendedItem(itemID.get(), (float) element.get()));
topItems.poll();
}
}
}
List<RecommendedItem> recommendations = new ArrayList<RecommendedItem>(topItems.size());
recommendations.addAll(topItems);
Collections.sort(recommendations);
output.collect(userID, new RecommendedItemsWritable(recommendations));
for (RecommendedItem recommendation : recommendations)
{
System.out.println(recommendation.getItemID()+":"+recommendation.getValue()); //debug here
//user.set(String.valueOf(userID));
//recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
//output.collect(user, recomScore);
}
System.out.println("user_id:end-----------"+userID.get());
}
add the out put and the log in attachment
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (MAHOUT-358) the pref value field of
output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has
negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854568#action_12854568 ]
Hui Wen Han edited comment on MAHOUT-358 at 4/7/10 4:15 PM:
------------------------------------------------------------
it's very good now,thanks Sean! :)
only one little issue ,can ignore line that has no recommendations ?
-2146704409350804772 [-5143306609724270245:0.0,-6887672525304767274:0.0,-7869337437647864448:0.0,1143520528292887196:0.0,4565256005884091248:0.0,-403573745216718449:0.0]
-2140684086652787668 [5066321748533664330:3.33,-6196380629601347193:2.22,1388936500341917293:2.22,-5143306609724270245:2.22,-6887672525304767274:2.22,-403573745216718449:2.22,4000074642144994975:0.0,1471545749001003492:0.0,8619821060300841906:0.0,-8796495370659715938:0.0]
-2087578807957211564 [3693944382462874249:0.0]
-2084997234534227590 []
-1994184181161252808 []
-1956807977260853620 []
-1905139778468734800 []
-1897443610249479948 []
-1861747039795463904 []
-1783644832320395644 []
-1759587438285537664 []
-1598157027497721376 []
-1524111942416072322 []
-1508995658783890206 []
-1478238285808155750 []
-1473372051895027798 []
-1360993164612489840 []
-1170376071408489552 [-8608197272004918866:0.0,-8235449048613678861:0.0]
-1106281807063652538 []
It seems that
vector.iterateNonZero() can not skip the zero value.
also in UserVectorToCooccurrenceReducer, the following code seems does not need.
because some user has little preferences,if we purge here ,maybe he can not get recommendation.
Iterator<Vector.Element> cooccurrences = cooccurrenceRow.iterateNonZero();
while (cooccurrences.hasNext()) {
Vector.Element element = cooccurrences.next();
if (element.get() <= 1.0) { // purge small values
element.set(0.0);
}
}
was (Author: huiwenhan):
it's very good now,thanks Sean! :)
only one little issue ,can ignore line that has no recommendations ?
-2146704409350804772 [-5143306609724270245:0.0,-6887672525304767274:0.0,-7869337437647864448:0.0,1143520528292887196:0.0,4565256005884091248:0.0,-403573745216718449:0.0]
-2140684086652787668 [5066321748533664330:3.33,-6196380629601347193:2.22,1388936500341917293:2.22,-5143306609724270245:2.22,-6887672525304767274:2.22,-403573745216718449:2.22,4000074642144994975:0.0,1471545749001003492:0.0,8619821060300841906:0.0,-8796495370659715938:0.0]
-2087578807957211564 [3693944382462874249:0.0]
-2084997234534227590 []
-1994184181161252808 []
-1956807977260853620 []
-1905139778468734800 []
-1897443610249479948 []
-1861747039795463904 []
-1783644832320395644 []
-1759587438285537664 []
-1598157027497721376 []
-1524111942416072322 []
-1508995658783890206 []
-1478238285808155750 []
-1473372051895027798 []
-1360993164612489840 []
-1170376071408489552 [-8608197272004918866:0.0,-8235449048613678861:0.0]
-1106281807063652538 []
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854073#action_12854073 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
if use Text as the out format ,everything is ok.
maybe something is wrong with RecommendedItemsWritable or IdentityReducer.
public final class RecommenderMapper extends MapReduceBase implements
Mapper<LongWritable,VectorWritable,Text,Text>
private final Text user = new Text();
private final Text recomScore = new Text();
private static final String FIELD_SEPERATOR = ",";
//output.collect(userID, new RecommendedItemsWritable(recommendations));
for (RecommendedItem recommendation : recommendations)
{
user.set(String.valueOf(userID));
recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
output.collect(user, recomScore);
}
JobConf recommenderConf = prepareJobConf(userVectorPath, outputPath,
SequenceFileInputFormat.class, RecommenderMapper.class, Text.class,
Text.class, IdentityReducer.class, Text.class,
Text.class, TextOutputFormat.class);
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-358:
-------------------------------
Attachment: screenshot-2.jpg
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-358:
-------------------------------
Attachment: screenshot-3.jpg
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854453#action_12854453 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
please see the screen shot ,
https://issues.apache.org/jira/secure/attachment/12441026/screenshot-3.jpg
the value of the final output has negative .
it may been caused by hadoop ,
maybe it the intermediate phase ,it read and write the float value using write and readFields mothod,
it use toString method just in the final output .
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854530#action_12854530 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
I see the latest is 931569
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-358) the score field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-358:
-------------------------------
Attachment: screenshot-1.jpg
the value has negative
> the score field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> the value has negative value ,also has decimal fraction (it should is integer )
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved MAHOUT-358.
------------------------------
Resolution: Fixed
Fix Version/s: 0.4
Assignee: Sean Owen
Great. Thanks much to both of you for your help in debugging this. Obviously it was a pretty subtle issue. This has been very helpful.
I changed it to not output recs when the list is empty, sure.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Assignee: Sean Owen
> Fix For: 0.4
>
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854445#action_12854445 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
1) Your input file shows negative user and item IDs, good, so that is as expected. The HTML log file you posted also does not show negative values. This is all good.
2) I can't quite see why this would happen -- you see how the code adds, then removes, one item each time when the queue is full. However I just made one change to the Comparator which *could* be related. You might try the new code to see if it addresses this.
3) Same, I am wondering if the comparator is the problem here.
4 + 5) I agree, and while this is very strange, it seems all your evidence points to this. Can you try my latest code? It still uses a custom writable, but, it uses the exact same to-string routine that the working example does.
If this does not somehow address the issues, I'll give up and use Text. This shouldn't be necesary, and is incorrect from an API perspective, but something working is better than nothing.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-358:
-------------------------------
Description:
In my test the input pref values all is positive.
the output score value has negative value ,
was:the value has negative value ,also has decimal fraction (it should is integer )
Summary: the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative (was: the score field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative)
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852830#action_12852830 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
I will do more test next week :)
we only have about 1 million users
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (MAHOUT-358) the pref value field of
output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has
negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854387#action_12854387 ]
Hui Wen Han edited comment on MAHOUT-358 at 4/7/10 8:59 AM:
------------------------------------------------------------
it seems that we got the dirty data here;
recommendations.addAll(topItems);
or List<RecommendedItem> recommendations has element that we do not fill it.
I want to get 10 recommendation items,but it exceeds 10 sometime and data is strange.
was (Author: huiwenhan):
it seems that we got the dirty data here;
recommendations.addAll(topItems);
I want to get 10 recommendation items,but it exceeds 10 sometime and data is strange.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (MAHOUT-358) the pref value field of
output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has
negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854485#action_12854485 ]
Hui Wen Han edited comment on MAHOUT-358 at 4/7/10 1:41 PM:
------------------------------------------------------------
I tested using the latest code ,it also has negative.
-8014130140593843068 [576620340710744639:-1.2199921E-19,-7951291328374999828:7.480107E-34,48126569744862489:2.384486E23,5238530791562049439:-18.706377,-6651253699672801114:-12.196953,-464715186549278845:1.880351E-20,-6440428942116472673:4.77204192E8,6438740092256274461:-0.0019952098,2643050031313122095:-1.2318361E30,2426032824269149291:-7.472027E35,-2169609120485731243:7.746303E31,-382524493349727536:3.1660177E-28,5087097254091687936:-7.0834307E-28,8301497619465671266:2.3044203E-31,8927269159904309646:2.7101177E27,610401645634619257:-1.3781605E-35,7370696371201048576:4.2912E-41,-4497505405333536768:2.33E-43,-4520727025047240704:1.9914E-41,2211716127952666624:6.623E-41,5612484004861902848:2.587E-41,-4971197659777990656:8.7238E-41,-1046870931778371584:7.325E-42,-355810823770996736:7.294E-42,8391453106506891264:7.5625E-41,1281462743019880448:2.87E-42,-7664059664162844430:3.8694505E-34,1057106694599084032:-6.377059E-25,3248152683699541900:4.2230997E-33,1054050648184134457:2.0395964E-7,5914024482073:8.073283E-22,3405161408365:3.4245277E-22,8798656774456:3.4267796E-35,576628966068045048:-1.6564941,-7382777955725068574:3.869972E-34,-189248019257882624:-1.3256971E-21,4288551937843961224:5.8435236E-11,-3577538155243191233:-4.1629006E-5,576636419702957968:7.3060636E21,-6729228338123451142:3.8709957E-34,-8208622251512428544:-4.9206086E-17,692388284898256100:-1.6553918E-10,501597207358536840:4.2866007E27,610419878133971201:-1.0265883E-29,2211716127952683022:1.2700088E-26,7370696371201048576:4.2912E-41,-4497505405333536768:2.33E-43,-4520727025047240704:8.7238E-41,-1046870931778371584:6.623E-41,5612484004861902848:2.587E-41]
was (Author: huiwenhan):
I tested use the latest code ,it also has negative.
-8014130140593843068 [576620340710744639:-1.2199921E-19,-7951291328374999828:7.480107E-34,48126569744862489:2.384486E23,5238530791562049439:-18.706377,-6651253699672801114:-12.196953,-464715186549278845:1.880351E-20,-6440428942116472673:4.77204192E8,6438740092256274461:-0.0019952098,2643050031313122095:-1.2318361E30,2426032824269149291:-7.472027E35,-2169609120485731243:7.746303E31,-382524493349727536:3.1660177E-28,5087097254091687936:-7.0834307E-28,8301497619465671266:2.3044203E-31,8927269159904309646:2.7101177E27,610401645634619257:-1.3781605E-35,7370696371201048576:4.2912E-41,-4497505405333536768:2.33E-43,-4520727025047240704:1.9914E-41,2211716127952666624:6.623E-41,5612484004861902848:2.587E-41,-4971197659777990656:8.7238E-41,-1046870931778371584:7.325E-42,-355810823770996736:7.294E-42,8391453106506891264:7.5625E-41,1281462743019880448:2.87E-42,-7664059664162844430:3.8694505E-34,1057106694599084032:-6.377059E-25,3248152683699541900:4.2230997E-33,1054050648184134457:2.0395964E-7,5914024482073:8.073283E-22,3405161408365:3.4245277E-22,8798656774456:3.4267796E-35,576628966068045048:-1.6564941,-7382777955725068574:3.869972E-34,-189248019257882624:-1.3256971E-21,4288551937843961224:5.8435236E-11,-3577538155243191233:-4.1629006E-5,576636419702957968:7.3060636E21,-6729228338123451142:3.8709957E-34,-8208622251512428544:-4.9206086E-17,692388284898256100:-1.6553918E-10,501597207358536840:4.2866007E27,610419878133971201:-1.0265883E-29,2211716127952683022:1.2700088E-26,7370696371201048576:4.2912E-41,-4497505405333536768:2.33E-43,-4520727025047240704:8.7238E-41,-1046870931778371584:6.623E-41,5612484004861902848:2.587E-41]
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-358:
-------------------------------
Attachment: 20100407_steer_item_out_part-00000
attempt_201004011623_0070_m_000000_0.htm
please notice user -9021091696683061572
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854485#action_12854485 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
I tested use the latest code ,it also has negative.
-8014130140593843068 [576620340710744639:-1.2199921E-19,-7951291328374999828:7.480107E-34,48126569744862489:2.384486E23,5238530791562049439:-18.706377,-6651253699672801114:-12.196953,-464715186549278845:1.880351E-20,-6440428942116472673:4.77204192E8,6438740092256274461:-0.0019952098,2643050031313122095:-1.2318361E30,2426032824269149291:-7.472027E35,-2169609120485731243:7.746303E31,-382524493349727536:3.1660177E-28,5087097254091687936:-7.0834307E-28,8301497619465671266:2.3044203E-31,8927269159904309646:2.7101177E27,610401645634619257:-1.3781605E-35,7370696371201048576:4.2912E-41,-4497505405333536768:2.33E-43,-4520727025047240704:1.9914E-41,2211716127952666624:6.623E-41,5612484004861902848:2.587E-41,-4971197659777990656:8.7238E-41,-1046870931778371584:7.325E-42,-355810823770996736:7.294E-42,8391453106506891264:7.5625E-41,1281462743019880448:2.87E-42,-7664059664162844430:3.8694505E-34,1057106694599084032:-6.377059E-25,3248152683699541900:4.2230997E-33,1054050648184134457:2.0395964E-7,5914024482073:8.073283E-22,3405161408365:3.4245277E-22,8798656774456:3.4267796E-35,576628966068045048:-1.6564941,-7382777955725068574:3.869972E-34,-189248019257882624:-1.3256971E-21,4288551937843961224:5.8435236E-11,-3577538155243191233:-4.1629006E-5,576636419702957968:7.3060636E21,-6729228338123451142:3.8709957E-34,-8208622251512428544:-4.9206086E-17,692388284898256100:-1.6553918E-10,501597207358536840:4.2866007E27,610419878133971201:-1.0265883E-29,2211716127952683022:1.2700088E-26,7370696371201048576:4.2912E-41,-4497505405333536768:2.33E-43,-4520727025047240704:8.7238E-41,-1046870931778371584:6.623E-41,5612484004861902848:2.587E-41]
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854375#action_12854375 ]
Sebastian Schelter commented on MAHOUT-358:
-------------------------------------------
I ran the job with some test data from my company (half a million ratings from an e-commerce platform) and I'm seeing the negative values too.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-358:
-------------------------------
Attachment: 2010_input_steer_item_in_prefs-1.dat
input file
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853977#action_12853977 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
I have no negative ratings.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854437#action_12854437 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
here is the question from test.
1) the value has negative in
https://issues.apache.org/jira/secure/attachment/12441008/20100407_steer_item_out_part-00000
2) the recommendation count in the output file exceeds the expected count sometimes
I want 10 recommendations, the file https://issues.apache.org/jira/secure/attachment/12441008/20100407_steer_item_out_part-00000 ,it's more sometimes
3) it seems that it not take the max N items some times, you can see in the log file
4) the log seems right ,but the final output file is wrong (negative and count)
5) when use Text as output class ,there is no negative ,the recommendation count also is right.
UserID is ok now .
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854371#action_12854371 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
Yes, I know how the method works. That's as I expected and should give the desired result.
However for simplicity, I just removed the BigDecimal call, and you can try again. This time it is truly writing exactly what was in the recommended item object.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854077#action_12854077 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
You mean that you do not see those negative values? Truly strange, since that means everything's working except output?
The output is simple:
@Override
public String toString() {
StringBuilder result = new StringBuilder(200);
result.append('[');
boolean first = true;
for (RecommendedItem item : recommended) {
if (first) {
first = false;
} else {
result.append(',');
}
result.append(item.getItemID());
result.append(':');
BigDecimal bd = new BigDecimal(item.getValue()).round(ROUNDING);
result.append(bd.toPlainString());
}
result.append(']');
return result.toString();
}
I can remove the BigDecimal call, which is somewhat new, but still looks entirely correct to me.
Are you sure the output is really different?
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853929#action_12853929 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
Maybe it also clarifies to say: those valus are *not* estimated preferences. They can be very large; they are not supposed to be in a certain range.
The negative values make sense if you have negative ratings, and I suppose we should allow for that.
In that case, your results seem correct. The only change you may want then is to output very large or small numbers in scientific notation instead of a long string. I can make that change easily.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854413#action_12854413 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
the simple way to resolve the issue is that construct the output in RecommenderMapper and not use custom writable
private final Text user = new Text();
private final Text recomScore = new Text();
StringBuilder result = new StringBuilder(200);
result.append('[');
boolean first = true;
for (RecommendedItem item : recommendations) {
if (first) { first = false; }
else { result.append(','); }
result.append(item.getItemID());
result.append(':');
result.append(item.getValue());
}
result.append(']');
user.set(String.valueOf(userID));
recomScore.set(result.toString);
output.collect(userID, recomScore);
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853919#action_12853919 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
I used the latest code test again ,
the final output also is strange.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853983#action_12853983 ]
Hui Wen Han commented on MAHOUT-358:
------------------------------------
I will debug and tell you the result .
Thanks :)
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853927#action_12853927 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
Yes, the very small zero values are now being output. I think it should filter very small zero values instead, which I can add.
The part I don't quite understand are the negative values. Do you have negative ratings?
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg, screenshot-2.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854452#action_12854452 ]
Sebastian Schelter commented on MAHOUT-358:
-------------------------------------------
My results looked kinda strange (maybe I got something wrong?), here's a sample output line with negative values:
4725 [194614:8.0,178574:8.0,126959:8.0,586593851465007104:7.9832724E-28,448135168:0.0,554696704:0.0,851312640:2.92E-42,4734:0.0,26020003446784:0.0,88808125104128:0.0,143032087216128:4.9351955E-34,310771712:0.0,4192923752129888256:0.0,4211782575569502208:0.0,4678749561932482596:0.0,20431159427072:2.3857E-41,4688247212092686336:8.876E-42,4665729213955833856:8.708E-42,4665729214092410880:0.0,1339257939189301248:2.6403101E-15,0:2.9293609E10,1:1.0442734E-4,8950711844864:6.679E-42,13535:16.0,64277:16.0,69153:16.0,586593851465007104:1.0917835E-27,976241224:0.0,617497120:0.0,1011106336:2.92E-42,4789:0.0,31228707209216:0.0,526558695522304:0.0,49065706389504:4.9351955E-34,314441728:0.0,1592866893205602304:0.0,4088986986675699712:0.0,-7932527793659705308:0.0,20637317857280:1.10651E-40,0:2.14E-42,0:1.19384E-40,136577024:0.0,1354739062908387329:-4.42924794E11,1:1.1437698E-35,1:1000460.0,8950711844864:6.746E-42,95556:10.0,95899:10.0,57319:10.0,586593851465007104:1.3505299E-27,302858528:0.0,1941520672:0.0,851312640:2.92E-42,4830:0.0,242048263258112:0.0,24366435794944:0.0,37370510442496:4.9351955E-34,317063168:0.0,2955840336133357568:1.4E-45,-7583708691820445696:1.4E-45,-7581738366983469020:0.0,20813411516416:1.8913E-41,0:1.1453E-41,0:2.1608E-41,136577024:0.0,1365998061976813570:1.4776256E22,1:5.119846E-28,2:4.1850966E-16,8950711844864:6.81E-42,64277:28.0,9896:8.0,3413:8.0,586593851465007104:1.6408307E-27,187580704:0.0,3670688032:0.0,9150873888:2.92E-42,4873:0.0,8199092568064:0.0,17575006175232:0.0,124906238902272:4.9351955E-34,319946752:1.4E-45,-8312447406524334080:1.4E-45,7469291587688726528:0.0,7912896013545768996:0.0,20998095110144:9.0071E-41,4701758010974797824:3.6774E-41,4683743612465315840:3.9393E-41,4683743612601892864:0.0,1378101485975371776:3.0353137E-30,1:2.05131665E13,1:-5.0857293E-33,8950711844864:6.869E-42,59216:0.0,116360:0.0,157916:0.0,586593851465007104:2.1709452E-27,803487936:0.0,2247311552:0.0,1256407232:2.92E-42,4916:0.0,36443890122752:0.0,42048822444032:0.0,53443370680320:4.9351955E-34,322699264:0.0,-7921760551166803968:0.0,1148417904979476480:0.0,2630102182384371748:0.0,21152713932800:2.8324E-41,4692750811720056832:1.37956E-40,4692750811720056832:4.403E-41,4692750811856633856:0.0,1388516060113666048:126.626465,1:-4.649059E-16,0:5.2475385E-17,8950711844864:6.934E-42,30298:10.0,6835:10.0,30668:10.0,586593851465007104:2.802034E-27,1062420672:0.0,766525632:0.0,1404453056:2.92E-42,4966:0.0,71154723192832:0.0,71373766524928:0.0,17987323035648:4.9351955E-34,326303744:0.0,8632346981731139584:0.0,5418464838655410176:2.8E-45]
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-358:
-------------------------------
Attachment: screenshot-1.jpg
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854587#action_12854587 ]
Sean Owen commented on MAHOUT-358:
----------------------------------
I think the cause must have been related to the PriorityQueue. GenericRecommendedItem implemented Comparable. The problem may have been that it wasn't quite consistent with equals() since it was determined only on value, not item ID. I switched to a Comparator to implement ordering. I'm still surprised if that was the difference...
I also tried in a few places using a brand new local Writable object to read values instead of reusing one. I figured maybe I mistook when that was safe to do.
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Assignee: Sean Owen
> Fix For: 0.4
>
> Attachments: 20100407_steer_item_out_part-00000, 2010_input_steer_item_in_prefs-1.dat, attempt_201004011623_0070_m_000000_0.htm, screenshot-1.jpg, screenshot-2.jpg, screenshot-3.jpg
>
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-358) the pref value field of output of
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-358:
-------------------------------
Attachment: (was: screenshot-1.jpg)
> the pref value field of output of org.apache.mahout.cf.taste.hadoop.item.RecommenderJob has negative
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-358
> URL: https://issues.apache.org/jira/browse/MAHOUT-358
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
>
> In my test the input pref values all is positive.
> the output score value has negative value ,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.