[jira] Commented: (MAHOUT-471) RowSimilarityJob-Mapper-EntriesToVectorsReducer failure

["Sebastian Schelter (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAHOUT-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898814#action_12898814 ] 

Sebastian Schelter commented on MAHOUT-471:
-------------------------------------------

Can't reproduce this.

We have a small intregration test that runs recommendation with boolean data which also involves RowSimilarityJob and works without problems (org.apache.mahout.cf.taste.hadoop.item.RecommenderJobTest.testCompleteJobBoolean()).

The only location in the code where SimilarityMatrixEntryKeys are created is org.apache.mahout.math.hadoop.similarity.RowSimilarityJob.SimilarityReducer.reduce(...) and I can't see a way how wrong data could find its way in there.

Han Hui, can you give us more details about your input data? Did you use any other parameters for the job?

> RowSimilarityJob-Mapper-EntriesToVectorsReducer  failure
> --------------------------------------------------------
>
>                 Key: MAHOUT-471
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-471
>             Project: Mahout
>          Issue Type: Bug
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Han Hui Wen 
>            Priority: Minor
>             Fix For: 0.4
>
>
> I used Boolean Data and SIMILARITY_TANIMOTO_COEFFICIENT
> java.io.IOException: Task: attempt_201008101359_0084_r_000000_0 - The reduce copier failed
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:380)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.io.IOException: Intermediate merge failed
> 	at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.doInMemMerge(ReduceTask.java:2576)
> 	at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.run(ReduceTask.java:2501)
> Caused by: java.lang.RuntimeException: java.io.EOFException
> 	at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:103)
> 	at org.apache.hadoop.mapred.Merger$MergeQueue.lessThan(Merger.java:373)
> 	at org.apache.hadoop.util.PriorityQueue.upHeap(PriorityQueue.java:123)
> 	at org.apache.hadoop.util.PriorityQueue.put(PriorityQueue.java:50)
> 	at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:447)
> 	at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:381)
> 	at org.apache.hadoop.mapred.Merger.merge(Merger.java:107)
> 	at org.apache.hadoop.mapred.Merger.merge(Merger.java:93)
> 	at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.doInMemMerge(ReduceTask.java:2551)
> 	... 1 more
> Caused by: java.io.EOFException
> 	at java.io.DataInputStream.readByte(DataInputStream.java:250)
> 	at org.apache.mahout.math.Varint.readUnsignedVarInt(Varint.java:159)
> 	at org.apache.mahout.math.Varint.readSignedVarInt(Varint.java:140)
> 	at org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey.readFields(SimilarityMatrixEntryKey.java:65)
> 	at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:97)
> 	... 9 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.