You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Han Hui Wen (JIRA)" <ji...@apache.org> on 2010/08/17 17:10:17 UTC
[jira] Issue Comment Edited: (MAHOUT-471)
RowSimilarityJob-Mapper-EntriesToVectorsReducer failure
[ https://issues.apache.org/jira/browse/MAHOUT-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899432#action_12899432 ]
Han Hui Wen edited comment on MAHOUT-471 at 8/17/10 11:08 AM:
---------------------------------------------------------------
one possible reason as following:
We now use Varint to write int ,it actually writes a few bytes instead of one int.
In writing phase ,it's no problem ,but in reading phase it may cause one int stored in several blocks (not sure here ),
so it maybe the stream is ended,but we have not found end tag (the first bit of the byte is "1").
was (Author: huiwenhan):
one possible reason as following:
We now use Varint to write int ,it actually writes a few bytes instead of one int.
In writing phase ,it's no problem ,but in reading phase it may cause one int stored in several blocks (not sure here ),
so it maybe the steam is ended,but we have not found end tag (the first bit of the byte is "1").
> RowSimilarityJob-Mapper-EntriesToVectorsReducer failure
> --------------------------------------------------------
>
> Key: MAHOUT-471
> URL: https://issues.apache.org/jira/browse/MAHOUT-471
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Han Hui Wen
> Priority: Minor
> Fix For: 0.4
>
>
> I used Boolean Data and SIMILARITY_TANIMOTO_COEFFICIENT
> java.io.IOException: Task: attempt_201008101359_0084_r_000000_0 - The reduce copier failed
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:380)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.io.IOException: Intermediate merge failed
> at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.doInMemMerge(ReduceTask.java:2576)
> at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.run(ReduceTask.java:2501)
> Caused by: java.lang.RuntimeException: java.io.EOFException
> at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:103)
> at org.apache.hadoop.mapred.Merger$MergeQueue.lessThan(Merger.java:373)
> at org.apache.hadoop.util.PriorityQueue.upHeap(PriorityQueue.java:123)
> at org.apache.hadoop.util.PriorityQueue.put(PriorityQueue.java:50)
> at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:447)
> at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:381)
> at org.apache.hadoop.mapred.Merger.merge(Merger.java:107)
> at org.apache.hadoop.mapred.Merger.merge(Merger.java:93)
> at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.doInMemMerge(ReduceTask.java:2551)
> ... 1 more
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readByte(DataInputStream.java:250)
> at org.apache.mahout.math.Varint.readUnsignedVarInt(Varint.java:159)
> at org.apache.mahout.math.Varint.readSignedVarInt(Varint.java:140)
> at org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey.readFields(SimilarityMatrixEntryKey.java:65)
> at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:97)
> ... 9 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.