You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "mawanqiang (JIRA)" <ji...@apache.org> on 2009/06/19 09:29:08 UTC
[jira] Created: (HADOOP-6091) Checksum Error
Checksum Error
--------------
Key: HADOOP-6091
URL: https://issues.apache.org/jira/browse/HADOOP-6091
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.19.1
Environment: linux ubuntu8.0.4 64bit
10datanode 4G of memory per node
Reporter: mawanqiang
Approximately 1 million data used to create index when nutch1.0 error.
The error is:
java.lang.RuntimeException: problem advancing post rec#6758513
at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:883)
at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:237)
at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:233)
at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:79)
at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:50)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436)
at org.apache.hadoop.mapred.Child.main(Child.java:158)
Caused by: org.apache.hadoop.fs.ChecksumException: Checksum Error
at org.apache.hadoop.mapred.IFileInputStream.doRead(IFileInputStream.java:153)
at org.apache.hadoop.mapred.IFileInputStream.read(IFileInputStream.java:90)
at org.apache.hadoop.mapred.IFile$Reader.readData(IFile.java:301)
at org.apache.hadoop.mapred.IFile$Reader.rejigData(IFile.java:331)
at org.apache.hadoop.mapred.IFile$Reader.readNextBlock(IFile.java:315)
at org.apache.hadoop.mapred.IFile$Reader.next(IFile.java:377)
at org.apache.hadoop.mapred.Merger$Segment.next(Merger.java:174)
at org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:277)
at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:297)
at org.apache.hadoop.mapred.Task$ValuesIterator.readNextKey(Task.java:922)
at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:881)
... 6 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-6091) Checksum Error
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon resolved HADOOP-6091.
---------------------------------
Resolution: Invalid
Please submit this to the NUTCH JIRA, or provide an explanation of why this is a Hadoop bug.
> Checksum Error
> --------------
>
> Key: HADOOP-6091
> URL: https://issues.apache.org/jira/browse/HADOOP-6091
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Environment: linux ubuntu8.0.4 64bit
> 10datanode 4G of memory per node
> Reporter: mawanqiang
>
> Approximately 1 million data used to create index when nutch1.0 error.
> The error is:
> java.lang.RuntimeException: problem advancing post rec#6758513
> at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:883)
> at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:237)
> at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:233)
> at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:79)
> at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:50)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436)
> at org.apache.hadoop.mapred.Child.main(Child.java:158)
> Caused by: org.apache.hadoop.fs.ChecksumException: Checksum Error
> at org.apache.hadoop.mapred.IFileInputStream.doRead(IFileInputStream.java:153)
> at org.apache.hadoop.mapred.IFileInputStream.read(IFileInputStream.java:90)
> at org.apache.hadoop.mapred.IFile$Reader.readData(IFile.java:301)
> at org.apache.hadoop.mapred.IFile$Reader.rejigData(IFile.java:331)
> at org.apache.hadoop.mapred.IFile$Reader.readNextBlock(IFile.java:315)
> at org.apache.hadoop.mapred.IFile$Reader.next(IFile.java:377)
> at org.apache.hadoop.mapred.Merger$Segment.next(Merger.java:174)
> at org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:277)
> at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:297)
> at org.apache.hadoop.mapred.Task$ValuesIterator.readNextKey(Task.java:922)
> at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:881)
> ... 6 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.