You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Zheng Lv <lv...@gmail.com> on 2010/01/18 03:49:34 UTC

Exception when using SequenceFile

Hello Everyone,
    We have been using SequenceFile(GZIP) to save some data in our product,
but we got the following exceptions recently, which never occurred before:

    java.io.EOFException at
java.io.DataInputStream.readFully(DataInputStream.java:197) at
java.io.DataInputStream.readFully(DataInputStream.java:169) at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450) at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1428) at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417) at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1412) at
org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.initialize(SequenceFileRecordReader.java:50)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:414)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:517) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:303) at
org.apache.hadoop.mapred.Child.main(Child.java:170)

  We googled it, but got nothing. Now we restart the hadoop and the
application and until now the exception doesn't occur. But we want to know
why the exception happened and how to avoid it.
  Any suggestion will help, thanks a lot.
  LvZheng.

Re: Exception when using SequenceFile

Posted by Rekha Joshi <re...@yahoo-inc.com>.

I have come across this error when my gzip file was corrupt, either incompletely formed or corrupted during the data copy over network.
Cheers,
/R

On 1/18/10 8:19 AM, "Zheng Lv" <lv...@gmail.com> wrote:

Hello Everyone,
    We have been using SequenceFile(GZIP) to save some data in our product,
but we got the following exceptions recently, which never occurred before:

    java.io.EOFException at
java.io.DataInputStream.readFully(DataInputStream.java:197) at
java.io.DataInputStream.readFully(DataInputStream.java:169) at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450) at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1428) at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417) at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1412) at
org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.initialize(SequenceFileRecordReader.java:50)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:414)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:517) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:303) at
org.apache.hadoop.mapred.Child.main(Child.java:170)

  We googled it, but got nothing. Now we restart the hadoop and the
application and until now the exception doesn't occur. But we want to know
why the exception happened and how to avoid it.
  Any suggestion will help, thanks a lot.
  LvZheng.