You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Dan Adkins <da...@quantcast.com> on 2010/09/08 20:27:51 UTC
ChecksumException on jobtracker restart
I restarted my jobtracker and got the following errors for _all_ active jobs at the time, after which the job history was subsequently deleted.
10/09/08 00:59:58 INFO mapred.JobTracker: Cleaning up the system directory
10/09/08 00:59:58 INFO jobhistory.JobHistory: Inited the done directory to file:/home/mr20/logs/history/done
10/09/08 00:59:58 INFO jobhistory.JobHistory: Moving log file from last run: file:/home/mr20/logs/history/job_201009080052_0001_dadkins
10/09/08 00:59:58 INFO jobhistory.JobHistory: Moving file:/home/mr20/logs/history/job_201009080052_0001_dadkins to file:/home/mr20/logs/history/done/job_201009080052_0001_dadkins.201009080059.old
10/09/08 00:59:58 INFO fs.FSInputChecker: Found checksum error: b[3072, 3072]=
org.apache.hadoop.fs.ChecksumException: Checksum error: file:/home/mr20/logs/history/job_201009080052_0001_dadkins at 523264
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:246)
at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:273)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:225)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:193)
at java.io.DataInputStream.read(DataInputStream.java:83)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:76)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:49)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:101)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:244)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:216)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:167)
at org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:60)
at org.apache.hadoop.fs.FileSystem.moveFromLocalFile(FileSystem.java:1482)
at org.apache.hadoop.mapreduce.jobhistory.JobHistory.moveToDoneNow(JobHistory.java:348)
at org.apache.hadoop.mapreduce.jobhistory.JobHistory.moveOldFiles(JobHistory.java:390)
at org.apache.hadoop.mapreduce.jobhistory.JobHistory.initDone(JobHistory.java:165)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1591)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:258)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:250)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:245)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4164)
10/09/08 00:59:58 WARN jobhistory.JobHistory: Unable to move file:/home/mr20/logs/history/job_201009080052_0001_dadkins, deleting it
Is this the expected behavior?
-Dan