You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Suman Sehgal (JIRA)" <ji...@apache.org> on 2008/10/23 15:38:44 UTC

[jira] Created: (HADOOP-4506) Exception during sort validation

Exception during sort validation
--------------------------------

                 Key: HADOOP-4506
                 URL: https://issues.apache.org/jira/browse/HADOOP-4506
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
    Affects Versions: 0.19.0
            Reporter: Suman Sehgal


During sort validation, after validation job is completed, following exception is observed: 
2008-10-23 11:15:32,686 INFO org.apache.hadoop.fs.FSInputChecker: java.io.IOException: Checksum ok was sent and should not be sent again
	at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1064)
	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1613)
	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1663)
	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1590)
	at java.io.DataInputStream.readInt(DataInputStream.java:370)
	at org.apache.hadoop.io.BytesWritable.readFields(BytesWritable.java:149)
	at org.apache.hadoop.mapred.JobClient$RawSplit.readFields(JobClient.java:900)
	at org.apache.hadoop.mapred.JobClient.readSplitFile(JobClient.java:976)
	at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:369)
	at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:55)
	at java.lang.Thread.run(Thread.java:619)

After that, "job successfully completed" message comes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-4506) Exception during sort validation

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642214#action_12642214 ] 

Hairong Kuang commented on HADOOP-4506:
---------------------------------------

The error message was introduced by HADOOP-3914 in 0.19 mainly for the debugging purpose. So Suman, please no panic! Every of your jobs should still be successfully finished. Raghu has already found out the cause of multiple checksum Ok messages. He is going to fix the problem and remove the logging in HADOOP-4499. I will close this jira and mark it as a duplicate of HADOOP-4499. Thanks Surman and please follow jira 4499.

> Exception during sort validation
> --------------------------------
>
>                 Key: HADOOP-4506
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4506
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Suman Sehgal
>
> During sort validation, after validation job is completed, following exception is observed: 
> 2008-10-23 11:15:32,686 INFO org.apache.hadoop.fs.FSInputChecker: java.io.IOException: Checksum ok was sent and should not be sent again
> 	at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1064)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1613)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1663)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1590)
> 	at java.io.DataInputStream.readInt(DataInputStream.java:370)
> 	at org.apache.hadoop.io.BytesWritable.readFields(BytesWritable.java:149)
> 	at org.apache.hadoop.mapred.JobClient$RawSplit.readFields(JobClient.java:900)
> 	at org.apache.hadoop.mapred.JobClient.readSplitFile(JobClient.java:976)
> 	at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:369)
> 	at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:55)
> 	at java.lang.Thread.run(Thread.java:619)
> After that, "job successfully completed" message comes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (HADOOP-4506) Exception during sort validation

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang resolved HADOOP-4506.
-----------------------------------

    Resolution: Duplicate

> Exception during sort validation
> --------------------------------
>
>                 Key: HADOOP-4506
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4506
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Suman Sehgal
>
> During sort validation, after validation job is completed, following exception is observed: 
> 2008-10-23 11:15:32,686 INFO org.apache.hadoop.fs.FSInputChecker: java.io.IOException: Checksum ok was sent and should not be sent again
> 	at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1064)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1613)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1663)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1590)
> 	at java.io.DataInputStream.readInt(DataInputStream.java:370)
> 	at org.apache.hadoop.io.BytesWritable.readFields(BytesWritable.java:149)
> 	at org.apache.hadoop.mapred.JobClient$RawSplit.readFields(JobClient.java:900)
> 	at org.apache.hadoop.mapred.JobClient.readSplitFile(JobClient.java:976)
> 	at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:369)
> 	at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:55)
> 	at java.lang.Thread.run(Thread.java:619)
> After that, "job successfully completed" message comes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-4506) Exception during sort validation

Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642205#action_12642205 ] 

Robert Chansler commented on HADOOP-4506:
-----------------------------------------

Hairong will take the lead investigating this.

Devaraj, thanks for the clue!






> Exception during sort validation
> --------------------------------
>
>                 Key: HADOOP-4506
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4506
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Suman Sehgal
>
> During sort validation, after validation job is completed, following exception is observed: 
> 2008-10-23 11:15:32,686 INFO org.apache.hadoop.fs.FSInputChecker: java.io.IOException: Checksum ok was sent and should not be sent again
> 	at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1064)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1613)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1663)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1590)
> 	at java.io.DataInputStream.readInt(DataInputStream.java:370)
> 	at org.apache.hadoop.io.BytesWritable.readFields(BytesWritable.java:149)
> 	at org.apache.hadoop.mapred.JobClient$RawSplit.readFields(JobClient.java:900)
> 	at org.apache.hadoop.mapred.JobClient.readSplitFile(JobClient.java:976)
> 	at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:369)
> 	at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:55)
> 	at java.lang.Thread.run(Thread.java:619)
> After that, "job successfully completed" message comes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-4506) Exception during sort validation

Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642201#action_12642201 ] 

Devaraj Das commented on HADOOP-4506:
-------------------------------------

Raghu could you please see whether this is related to hadoop-4499?
Thanks,
Devaraj






> Exception during sort validation
> --------------------------------
>
>                 Key: HADOOP-4506
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4506
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Suman Sehgal
>
> During sort validation, after validation job is completed, following exception is observed: 
> 2008-10-23 11:15:32,686 INFO org.apache.hadoop.fs.FSInputChecker: java.io.IOException: Checksum ok was sent and should not be sent again
> 	at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1064)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1613)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1663)
> 	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1590)
> 	at java.io.DataInputStream.readInt(DataInputStream.java:370)
> 	at org.apache.hadoop.io.BytesWritable.readFields(BytesWritable.java:149)
> 	at org.apache.hadoop.mapred.JobClient$RawSplit.readFields(JobClient.java:900)
> 	at org.apache.hadoop.mapred.JobClient.readSplitFile(JobClient.java:976)
> 	at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:369)
> 	at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:55)
> 	at java.lang.Thread.run(Thread.java:619)
> After that, "job successfully completed" message comes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.