You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "dhruba borthakur (JIRA)" <ji...@apache.org> on 2007/04/17 00:27:15 UTC
[jira] Created: (HADOOP-1262) file corruption detected because dfs
client does not use replica blocks for checksum file
file corruption detected because dfs client does not use replica blocks for checksum file
-----------------------------------------------------------------------------------------
Key: HADOOP-1262
URL: https://issues.apache.org/jira/browse/HADOOP-1262
Project: Hadoop
Issue Type: Bug
Components: dfs
Reporter: dhruba borthakur
A block of a crc file was corrupted. This caused the DFS client to detect a CRc corruption. The client tried all the three replicas of the data file. It did not try any replicas of the CRC file. This caused the client to abort the read request with a bad-CRC message.
07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx:50010
07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=zz:50010
07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx.:50010
07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1262) file corruption detected because dfs
client does not use replica blocks for checksum file
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-1262:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I've just committed this. Thanks Hairong!
> file corruption detected because dfs client does not use replica blocks for checksum file
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1262
> URL: https://issues.apache.org/jira/browse/HADOOP-1262
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.3
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: newSource.patch
>
>
> A block of a crc file was corrupted. This caused the DFS client to detect a CRc corruption. The client tried all the three replicas of the data file. It did not try any replicas of the CRC file. This caused the client to abort the read request with a bad-CRC message.
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=zz:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx.:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1262) file corruption detected because dfs
client does not use replica blocks for checksum file
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang reassigned HADOOP-1262:
-------------------------------------
Assignee: Hairong Kuang
> file corruption detected because dfs client does not use replica blocks for checksum file
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1262
> URL: https://issues.apache.org/jira/browse/HADOOP-1262
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
>
> A block of a crc file was corrupted. This caused the DFS client to detect a CRc corruption. The client tried all the three replicas of the data file. It did not try any replicas of the CRC file. This caused the client to abort the read request with a bad-CRC message.
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=zz:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx.:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1262) file corruption detected because
dfs client does not use replica blocks for checksum file
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12491478 ]
dhruba borthakur commented on HADOOP-1262:
------------------------------------------
+1. looks good.
> file corruption detected because dfs client does not use replica blocks for checksum file
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1262
> URL: https://issues.apache.org/jira/browse/HADOOP-1262
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Attachments: newSource.patch
>
>
> A block of a crc file was corrupted. This caused the DFS client to detect a CRc corruption. The client tried all the three replicas of the data file. It did not try any replicas of the CRC file. This caused the client to abort the read request with a bad-CRC message.
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=zz:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx.:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1262) file corruption detected because
dfs client does not use replica blocks for checksum file
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492252 ]
Hadoop QA commented on HADOOP-1262:
-----------------------------------
Integrated in Hadoop-Nightly #71 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/71/)
> file corruption detected because dfs client does not use replica blocks for checksum file
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1262
> URL: https://issues.apache.org/jira/browse/HADOOP-1262
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.3
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: newSource.patch
>
>
> A block of a crc file was corrupted. This caused the DFS client to detect a CRc corruption. The client tried all the three replicas of the data file. It did not try any replicas of the CRC file. This caused the client to abort the read request with a bad-CRC message.
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=zz:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx.:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1262) file corruption detected because dfs
client does not use replica blocks for checksum file
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1262:
----------------------------------
Attachment: newSource.patch
This patch makes sure that a dfs client tries to read from a different replica of the checksum file when a checksum error is detected.
> file corruption detected because dfs client does not use replica blocks for checksum file
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1262
> URL: https://issues.apache.org/jira/browse/HADOOP-1262
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Attachments: newSource.patch
>
>
> A block of a crc file was corrupted. This caused the DFS client to detect a CRc corruption. The client tried all the three replicas of the data file. It did not try any replicas of the CRC file. This caused the client to abort the read request with a bad-CRC message.
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=zz:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx.:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1262) file corruption detected because
dfs client does not use replica blocks for checksum file
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12491488 ]
Hadoop QA commented on HADOOP-1262:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12356185/newSource.patch applied and successfully tested against trunk revision r532083.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/74/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/74/console
> file corruption detected because dfs client does not use replica blocks for checksum file
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1262
> URL: https://issues.apache.org/jira/browse/HADOOP-1262
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.3
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: newSource.patch
>
>
> A block of a crc file was corrupted. This caused the DFS client to detect a CRc corruption. The client tried all the three replicas of the data file. It did not try any replicas of the CRC file. This caused the client to abort the read request with a bad-CRC message.
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=zz:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx.:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1262) file corruption detected because dfs
client does not use replica blocks for checksum file
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1262:
----------------------------------
Fix Version/s: 0.13.0
Affects Version/s: 0.12.3
Status: Patch Available (was: Open)
> file corruption detected because dfs client does not use replica blocks for checksum file
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1262
> URL: https://issues.apache.org/jira/browse/HADOOP-1262
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.3
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: newSource.patch
>
>
> A block of a crc file was corrupted. This caused the DFS client to detect a CRc corruption. The client tried all the three replicas of the data file. It did not try any replicas of the CRC file. This caused the client to abort the read request with a bad-CRC message.
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=zz:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in data stream at block=blk_6205660483922449140 on datanode=xx.:50010
> 07/04/16 20:42:26 INFO fs.FileSystem: Found checksum error in checksum stream at block=blk_-3722915954820866561 on datanode=yy:50010
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.