You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2010/12/01 19:03:13 UTC
[jira] Created: (HDFS-1524) Image loader should make sure to read
every byte in image file
Image loader should make sure to read every byte in image file
--------------------------------------------------------------
Key: HDFS-1524
URL: https://issues.apache.org/jira/browse/HDFS-1524
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 0.22.0, 0.23.0
Reporter: Hairong Kuang
Assignee: Hairong Kuang
Fix For: 0.22.0, 0.23.0
When I work on HDFS-1070, I come across a very strange bug. Occasionally when loading a compressed image, NameNode throws an exception complaining that the image file is corrupt. However, the result returned by the md5sum utility matches the checksum value stored in the VERSION file.
It turns out the image loader leaves 4 bytes unread after loading all the real data of an image. Those unread bytes may be some compression-related meta-info. The image loader should make sure to read to the end of file in order for an correct checksum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1524) Image loader should make sure to read
every byte in image file
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HDFS-1524:
--------------------------------
Attachment: eofImageFile.patch
> Image loader should make sure to read every byte in image file
> --------------------------------------------------------------
>
> Key: HDFS-1524
> URL: https://issues.apache.org/jira/browse/HDFS-1524
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0, 0.23.0
>
> Attachments: eofImageFile.patch
>
>
> When I work on HDFS-1070, I come across a very strange bug. Occasionally when loading a compressed image, NameNode throws an exception complaining that the image file is corrupt. However, the result returned by the md5sum utility matches the checksum value stored in the VERSION file.
> It turns out the image loader leaves 4 bytes unread after loading all the real data of an image. Those unread bytes may be some compression-related meta-info. The image loader should make sure to read to the end of file in order for an correct checksum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (HDFS-1524) Image loader should make sure to
read every byte in image file
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023029#comment-13023029 ]
Hudson commented on HDFS-1524:
------------------------------
Integrated in Hadoop-Hdfs-22-branch #35 (See [https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/])
> Image loader should make sure to read every byte in image file
> --------------------------------------------------------------
>
> Key: HDFS-1524
> URL: https://issues.apache.org/jira/browse/HDFS-1524
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.22.0, 0.23.0
>
> Attachments: eofImageFile.patch
>
>
> When I work on HDFS-1070, I come across a very strange bug. Occasionally when loading a compressed image, NameNode throws an exception complaining that the image file is corrupt. However, the result returned by the md5sum utility matches the checksum value stored in the VERSION file.
> It turns out the image loader leaves 4 bytes unread after loading all the real data of an image. Those unread bytes may be some compression-related meta-info. The image loader should make sure to read to the end of file in order for an correct checksum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1524) Image loader should make sure to read
every byte in image file
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12965873#action_12965873 ]
Hairong Kuang commented on HDFS-1524:
-------------------------------------
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appe
[exec] ar to include any new or modified tests.
[exec] Please justify why no new tests are needed for this patch.
[exec] Also please list what manual steps were performed to verify this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec] +1 system test framework. The patch passed system test framework compile.
[exec]
Failed unit tests are all known ones:
TestBlockRecovery
TestBlockTokenWithDFS
TestStorageRestore
TestBackupNode
TestHDFSTrash (timeout)
Since the change is trivial and it is hard to come up with the test case, I will not provide a unit test and commit the patch.
> Image loader should make sure to read every byte in image file
> --------------------------------------------------------------
>
> Key: HDFS-1524
> URL: https://issues.apache.org/jira/browse/HDFS-1524
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.22.0, 0.23.0
>
> Attachments: eofImageFile.patch
>
>
> When I work on HDFS-1070, I come across a very strange bug. Occasionally when loading a compressed image, NameNode throws an exception complaining that the image file is corrupt. However, the result returned by the md5sum utility matches the checksum value stored in the VERSION file.
> It turns out the image loader leaves 4 bytes unread after loading all the real data of an image. Those unread bytes may be some compression-related meta-info. The image loader should make sure to read to the end of file in order for an correct checksum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1524) Image loader should make sure to read
every byte in image file
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HDFS-1524:
--------------------------------
Priority: Blocker (was: Major)
> Image loader should make sure to read every byte in image file
> --------------------------------------------------------------
>
> Key: HDFS-1524
> URL: https://issues.apache.org/jira/browse/HDFS-1524
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.22.0, 0.23.0
>
> Attachments: eofImageFile.patch
>
>
> When I work on HDFS-1070, I come across a very strange bug. Occasionally when loading a compressed image, NameNode throws an exception complaining that the image file is corrupt. However, the result returned by the md5sum utility matches the checksum value stored in the VERSION file.
> It turns out the image loader leaves 4 bytes unread after loading all the real data of an image. Those unread bytes may be some compression-related meta-info. The image loader should make sure to read to the end of file in order for an correct checksum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1524) Image loader should make sure to read
every byte in image file
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HDFS-1524:
--------------------------------
Attachment: (was: eofImageFile.patch)
> Image loader should make sure to read every byte in image file
> --------------------------------------------------------------
>
> Key: HDFS-1524
> URL: https://issues.apache.org/jira/browse/HDFS-1524
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.22.0, 0.23.0
>
> Attachments: eofImageFile.patch
>
>
> When I work on HDFS-1070, I come across a very strange bug. Occasionally when loading a compressed image, NameNode throws an exception complaining that the image file is corrupt. However, the result returned by the md5sum utility matches the checksum value stored in the VERSION file.
> It turns out the image loader leaves 4 bytes unread after loading all the real data of an image. Those unread bytes may be some compression-related meta-info. The image loader should make sure to read to the end of file in order for an correct checksum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1524) Image loader should make sure to read
every byte in image file
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12965864#action_12965864 ]
Konstantin Boudnik commented on HDFS-1524:
------------------------------------------
+1 patch looks good.
> Image loader should make sure to read every byte in image file
> --------------------------------------------------------------
>
> Key: HDFS-1524
> URL: https://issues.apache.org/jira/browse/HDFS-1524
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.22.0, 0.23.0
>
> Attachments: eofImageFile.patch
>
>
> When I work on HDFS-1070, I come across a very strange bug. Occasionally when loading a compressed image, NameNode throws an exception complaining that the image file is corrupt. However, the result returned by the md5sum utility matches the checksum value stored in the VERSION file.
> It turns out the image loader leaves 4 bytes unread after loading all the real data of an image. Those unread bytes may be some compression-related meta-info. The image loader should make sure to read to the end of file in order for an correct checksum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-1524) Image loader should make sure to read
every byte in image file
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang resolved HDFS-1524.
---------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
I've just committed this.
> Image loader should make sure to read every byte in image file
> --------------------------------------------------------------
>
> Key: HDFS-1524
> URL: https://issues.apache.org/jira/browse/HDFS-1524
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.22.0, 0.23.0
>
> Attachments: eofImageFile.patch
>
>
> When I work on HDFS-1070, I come across a very strange bug. Occasionally when loading a compressed image, NameNode throws an exception complaining that the image file is corrupt. However, the result returned by the md5sum utility matches the checksum value stored in the VERSION file.
> It turns out the image loader leaves 4 bytes unread after loading all the real data of an image. Those unread bytes may be some compression-related meta-info. The image loader should make sure to read to the end of file in order for an correct checksum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1524) Image loader should make sure to read
every byte in image file
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HDFS-1524:
--------------------------------
Attachment: eofImageFile.patch
> Image loader should make sure to read every byte in image file
> --------------------------------------------------------------
>
> Key: HDFS-1524
> URL: https://issues.apache.org/jira/browse/HDFS-1524
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.22.0, 0.23.0
>
> Attachments: eofImageFile.patch
>
>
> When I work on HDFS-1070, I come across a very strange bug. Occasionally when loading a compressed image, NameNode throws an exception complaining that the image file is corrupt. However, the result returned by the md5sum utility matches the checksum value stored in the VERSION file.
> It turns out the image loader leaves 4 bytes unread after loading all the real data of an image. Those unread bytes may be some compression-related meta-info. The image loader should make sure to read to the end of file in order for an correct checksum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.