You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Rodrigo Schmidt (JIRA)" <ji...@apache.org> on 2009/02/25 13:50:01 UTC
[jira] Created: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
------------------------------------------------------------------------------
Key: HADOOP-5326
URL: https://issues.apache.org/jira/browse/HADOOP-5326
Project: Hadoop Core
Issue Type: Bug
Components: io
Affects Versions: 0.19.0
Reporter: Rodrigo Schmidt
Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
* In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-5326:
------------------------------------
Release Note: (was: HADOOP-5326. Fixes CBZip2OutputStream data corruption problem. (Rodrigo Schmidt via zshao))
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rodrigo Schmidt updated HADOOP-5326:
------------------------------------
Fix Version/s: 0.19.0
0.19.1
Affects Version/s: 0.21.0
0.20.0
0.19.2
0.19.1
Status: Patch Available (was: Reopened)
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.1, 0.19.0, 0.19.2, 0.20.0, 0.21.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0, 0.19.1, 0.19.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677563#action_12677563 ]
Zheng Shao commented on HADOOP-5326:
------------------------------------
Nigel, sorry I should have included the hadoop unit test results in my comments. We did test the patch a lot internally here. Also I ran the hadoop unit test myself.
The reason for not adding a separate test is just like Rodrigo has said. The data that corrupts the current implementation is about 1MB in size but we cannot disclose it. There is another public data set that breaks the old code, but it is about 20MB in size. I don't think we want to include that big amount of data into the hadoop codebase.
Also as you can see, the patch is just 2 bytes inside the BZip2 algorithm itself, literally.
I will definitely be more carefully next time.
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-5326) bzip2 codec
(CBZip2OutputStream) creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676959#action_12676959 ]
rschmidt edited comment on HADOOP-5326 at 2/26/09 3:07 AM:
------------------------------------------------------------------
I found the problem, corrected it, and tested with my data and the data available in https://issues.apache.org/bugzilla/show_bug.cgi?id=41596
was (Author: rschmidt):
I found the problem, corrected it, and tested with the my data, and the data available in https://issues.apache.org/bugzilla/show_bug.cgi?id=41596
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Fix For: 0.19.0
>
> Attachments: HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Giridharan Kesavan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677279#action_12677279 ]
Giridharan Kesavan commented on HADOOP-5326:
--------------------------------------------
Could you resubmit the patch ?
When hudson triggered the patch build, jira was down so hudson coudn't pick up the patch from jira..
Thanks
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rodrigo Schmidt updated HADOOP-5326:
------------------------------------
Attachment: HADOOP-5326.patch
I found the problem, corrected it, and tested with the my data, and the data available in https://issues.apache.org/bugzilla/show_bug.cgi?id=41596
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Fix For: 0.19.0
>
> Attachments: HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677739#action_12677739 ]
Hadoop QA commented on HADOOP-5326:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12401072/HADOOP-5326.2.patch
against trunk revision 748861.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/21/console
This message is automatically generated.
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rodrigo Schmidt reopened HADOOP-5326:
-------------------------------------
reopening the issue to resubmit the patch.
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reassigned HADOOP-5326:
-------------------------------------
Assignee: Rodrigo Schmidt
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rodrigo Schmidt updated HADOOP-5326:
------------------------------------
Attachment: HADOOP-5326.2.patch
New patch contains modified and tested code.
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Fix For: 0.19.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677160#action_12677160 ]
Zheng Shao commented on HADOOP-5326:
------------------------------------
nitpick:
The original code does only one "+" operation, while the new code requires many (depending on how compiler optimizes it).
What about a code block like this?
{code}
int i = lastShadow + NUM_OVERSHOOT_BYTES;
do {
quadrant[i] = 0;
} while (--i > 0);
{code}
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Fix For: 0.19.0
>
> Attachments: HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rodrigo Schmidt updated HADOOP-5326:
------------------------------------
Fix Version/s: 0.19.0
Status: Patch Available (was: Open)
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Fix For: 0.19.0
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677167#action_12677167 ]
Rodrigo Schmidt commented on HADOOP-5326:
-----------------------------------------
I doubt the compiler doesn't optimize that. Besides, this is just an initialization loop, executed once per block.
I can rewrite it so that it looks closer to the original. I don't like this style, though.
{noformat}
for (int i = lastShadow + NUM_OVERSHOOT_BYTES +1; --i >= 0;) {
quadrant[i] = 0;
}
{noformat}
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Fix For: 0.19.0
>
> Attachments: HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rodrigo Schmidt updated HADOOP-5326:
------------------------------------
Fix Version/s: (was: 0.19.1)
(was: 0.19.0)
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677256#action_12677256 ]
Nigel Daley commented on HADOOP-5326:
-------------------------------------
Rodrigo, why no unit test?
Zheng, why didn't you wait for +1 from Hudson?
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677395#action_12677395 ]
Hudson commented on HADOOP-5326:
--------------------------------
Integrated in Hadoop-trunk #767 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/767/])
. Fixes CBZip2OutputStream data corruption problem. (Rodrigo Schmidt via zshao)
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rodrigo Schmidt updated HADOOP-5326:
------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677277#action_12677277 ]
Rodrigo Schmidt commented on HADOOP-5326:
-----------------------------------------
Hi Nigel,
Thanks for the feedback.
I assumed the bzip2 codec already had meaningful unit tests. As you can see in the patch, it corrects a very simple bug and I didn't think that adding a specific test for it would be any helpful/useful (unless someone deletes the +1 I added to the loop initialization).
Besides, the data that triggered the bug and I was using for testing cannot be disclosed.
Cheers,
Rodrigo
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677177#action_12677177 ]
Zheng Shao commented on HADOOP-5326:
------------------------------------
+1
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Fix For: 0.19.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5326) bzip2 codec (CBZip2OutputStream)
creates corrupted output file for some inputs
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HADOOP-5326:
-------------------------------
Resolution: Fixed
Fix Version/s: (was: 0.19.0)
0.21.0
0.20.0
0.19.2
Release Note: HADOOP-5326. Fixes CBZip2OutputStream data corruption problem. (Rodrigo Schmidt via zshao)
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Thanks Rodrigo!
> bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs
> ------------------------------------------------------------------------------
>
> Key: HADOOP-5326
> URL: https://issues.apache.org/jira/browse/HADOOP-5326
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.19.0
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Fix For: 0.19.2, 0.20.0, 0.21.0
>
> Attachments: HADOOP-5326.2.patch, HADOOP-5326.patch
>
>
> Bzip2 codec generated corrupted output files in some test executions I performed. This bug is probably related to https://issues.apache.org/bugzilla/show_bug.cgi?id=41596.
> * In my case, the problem seems to be at the BWT (Burrows-Wheeler Transform) implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.