You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Chris Douglas (JIRA)" <ji...@apache.org> on 2009/09/02 22:49:32 UTC
[jira] Created: (MAPREDUCE-946) Fix regression in LineRecordReader
to comply with line length parameters
Fix regression in LineRecordReader to comply with line length parameters
------------------------------------------------------------------------
Key: MAPREDUCE-946
URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Chris Douglas
MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-946) Fix regression in
LineRecordReader to comply with line length parameters
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756423#action_12756423 ]
Hudson commented on MAPREDUCE-946:
----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #43 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/43/])
. Fix a regression in LineRecordReader where the
maxBytesToConsume parameter is not set correctly.
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch, M946-1.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-946) Fix regression in LineRecordReader
to comply with line length parameters
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-946:
------------------------------------
Priority: Blocker (was: Major)
Fix Version/s: 0.21.0
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-946) Fix regression in LineRecordReader
to comply with line length parameters
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-946:
------------------------------------
Status: Patch Available (was: Open)
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch, M946-1.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-946) Fix regression in
LineRecordReader to comply with line length parameters
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755050#action_12755050 ]
Chris Douglas commented on MAPREDUCE-946:
-----------------------------------------
The test failure, TestQueueManager.testUserDisabledACLForJobKill, is not related
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch, M946-1.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-946) Fix regression in LineRecordReader
to comply with line length parameters
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-946:
------------------------------------
Attachment: M946-1.patch
Restored the previous behavior for uncompressed data and updated to use maxint for bytesconsumed from compressed streams. One could update LineReader to pass a Seekable to track bytes consumed from the underlying stream, which would fit the contract for HADOOP-3144 fairly well. However, this would only be effective when one read compressed, corrupt text data using a splittable codec. This should be sufficient.
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch, M946-1.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-946) Fix regression in
LineRecordReader to comply with line length parameters
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754931#action_12754931 ]
Hadoop QA commented on MAPREDUCE-946:
-------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12419496/M946-1.patch
against trunk revision 814467.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/73/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/73/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/73/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/73/console
This message is automatically generated.
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch, M946-1.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-946) Fix regression in LineRecordReader
to comply with line length parameters
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-946:
------------------------------------
Status: Open (was: Patch Available)
Hong points out that this patch offers the max bytes consumed in compressed bytes to the LineReader, which tracks uncompressed bytes. If this is to work for splittable codecs per MAPREDUCE-830 and maintain progress as in MAPREDUCE-773, we'll need to come up with a better solution.
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-946) Fix regression in
LineRecordReader to comply with line length parameters
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754109#action_12754109 ]
Hadoop QA commented on MAPREDUCE-946:
-------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12419279/M946-0.patch
against trunk revision 813660.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/63/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/63/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/63/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/63/console
This message is automatically generated.
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-946) Fix regression in LineRecordReader
to comply with line length parameters
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-946:
------------------------------------
Assignee: Chris Douglas
Status: Patch Available (was: Open)
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-946) Fix regression in
LineRecordReader to comply with line length parameters
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756220#action_12756220 ]
Hong Tang commented on MAPREDUCE-946:
-------------------------------------
patch looks good. +1.
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch, M946-1.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-946) Fix regression in LineRecordReader
to comply with line length parameters
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-946:
------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I committed this.
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch, M946-1.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-946) Fix regression in LineRecordReader
to comply with line length parameters
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-946:
------------------------------------
Attachment: M946-0.patch
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-946) Fix regression in
LineRecordReader to comply with line length parameters
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756645#action_12756645 ]
Hudson commented on MAPREDUCE-946:
----------------------------------
Integrated in Hadoop-Mapreduce-trunk #85 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/85/])
. Fix a regression in LineRecordReader where the
maxBytesToConsume parameter is not set correctly.
> Fix regression in LineRecordReader to comply with line length parameters
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-946
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-946
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: M946-0.patch, M946-1.patch
>
>
> MAPREDUCE-773 accidentally changed code introduced in HADOOP-3144 controlling max line lengths. The behavior should be restored.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.