You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Ankit Modi (JIRA)" <ji...@apache.org> on 2009/11/25 04:20:39 UTC
[jira] Created: (PIG-1107) PigLineRecordReader bails out on an
empty line for compressed data
PigLineRecordReader bails out on an empty line for compressed data
------------------------------------------------------------------
Key: PIG-1107
URL: https://issues.apache.org/jira/browse/PIG-1107
Project: Pig
Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Ankit Modi
Assignee: Ankit Modi
Fix For: 0.6.0
PigLineRecordReader bails out with an exception when it encounters an empty line in a compressed file
java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.pig.impl.io.PigLineRecordReader$LineReader.getNext(PigLineRecordReader.java:136)
at org.apache.pig.impl.io.PigLineRecordReader.next(PigLineRecordReader.java:57)
at org.apache.pig.builtin.PigStorage.getNext(PigStorage.java:121)
at org.apache.pig.backend.executionengine.PigSlice.next(PigSlice.java:139)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:164)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:140)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1107) PigLineRecordReader bails out on an
empty line for compressed data
Posted by "Ankit Modi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ankit Modi updated PIG-1107:
----------------------------
Status: Patch Available (was: Open)
> PigLineRecordReader bails out on an empty line for compressed data
> ------------------------------------------------------------------
>
> Key: PIG-1107
> URL: https://issues.apache.org/jira/browse/PIG-1107
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Ankit Modi
> Assignee: Ankit Modi
> Fix For: 0.6.0
>
> Attachments: pig_piglinerecordreader_bug.patch
>
>
> PigLineRecordReader bails out with an exception when it encounters an empty line in a compressed file
> java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.pig.impl.io.PigLineRecordReader$LineReader.getNext(PigLineRecordReader.java:136)
> at org.apache.pig.impl.io.PigLineRecordReader.next(PigLineRecordReader.java:57)
> at org.apache.pig.builtin.PigStorage.getNext(PigStorage.java:121)
> at org.apache.pig.backend.executionengine.PigSlice.next(PigSlice.java:139)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:164)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:140)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1107) PigLineRecordReader bails out on an
empty line for compressed data
Posted by "Ankit Modi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ankit Modi updated PIG-1107:
----------------------------
Attachment: pig_piglinerecordreader_bug.patch
Submitting a small patch. It has 2 new unit tests for the patch applied.
> PigLineRecordReader bails out on an empty line for compressed data
> ------------------------------------------------------------------
>
> Key: PIG-1107
> URL: https://issues.apache.org/jira/browse/PIG-1107
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Ankit Modi
> Assignee: Ankit Modi
> Fix For: 0.6.0
>
> Attachments: pig_piglinerecordreader_bug.patch
>
>
> PigLineRecordReader bails out with an exception when it encounters an empty line in a compressed file
> java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.pig.impl.io.PigLineRecordReader$LineReader.getNext(PigLineRecordReader.java:136)
> at org.apache.pig.impl.io.PigLineRecordReader.next(PigLineRecordReader.java:57)
> at org.apache.pig.builtin.PigStorage.getNext(PigStorage.java:121)
> at org.apache.pig.backend.executionengine.PigSlice.next(PigSlice.java:139)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:164)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:140)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1107) PigLineRecordReader bails out on an
empty line for compressed data
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783674#action_12783674 ]
Olga Natkovich commented on PIG-1107:
-------------------------------------
Looks like this fix is specific to windows; however, we are seeing this issue on Linux. There must be more to this issue.
> PigLineRecordReader bails out on an empty line for compressed data
> ------------------------------------------------------------------
>
> Key: PIG-1107
> URL: https://issues.apache.org/jira/browse/PIG-1107
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Ankit Modi
> Assignee: Ankit Modi
> Fix For: 0.6.0
>
> Attachments: pig_piglinerecordreader_bug.patch
>
>
> PigLineRecordReader bails out with an exception when it encounters an empty line in a compressed file
> java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.pig.impl.io.PigLineRecordReader$LineReader.getNext(PigLineRecordReader.java:136)
> at org.apache.pig.impl.io.PigLineRecordReader.next(PigLineRecordReader.java:57)
> at org.apache.pig.builtin.PigStorage.getNext(PigStorage.java:121)
> at org.apache.pig.backend.executionengine.PigSlice.next(PigSlice.java:139)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:164)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:140)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1107) PigLineRecordReader bails out on an
empty line for compressed data
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-1107:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I was incorrect in my previous comment. The offending code is executed regardless of the OS. Committed the patch to the trunk and 0.6 branch.
> PigLineRecordReader bails out on an empty line for compressed data
> ------------------------------------------------------------------
>
> Key: PIG-1107
> URL: https://issues.apache.org/jira/browse/PIG-1107
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Ankit Modi
> Assignee: Ankit Modi
> Fix For: 0.6.0
>
> Attachments: pig_piglinerecordreader_bug.patch
>
>
> PigLineRecordReader bails out with an exception when it encounters an empty line in a compressed file
> java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.pig.impl.io.PigLineRecordReader$LineReader.getNext(PigLineRecordReader.java:136)
> at org.apache.pig.impl.io.PigLineRecordReader.next(PigLineRecordReader.java:57)
> at org.apache.pig.builtin.PigStorage.getNext(PigStorage.java:121)
> at org.apache.pig.backend.executionengine.PigSlice.next(PigSlice.java:139)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:164)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:140)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1107) PigLineRecordReader bails out on an
empty line for compressed data
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782806#action_12782806 ]
Hadoop QA commented on PIG-1107:
--------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12426168/pig_piglinerecordreader_bug.patch
against trunk revision 884235.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/57/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/57/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/57/console
This message is automatically generated.
> PigLineRecordReader bails out on an empty line for compressed data
> ------------------------------------------------------------------
>
> Key: PIG-1107
> URL: https://issues.apache.org/jira/browse/PIG-1107
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Ankit Modi
> Assignee: Ankit Modi
> Fix For: 0.6.0
>
> Attachments: pig_piglinerecordreader_bug.patch
>
>
> PigLineRecordReader bails out with an exception when it encounters an empty line in a compressed file
> java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.pig.impl.io.PigLineRecordReader$LineReader.getNext(PigLineRecordReader.java:136)
> at org.apache.pig.impl.io.PigLineRecordReader.next(PigLineRecordReader.java:57)
> at org.apache.pig.builtin.PigStorage.getNext(PigStorage.java:121)
> at org.apache.pig.backend.executionengine.PigSlice.next(PigSlice.java:139)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:164)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:140)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.