You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Mike Percy (JIRA)" <ji...@apache.org> on 2012/05/20 13:29:40 UTC

[jira] [Commented] (PIG-2709) PigAvroRecordReader should specify which file has a problem when throwing IOException

    [ https://issues.apache.org/jira/browse/PIG-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279725#comment-13279725 ] 

Mike Percy commented on PIG-2709:
---------------------------------

Old stack trace:

java.lang.RuntimeException: java.io.IOException: Not a data file.
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:236)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.<init>(PigRecordReader.java:109)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:118)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:614)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
	at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.io.IOException: Not a data file.
	at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:102)
	at org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97)
	at org.apache.pig.piggybank.storage.avro.PigAvroRecordReader.<init>(PigAvroRecordReader.java:53)
	at org.apache.pig.piggybank.storage.avro.PigAvroInputFormat.createRecordReader(PigAvroInputFormat.java:66)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:227)
	... 9 more


New stack trace:

java.lang.RuntimeException: java.io.IOException: Error initializing data file reader for file (hdfs://localhost/logs/data.avro)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:236)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.<init>(PigRecordReader.java:109)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:118)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:614)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
	at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.io.IOException: Error initializing data file reader for file (hdfs://localhost/logs/data.avro)
	at org.apache.pig.piggybank.storage.avro.PigAvroRecordReader.<init>(PigAvroRecordReader.java:56)
	at org.apache.pig.piggybank.storage.avro.PigAvroInputFormat.createRecordReader(PigAvroInputFormat.java:66)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:227)
	... 9 more
Caused by: java.io.IOException: Not a data file.
	at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:102)
	at org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97)
	at org.apache.pig.piggybank.storage.avro.PigAvroRecordReader.<init>(PigAvroRecordReader.java:54)
	... 11 more

                
> PigAvroRecordReader should specify which file has a problem when throwing IOException
> -------------------------------------------------------------------------------------
>
>                 Key: PIG-2709
>                 URL: https://issues.apache.org/jira/browse/PIG-2709
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Mike Percy
>         Attachments: PIG-2709-1.patch
>
>
> Today, if AvroStorage opens a file that is not valid Avro, an exception is thrown but it's unclear which file actually had the problem form looking at the M/R logs. The PigAvroRecordReader should specify the file being opened when an exception is thrown.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira