You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by John Smith <le...@gmail.com> on 2016/01/29 01:41:26 UTC

AvroStorage load doesnt work in mapreduce mode

Hi,

Im trying to load avro file or directory that contains avro file using
AvroStorage in Mapreduce mode. I tried almost all the combinations
(hdfs://, / , hdfs://ip:port/file ... ) but nothing works.


I got error:

set = load '/spool-dir/CustomerData-20160128-1501807/' USING
org.apache.pig.piggybank.storage.avro.AvroStorage ();

2016-01-29 00:25:02,568 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.Launcher - Backend error
message
Error: java.lang.RuntimeException: could not instantiate
'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments
'[no_schema_check]'
        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:772)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getLoadFunc(PigInputFormat.java:149)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:97)

or

set = load '/spool-dir/CustomerData-20160128-1501807/' USING
org.apache.pig.piggybank.storage.avro.AvroStorage ('no_schema_check');


2016-01-29 00:25:02,767 [main] ERROR org.apache.pig.tools.grunt.Grunt
- org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
Unable to open iterator for alias sensitiveSet. Backend error :
java.lang.RuntimeException: could not instantiate
'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments
'[no_schema_check]'
        at org.apache.pig.PigServer.openIterator(PigServer.java:925)


Thank you