Re: Pig AvroStorage

[Russell Jurney <ru...@gmail.com>]

Try Pig 0.10, and it will probably work.

Russell Jurney http://datasyndrome.com

On Jul 31, 2012, at 1:32 PM, Meghana Narasimhan
<Me...@demandmedia.com> wrote:

> Hi ,
>
> I am using pig-0.9.2 with avro storage. When I try to use an external schema file to store my output, I get an exception with :
>
> Caused by: java.io.IOException: Invalid parameter:schema_file
>
> I followed the code at:
>
> http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java
>
> Here is my schema file :
>
>
> 1.
> {
> 2.
> "type":"record",
> 3.
> "name":"TUPLE_2",
> 4.
> "fields":[
> 5.
>  {"name":"logTimestampMs","type":["null","long"]},
> 6.
>  {"name":"queueTimestampMs","type":["null","long"]}
> 7.
> ]
> 8.
> }
> 9.
>
> 10.
>
> 11.
> Here is my Pig Storage statement :
> 12.
>
> 13.
> store imp into 'output' using org.apache.pig.piggybank.storage.avro.AvroStorage('{"schema_file":"/Users/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}');
>
> And here is my exception:
>
> ERROR 1200: could not instantiate 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments '[{"schema_file":"/user/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}]'
>
> Failed to parse: could not instantiate 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments '[{"schema_file":"/user/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}]'
>        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:180)
>        at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
>        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
>        at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
>        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>        at org.apache.pig.Main.run(Main.java:495)
>        at org.apache.pig.Main.main(Main.java:111)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.RuntimeException: could not instantiate 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments '[{"schema_file":"/user/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}]'
>        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:493)
>        at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:723)
>        at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:712)
>        at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:4340)
>        at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:5956)
>        at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1122)
>        at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:683)
>        at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:483)
>        at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:369)
>        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:171)
>        ... 15 more
> Caused by: java.lang.reflect.InvocationTargetException
>        at sun.reflect.GeneratedConstructorAccessor13.newInstance(Unknown Source)
>        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:483)
>        ... 24 more
> Caused by: java.io.IOException: Invalid parameter:schema_file
>        at org.apache.pig.piggybank.storage.avro.AvroStorage.init(AvroStorage.java:447)
>        at org.apache.pig.piggybank.storage.avro.AvroStorage.<init>(AvroStorage.java:120)
>        ... 28 more
>
>
> Thanks,
> Meg
>
> ________________________________
> Please NOTE: This electronic message, including any attachments, may include privileged, confidential and/or inside information owned by Demand Media, Inc. Any distribution or use of this communication by anyone other than the intended recipient(s) is strictly prohibited and may be unlawful. If you are not the intended recipient, please notify the sender by replying to this message and then delete it from your system. Thank you.