You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Pavel Mizenin <pa...@ecofactor.com> on 2011/11/23 01:26:12 UTC

how to save pig bag in json format

I'm running Pig

    example$ pig --version
    Apache Pig version 0.8.1-cdh3u1 (rexported)
    compiled Jul 18 2011, 08:29:40

on very simple dataset

    example$ hadoop fs -cat /user/pavel/trivial.log
    1    one
    2    two
    3    three

I'm trying to save the bag format as json by using the following script:

    REGISTER ./pig.jar;
    A = LOAD 'trivial.log' USING PigStorage('\t') AS (mynum: int, mynumstr:
chararray);
    B = GROUP A BY mynum;
    DUMP B;
    STORE B into 'trivial_json.out' USING JsonStorage();


and I get an error:
Backend error message

    ---------------------
    java.lang.NullPointerException
        at org.apache.pig.ResourceSchema.<init>(ResourceSchema.java:239)
        at
org.apache.pig.builtin.JsonStorage.prepareToWrite(JsonStorage.java:129)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:124)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:85)
        at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:559)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
        at org.apache.hadoop.mapred.Child.main(Child.java:264)

    Pig Stack Trace
    ---------------
    ERROR 2997: Unable to recreate exception from backed error:
java.lang.NullPointerException

    org.apache.pig.backend.executionengine.ExecException: ERROR 2997:
Unable to recreate exception from backed error:
java.lang.NullPointerException
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:221)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:154)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:337)
        at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
        at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
        at org.apache.pig.PigServer.execute(PigServer.java:1201)
        at org.apache.pig.PigServer.access$100(PigServer.java:129)
        at org.apache.pig.PigServer$Graph.execute(PigServer.java:1528)
        at org.apache.pig.PigServer.executeBatchEx(PigServer.java:373)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:340)
        at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:115)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:172)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
        at org.apache.pig.Main.run(Main.java:396)
        at org.apache.pig.Main.main(Main.java:107)

================================================================================


I'm not strong enough in Java to debug in minutes, can somebody suggest
what might be going on?

Thanks much!
-Pavel