You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Sergey Svinarchuk (JIRA)" <ji...@apache.org> on 2014/01/08 15:35:51 UTC
[jira] [Commented] (PIG-3406) JsonStorage generates NPE after
successfully saving records
[ https://issues.apache.org/jira/browse/PIG-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13865481#comment-13865481 ]
Sergey Svinarchuk commented on PIG-3406:
----------------------------------------
I wrote unit test that must be reproduce exception, but problem that was description in this ticket not reproduced.
@Test
public void testJsongIssueReproduce() throws Exception {
PigServer pigServer = new PigServer("local");
File input = Util.createInputFile("tmp", "",
new String[] {"A,1", "B,2", "C,3", "D,2",
"A,5", "B,5", "C,8", "A,8",
"D,8", "A,9"});
pigServer.registerQuery("IP = load '"
+ Util.generateURI(Util.encodeEscape(input.toString()),
pigServer.getPigContext()) + "' using "
+ PigStorage.class.getName() + "(',');");
pigServer.registerQuery("FILTERED_DATA = filter IP by $1 > '3';");
pigServer.registerQuery("STREAMED_DATA = stream FILTERED_DATA through `perl -ne 'print $_'` as (f0:chararray, f1:int);");
pigServer.registerQuery("OP = filter STREAMED_DATA by f1 > 6;");
pigServer.store("OP", "/home/ssvinarchuk/obs.json", "org.apache.pig.builtin.JsonStorage");
pigServer = null;
}
> JsonStorage generates NPE after successfully saving records
> -----------------------------------------------------------
>
> Key: PIG-3406
> URL: https://issues.apache.org/jira/browse/PIG-3406
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: OSX 10.8.4, running PigServer in local mode from a Groovy script.
> Reporter: Kirk Stork
>
> Make a new PigServer("local") instance, process some queries and save an alias using
> PigServer piggy = new PigServer("local")
> ... some queries..
> def job = piggy.store("c_observer_id", "obs.json", 'org.apache.pig.builtin.JsonStorage');
> The local directory obs.json is created and populated with correct results.
> but then an NPE is thrown, apparently during some attempt to store the schema.
> HadoopVersion PigVersion UserId StartedAt FinishedAt Features
> 1.1.0 0.11.1 kirk 2013-08-01 14:09:41 2013-08-01 14:09:42 GROUP_BY
> Success!
> Job Stats (time in seconds):
> JobId Alias Feature Outputs
> job_local_0001 c_observer_id,g_observer_id,obs GROUP_BY,COMBINER obs.json,
> Input(s):
> Successfully read records from: "/Volumes/Work/work/combatxxi-acceptance-testing/pig-input/Ambush_Mine_RPG-7.cxxi/Replication_1_SIMKIT_CONGRUENTIAL/ObserveLogger.log"
> Output(s):
> Successfully stored records in: "obs.json"
> Job DAG:
> job_local_0001
> org.apache.pig.PigException: ERROR 1002: Unable to store alias c_observer_id
> at org.apache.pig.PigServer.storeEx(PigServer.java:935)
> at org.apache.pig.PigServer.store(PigServer.java:898)
> at org.apache.pig.PigServer$store.call(Unknown Source)
> at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
> at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
> at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
> at edu.nps.cxxi.testbench.lff33.ObserveLoggerService.oink(ObserveLoggerService.groovy:27)
> at edu.nps.cxxi.testbench.lff33.ObserveLoggerService$oink.call(Unknown Source)
> at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
> at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
> at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
> at edu.nps.cxxi.testbench.lff33.ObserveLoggerServiceTests.testSomething(ObserveLoggerServiceTests.groovy:42)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1275)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
> at org.apache.pig.PigServer.storeEx(PigServer.java:931)
> ... 41 more
> Caused by: java.lang.NullPointerException
> at org.apache.pig.builtin.JsonMetadata.storeSchema(JsonMetadata.java:314)
> at org.apache.pig.builtin.JsonStorage.storeSchema(JsonStorage.java:274)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.storeCleanup(PigOutputCommitter.java:141)
> at org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.storeSchemaForLocal(HadoopShims.java:85)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:427)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
> ... 43 more
> 13/08/01 14:09:42 INFO spring.GrailsWebApplicationContext: Closing org.codehaus.groovy.grails.commons.spring.GrailsWebApplicationContext@784ed19b: startup date [Thu Aug 01 14:09:39 PDT 2013]; root of context hierarchy
> 13/08/01 14:09:42 INFO support.DefaultListableBeanFactory: Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@4d838b: defining beans [grailsApplication,pluginManager,messageSource,orgGrailsBeansConstraintsEvaluator,grailsProxyHandler,observeLoggerService]; root of factory hierarchy
> If I save to PigStorage, it works fine.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)