You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Divya Gehlot <di...@gmail.com> on 2015/07/07 10:56:35 UTC
error when using javscript udf
Hi,
I have sritten a javscript UDF and as mentioned in pig guide
<http://pig.apache.org/docs/r0.9.2/udf.html#udf-java> .I have declared the
outputschema in my js file
parseString.outputSchema = "dataMap:chararray";
whereas same datatype is availble in my pig script too.
A = LOAD '$input_file' AS(
number3:int,number4:float,dataMap:chararray);
Still I am getting below error
2015-07-07 01:46:59,466 [main] ERROR org.apache.pig.tools.grunt.GruntParser
- ERROR 2997: Unable to recreate exception from backed error: Error:
org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
while executing [POUserFunc (Name:
POUserFunc(org.apache.pig.scripting.js.JsFunction)[chararray] - scope-35
Operator Key: scope-35) children: null at []]:
org.mozilla.javascript.EcmaError: ReferenceError: "dataMap" is not defined.
(/x/home/lrennison/monitoring.js#9)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:366)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:383)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:303)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:279)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:274)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.mozilla.javascript.EcmaError: ReferenceError: "dataMap" is
not defined. (/x/home/lrennison/monitoring.js#9)
at
org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3654)
at
org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3632)
at
org.mozilla.javascript.ScriptRuntime.notFoundError(ScriptRuntime.java:3717)
at
org.mozilla.javascript.ScriptRuntime.nameOrFunction(ScriptRuntime.java:1758)
at org.mozilla.javascript.ScriptRuntime.name
(ScriptRuntime.java:1697)
at
org.mozilla.javascript.gen.c2._c2(/x/home/lrennison/monitoring.js:9)
at
org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
at
org.mozilla.javascript.NativeArray.iterativeMethod(NativeArray.java:1582)
at
org.mozilla.javascript.NativeArray.execIdCall(NativeArray.java:322)
at
org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:129)
at
org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:66)
at
org.mozilla.javascript.gen.c2._c1(/x/home/lrennison/monitoring.js:4)
at
org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
at
org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:398)
at
org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3065)
at
org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
at
org.apache.pig.scripting.js.JsScriptEngine.jsCall(JsScriptEngine.java:170)
at org.apache.pig.scripting.js.JsFunction.exec(JsFunction.java:215)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:323)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextString(POUserFunc.java:423)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:343)
... 13 more
Can somebody point pout the missing area .
Would really appreciate your help .
Re: error when using javscript udf
Posted by Debabrata Pani <an...@gmail.com>.
Hi Divya,
Is it possible that the that result object , does not contain a proper
tuple ?
As per the documentation here :
http://pig.apache.org/docs/r0.9.2/udf.html#js-udfs
complex.outputSchema = "word:chararray,num:long";
function complex(word){
return {word:word, num:word.length};
}
you have to return an object with the same key names as mentioned in the
schema.
In case arr[1] is not True, the "result" object will be empty.
And it is possible that an error like the one we encounter, can be thrown.
Can you verify the output and ascertain whether that is the case or not ?
Regards,
Debabrata
On Mon, Jul 13, 2015 at 4:00 PM, Divya Gehlot <di...@gmail.com>
wrote:
> Hi Debabrata.
> Thank you very responding.
> Below is moy javascript snippet.
>
> Javascript :
>
> parseString.outputSchema = "dataMap:chararray";
> function parseString(str) {
> var result = {};
> str.split('&').forEach(function(splitString) {
> var arr = splitString.split('=');
> arr[1] && (result[arr[0]] = arr[1]);
> //var oldJSONStringify = JSON.stringify;
> //console.log(oldJSONStringify(result));
> return result;
> //return oldJSONStringify(result);
>
> });
>
> }
>
>
>
> On 13 July 2015 at 13:37, Debabrata Pani <an...@gmail.com> wrote:
>
>> A snippet of your parseString javascript UDF may help .
>> Can you share it ?
>> Additionally, you have just mentioned the LOAD statement in your pig
>> script.
>> Can you show how you "register"ed your javascript UDF and also how you
>> used
>> your js UDF ?
>>
>> Regards,
>> Debabrata
>>
>> On Tue, Jul 7, 2015 at 2:26 PM, Divya Gehlot <di...@gmail.com>
>> wrote:
>>
>> > Hi,
>> > I have sritten a javscript UDF and as mentioned in pig guide
>> > <http://pig.apache.org/docs/r0.9.2/udf.html#udf-java> .I have declared
>> the
>> > outputschema in my js file
>> >
>> > parseString.outputSchema = "dataMap:chararray";
>> >
>> > whereas same datatype is availble in my pig script too.
>> > A = LOAD '$input_file' AS(
>> > number3:int,number4:float,dataMap:chararray);
>> >
>> > Still I am getting below error
>> > 2015-07-07 01:46:59,466 [main] ERROR
>> org.apache.pig.tools.grunt.GruntParser
>> > - ERROR 2997: Unable to recreate exception from backed error: Error:
>> > org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
>> > while executing [POUserFunc (Name:
>> > POUserFunc(org.apache.pig.scripting.js.JsFunction)[chararray] - scope-35
>> > Operator Key: scope-35) children: null at []]:
>> > org.mozilla.javascript.EcmaError: ReferenceError: "dataMap" is not
>> defined.
>> > (/x/home/lrennison/monitoring.js#9)
>> > at
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:366)
>> > at
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:383)
>> > at
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:303)
>> > at
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:279)
>> > at
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:274)
>> > at
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
>> > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
>> > at
>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>> > at java.security.AccessController.doPrivileged(Native Method)
>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>> > at
>> >
>> >
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>> > Caused by: org.mozilla.javascript.EcmaError: ReferenceError: "dataMap"
>> is
>> > not defined. (/x/home/lrennison/monitoring.js#9)
>> > at
>> >
>> >
>> org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3654)
>> > at
>> >
>> >
>> org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3632)
>> > at
>> >
>> org.mozilla.javascript.ScriptRuntime.notFoundError(ScriptRuntime.java:3717)
>> > at
>> >
>> >
>> org.mozilla.javascript.ScriptRuntime.nameOrFunction(ScriptRuntime.java:1758)
>> > at org.mozilla.javascript.ScriptRuntime.name
>> > (ScriptRuntime.java:1697)
>> > at
>> > org.mozilla.javascript.gen.c2._c2(/x/home/lrennison/monitoring.js:9)
>> > at
>> > org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
>> > at
>> >
>> org.mozilla.javascript.NativeArray.iterativeMethod(NativeArray.java:1582)
>> > at
>> > org.mozilla.javascript.NativeArray.execIdCall(NativeArray.java:322)
>> > at
>> > org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:129)
>> > at
>> > org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:66)
>> > at
>> > org.mozilla.javascript.gen.c2._c1(/x/home/lrennison/monitoring.js:4)
>> > at
>> > org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
>> > at
>> > org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:398)
>> > at
>> > org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3065)
>> > at
>> > org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
>> > at
>> >
>> org.apache.pig.scripting.js.JsScriptEngine.jsCall(JsScriptEngine.java:170)
>> > at
>> org.apache.pig.scripting.js.JsFunction.exec(JsFunction.java:215)
>> > at
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:323)
>> > at
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextString(POUserFunc.java:423)
>> > at
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:343)
>> > ... 13 more
>> >
>> > Can somebody point pout the missing area .
>> > Would really appreciate your help .
>> >
>>
>
>
Re: error when using javscript udf
Posted by Divya Gehlot <di...@gmail.com>.
Hi Debabrata.
Thank you very responding.
Below is moy javascript snippet.
Javascript :
parseString.outputSchema = "dataMap:chararray";
function parseString(str) {
var result = {};
str.split('&').forEach(function(splitString) {
var arr = splitString.split('=');
arr[1] && (result[arr[0]] = arr[1]);
//var oldJSONStringify = JSON.stringify;
//console.log(oldJSONStringify(result));
return result;
//return oldJSONStringify(result);
});
}
On 13 July 2015 at 13:37, Debabrata Pani <an...@gmail.com> wrote:
> A snippet of your parseString javascript UDF may help .
> Can you share it ?
> Additionally, you have just mentioned the LOAD statement in your pig
> script.
> Can you show how you "register"ed your javascript UDF and also how you used
> your js UDF ?
>
> Regards,
> Debabrata
>
> On Tue, Jul 7, 2015 at 2:26 PM, Divya Gehlot <di...@gmail.com>
> wrote:
>
> > Hi,
> > I have sritten a javscript UDF and as mentioned in pig guide
> > <http://pig.apache.org/docs/r0.9.2/udf.html#udf-java> .I have declared
> the
> > outputschema in my js file
> >
> > parseString.outputSchema = "dataMap:chararray";
> >
> > whereas same datatype is availble in my pig script too.
> > A = LOAD '$input_file' AS(
> > number3:int,number4:float,dataMap:chararray);
> >
> > Still I am getting below error
> > 2015-07-07 01:46:59,466 [main] ERROR
> org.apache.pig.tools.grunt.GruntParser
> > - ERROR 2997: Unable to recreate exception from backed error: Error:
> > org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> > while executing [POUserFunc (Name:
> > POUserFunc(org.apache.pig.scripting.js.JsFunction)[chararray] - scope-35
> > Operator Key: scope-35) children: null at []]:
> > org.mozilla.javascript.EcmaError: ReferenceError: "dataMap" is not
> defined.
> > (/x/home/lrennison/monitoring.js#9)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:366)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:383)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:303)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:279)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:274)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> > at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> > at
> >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: org.mozilla.javascript.EcmaError: ReferenceError: "dataMap" is
> > not defined. (/x/home/lrennison/monitoring.js#9)
> > at
> >
> >
> org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3654)
> > at
> >
> >
> org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3632)
> > at
> >
> org.mozilla.javascript.ScriptRuntime.notFoundError(ScriptRuntime.java:3717)
> > at
> >
> >
> org.mozilla.javascript.ScriptRuntime.nameOrFunction(ScriptRuntime.java:1758)
> > at org.mozilla.javascript.ScriptRuntime.name
> > (ScriptRuntime.java:1697)
> > at
> > org.mozilla.javascript.gen.c2._c2(/x/home/lrennison/monitoring.js:9)
> > at
> > org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
> > at
> > org.mozilla.javascript.NativeArray.iterativeMethod(NativeArray.java:1582)
> > at
> > org.mozilla.javascript.NativeArray.execIdCall(NativeArray.java:322)
> > at
> > org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:129)
> > at
> > org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:66)
> > at
> > org.mozilla.javascript.gen.c2._c1(/x/home/lrennison/monitoring.js:4)
> > at
> > org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
> > at
> > org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:398)
> > at
> > org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3065)
> > at
> > org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
> > at
> >
> org.apache.pig.scripting.js.JsScriptEngine.jsCall(JsScriptEngine.java:170)
> > at
> org.apache.pig.scripting.js.JsFunction.exec(JsFunction.java:215)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:323)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextString(POUserFunc.java:423)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:343)
> > ... 13 more
> >
> > Can somebody point pout the missing area .
> > Would really appreciate your help .
> >
>
Re: error when using javscript udf
Posted by Debabrata Pani <an...@gmail.com>.
A snippet of your parseString javascript UDF may help .
Can you share it ?
Additionally, you have just mentioned the LOAD statement in your pig script.
Can you show how you "register"ed your javascript UDF and also how you used
your js UDF ?
Regards,
Debabrata
On Tue, Jul 7, 2015 at 2:26 PM, Divya Gehlot <di...@gmail.com>
wrote:
> Hi,
> I have sritten a javscript UDF and as mentioned in pig guide
> <http://pig.apache.org/docs/r0.9.2/udf.html#udf-java> .I have declared the
> outputschema in my js file
>
> parseString.outputSchema = "dataMap:chararray";
>
> whereas same datatype is availble in my pig script too.
> A = LOAD '$input_file' AS(
> number3:int,number4:float,dataMap:chararray);
>
> Still I am getting below error
> 2015-07-07 01:46:59,466 [main] ERROR org.apache.pig.tools.grunt.GruntParser
> - ERROR 2997: Unable to recreate exception from backed error: Error:
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing [POUserFunc (Name:
> POUserFunc(org.apache.pig.scripting.js.JsFunction)[chararray] - scope-35
> Operator Key: scope-35) children: null at []]:
> org.mozilla.javascript.EcmaError: ReferenceError: "dataMap" is not defined.
> (/x/home/lrennison/monitoring.js#9)
> at
>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:366)
> at
>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:383)
> at
>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:303)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:279)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:274)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.mozilla.javascript.EcmaError: ReferenceError: "dataMap" is
> not defined. (/x/home/lrennison/monitoring.js#9)
> at
>
> org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3654)
> at
>
> org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3632)
> at
> org.mozilla.javascript.ScriptRuntime.notFoundError(ScriptRuntime.java:3717)
> at
>
> org.mozilla.javascript.ScriptRuntime.nameOrFunction(ScriptRuntime.java:1758)
> at org.mozilla.javascript.ScriptRuntime.name
> (ScriptRuntime.java:1697)
> at
> org.mozilla.javascript.gen.c2._c2(/x/home/lrennison/monitoring.js:9)
> at
> org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
> at
> org.mozilla.javascript.NativeArray.iterativeMethod(NativeArray.java:1582)
> at
> org.mozilla.javascript.NativeArray.execIdCall(NativeArray.java:322)
> at
> org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:129)
> at
> org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:66)
> at
> org.mozilla.javascript.gen.c2._c1(/x/home/lrennison/monitoring.js:4)
> at
> org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
> at
> org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:398)
> at
> org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3065)
> at
> org.mozilla.javascript.gen.c2.call(/x/home/lrennison/monitoring.js)
> at
> org.apache.pig.scripting.js.JsScriptEngine.jsCall(JsScriptEngine.java:170)
> at org.apache.pig.scripting.js.JsFunction.exec(JsFunction.java:215)
> at
>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:323)
> at
>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextString(POUserFunc.java:423)
> at
>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:343)
> ... 13 more
>
> Can somebody point pout the missing area .
> Would really appreciate your help .
>