You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by bhidevivek <bh...@gmail.com> on 2017/05/10 20:53:03 UTC
Error while using AvroToPojo operator
I am trying to read the hdfs audit logs from avro format and trying to
convert it to POJO using the combination of AvroFileInputOperator and
AvroToPojo but the application launch fails with below error
I have verified that property TUPLE_CLASS for AvroToPojo operator has set to
correct POJO class from project and it has all the required public fields
with getter and setter. Is there anything that I am missing here?
2017-05-10 13:31:28,823 DEBUG com.datatorrent.stram.FSEventRecorder: Adding
event OperatorError to the queue
2017-05-10 13:31:28,823 INFO com.datatorrent.stram.StreamingContainerParent:
child msg: Abandoning deployment due to setup failure.
java.lang.RuntimeException: org.codehaus.commons.compiler.CompileException:
Line 1, Column 0: cannot find symbol
symbol: class PojoUtils$Setter
location: package com.datatorrent.lib.util
(compiler.err.cant.resolve.location)
at com.datatorrent.lib.util.PojoUtils.compileExpression(PojoUtils.java:778)
at com.datatorrent.lib.util.PojoUtils.compileExpression(PojoUtils.java:746)
at com.datatorrent.lib.util.PojoUtils.createSetter(PojoUtils.java:684)
at com.datatorrent.lib.util.PojoUtils.createSetter(PojoUtils.java:442)
at com.datatorrent.lib.util.PojoUtils.createSetter(PojoUtils.java:432)
at
com.tgt.dqs.operator.AvroToPojoCustom.initializeActiveFieldSetters(AvroToPojoCustom.java:400)
at
com.tgt.dqs.operator.AvroToPojoCustom.access$700(AvroToPojoCustom.java:69)
at com.tgt.dqs.operator.AvroToPojoCustom$2.setup(AvroToPojoCustom.java:290)
at com.tgt.dqs.operator.AvroToPojoCustom$2.setup(AvroToPojoCustom.java:272)
at
com.datatorrent.stram.engine.StreamingContainer.setupNode(StreamingContainer.java:1357)
at
com.datatorrent.stram.engine.StreamingContainer.access$100(StreamingContainer.java:129)
at
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1413)
Caused by: org.codehaus.commons.compiler.CompileException: Line 1, Column 0:
cannot find symbol
symbol: class PojoUtils$Setter
location: package com.datatorrent.lib.util
(compiler.err.cant.resolve.location)
at
org.codehaus.commons.compiler.jdk.SimpleCompiler$3.report(SimpleCompiler.java:322)
at
com.sun.tools.javac.api.ClientCodeWrapper$WrappedDiagnosticListener.report(ClientCodeWrapper.java:593)
at com.sun.tools.javac.util.Log.writeDiagnostic(Log.java:616)
at
com.sun.tools.javac.util.Log$DefaultDiagnosticHandler.report(Log.java:600)
at com.sun.tools.javac.util.Log.report(Log.java:562)
at com.sun.tools.javac.comp.Resolve.logResolveError(Resolve.java:3514)
at com.sun.tools.javac.comp.Resolve.accessInternal(Resolve.java:2219)
at com.sun.tools.javac.comp.Resolve.accessBase(Resolve.java:2262)
at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3390)
at com.sun.tools.javac.comp.Attr.visitSelect(Attr.java:3278)
at com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:1897)
at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:576)
at com.sun.tools.javac.comp.Attr.attribType(Attr.java:638)
at com.sun.tools.javac.comp.Attr.attribType(Attr.java:631)
at com.sun.tools.javac.comp.Attr.attribBase(Attr.java:786)
at com.sun.tools.javac.comp.MemberEnter.complete(MemberEnter.java:1072)
at com.sun.tools.javac.code.Symbol.complete(Symbol.java:574)
at com.sun.tools.javac.code.Symbol$ClassSymbol.complete(Symbol.java:1037)
at com.sun.tools.javac.comp.Enter.complete(Enter.java:493)
at com.sun.tools.javac.comp.Enter.main(Enter.java:471)
at com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:982)
at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:857)
at com.sun.tools.javac.main.Main.compile(Main.java:523)
at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
at
org.codehaus.commons.compiler.jdk.SimpleCompiler.cook(SimpleCompiler.java:353)
at
org.codehaus.commons.compiler.jdk.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:197)
at
org.codehaus.commons.compiler.jdk.ScriptEvaluator.cook(ScriptEvaluator.java:426)
at
org.codehaus.commons.compiler.jdk.ScriptEvaluator.cook(ScriptEvaluator.java:323)
at
org.codehaus.commons.compiler.jdk.ScriptEvaluator.cook(ScriptEvaluator.java:273)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:50)
at
org.codehaus.commons.compiler.jdk.ScriptEvaluator.createFastEvaluator(ScriptEvaluator.java:566)
at
org.codehaus.commons.compiler.jdk.ScriptEvaluator.createFastEvaluator(ScriptEvaluator.java:524)
at com.datatorrent.lib.util.PojoUtils.compileExpression(PojoUtils.java:776)
... 11 more
--
View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/Error-while-using-AvroToPojo-operator-tp1593.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.
Re: Error while using AvroToPojo operator
Posted by bhidevivek <bh...@gmail.com>.
Thanks vlad.. It worked
Appreciate your help
Regards
Vivek
--
View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/Error-while-using-AvroToPojo-operator-tp1593p1604.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.
Re: Error while using AvroToPojo operator
Posted by Vlad Rozov <v....@datatorrent.com>.
Do not use janino commons-compiler-jdk, it has a bug that prevents it
from properly working with PojoUtils. Replace it with janino
commons-compiler.
Thank you,
Vlad
On 5/10/17 13:53, bhidevivek wrote:
> I am trying to read the hdfs audit logs from avro format and trying to
> convert it to POJO using the combination of AvroFileInputOperator and
> AvroToPojo but the application launch fails with below error
> I have verified that property TUPLE_CLASS for AvroToPojo operator has set to
> correct POJO class from project and it has all the required public fields
> with getter and setter. Is there anything that I am missing here?
>
> 2017-05-10 13:31:28,823 DEBUG com.datatorrent.stram.FSEventRecorder: Adding
> event OperatorError to the queue
> 2017-05-10 13:31:28,823 INFO com.datatorrent.stram.StreamingContainerParent:
> child msg: Abandoning deployment due to setup failure.
> java.lang.RuntimeException: org.codehaus.commons.compiler.CompileException:
> Line 1, Column 0: cannot find symbol
> symbol: class PojoUtils$Setter
> location: package com.datatorrent.lib.util
> (compiler.err.cant.resolve.location)
> at com.datatorrent.lib.util.PojoUtils.compileExpression(PojoUtils.java:778)
> at com.datatorrent.lib.util.PojoUtils.compileExpression(PojoUtils.java:746)
> at com.datatorrent.lib.util.PojoUtils.createSetter(PojoUtils.java:684)
> at com.datatorrent.lib.util.PojoUtils.createSetter(PojoUtils.java:442)
> at com.datatorrent.lib.util.PojoUtils.createSetter(PojoUtils.java:432)
> at
> com.tgt.dqs.operator.AvroToPojoCustom.initializeActiveFieldSetters(AvroToPojoCustom.java:400)
> at
> com.tgt.dqs.operator.AvroToPojoCustom.access$700(AvroToPojoCustom.java:69)
> at com.tgt.dqs.operator.AvroToPojoCustom$2.setup(AvroToPojoCustom.java:290)
> at com.tgt.dqs.operator.AvroToPojoCustom$2.setup(AvroToPojoCustom.java:272)
> at
> com.datatorrent.stram.engine.StreamingContainer.setupNode(StreamingContainer.java:1357)
> at
> com.datatorrent.stram.engine.StreamingContainer.access$100(StreamingContainer.java:129)
> at
> com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1413)
> Caused by: org.codehaus.commons.compiler.CompileException: Line 1, Column 0:
> cannot find symbol
> symbol: class PojoUtils$Setter
> location: package com.datatorrent.lib.util
> (compiler.err.cant.resolve.location)
> at
> org.codehaus.commons.compiler.jdk.SimpleCompiler$3.report(SimpleCompiler.java:322)
> at
> com.sun.tools.javac.api.ClientCodeWrapper$WrappedDiagnosticListener.report(ClientCodeWrapper.java:593)
> at com.sun.tools.javac.util.Log.writeDiagnostic(Log.java:616)
> at
> com.sun.tools.javac.util.Log$DefaultDiagnosticHandler.report(Log.java:600)
> at com.sun.tools.javac.util.Log.report(Log.java:562)
> at com.sun.tools.javac.comp.Resolve.logResolveError(Resolve.java:3514)
> at com.sun.tools.javac.comp.Resolve.accessInternal(Resolve.java:2219)
> at com.sun.tools.javac.comp.Resolve.accessBase(Resolve.java:2262)
> at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3390)
> at com.sun.tools.javac.comp.Attr.visitSelect(Attr.java:3278)
> at com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:1897)
> at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:576)
> at com.sun.tools.javac.comp.Attr.attribType(Attr.java:638)
> at com.sun.tools.javac.comp.Attr.attribType(Attr.java:631)
> at com.sun.tools.javac.comp.Attr.attribBase(Attr.java:786)
> at com.sun.tools.javac.comp.MemberEnter.complete(MemberEnter.java:1072)
> at com.sun.tools.javac.code.Symbol.complete(Symbol.java:574)
> at com.sun.tools.javac.code.Symbol$ClassSymbol.complete(Symbol.java:1037)
> at com.sun.tools.javac.comp.Enter.complete(Enter.java:493)
> at com.sun.tools.javac.comp.Enter.main(Enter.java:471)
> at com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:982)
> at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:857)
> at com.sun.tools.javac.main.Main.compile(Main.java:523)
> at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
> at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
> at
> org.codehaus.commons.compiler.jdk.SimpleCompiler.cook(SimpleCompiler.java:353)
> at
> org.codehaus.commons.compiler.jdk.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:197)
> at
> org.codehaus.commons.compiler.jdk.ScriptEvaluator.cook(ScriptEvaluator.java:426)
> at
> org.codehaus.commons.compiler.jdk.ScriptEvaluator.cook(ScriptEvaluator.java:323)
> at
> org.codehaus.commons.compiler.jdk.ScriptEvaluator.cook(ScriptEvaluator.java:273)
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:50)
> at
> org.codehaus.commons.compiler.jdk.ScriptEvaluator.createFastEvaluator(ScriptEvaluator.java:566)
> at
> org.codehaus.commons.compiler.jdk.ScriptEvaluator.createFastEvaluator(ScriptEvaluator.java:524)
> at com.datatorrent.lib.util.PojoUtils.compileExpression(PojoUtils.java:776)
> ... 11 more
>
>
>
> --
> View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/Error-while-using-AvroToPojo-operator-tp1593.html
> Sent from the Apache Apex Users list mailing list archive at Nabble.com.