You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Charles Givre <cg...@gmail.com> on 2018/05/29 02:56:38 UTC
UDF Issue
Hello there,
I am trying to write a UDF for Apache Drill which wraps a h2o MOJO but I'm running into a problem. When I copy `h2o-genmodel.jar` to the Drill path where you put jars, I get the following errors when I attempt to execute a query, regardless of whether the query has anything to do with h2o:
```
2018-05-28 22:48:17,563 ERROR [24f3408d-9cd2-f5ee-6bf6-30274b289811:frag:0:0] record.AbstractSingleRecordBatch: Failure during query
org.apache.drill.exec.exception.SchemaChangeException: Failure while attempting to load generated class
at org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.getGenerated2Copier(RemovingRecordBatch.java:232)
at org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.setupNewSchema(RemovingRecordBatch.java:70)
at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:82)
at org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:93)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:105)
at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:79)
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:95)
at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:234)
at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:227)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:227)
at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.drill.exec.exception.ClassTransformationException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalStateException: Problem found with implementationClasses
at org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:197)
at org.apache.drill.exec.compile.CodeCompiler.createInstance(CodeCompiler.java:163)
at org.apache.drill.exec.ops.BaseFragmentContext.getImplementationClass(BaseFragmentContext.java:60)
at org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.getGenerated2Copier(RemovingRecordBatch.java:224)
... 22 more
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalStateException: Problem found with implementationClasses
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)
at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
at org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:186)
... 25 more
Caused by: java.lang.IllegalStateException: Problem found with implementationClasses
at org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:252)
at org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:229)
at org.apache.drill.exec.compile.CodeCompiler$CodeGenCompiler.compile(CodeCompiler.java:79)
at org.apache.drill.exec.compile.CodeCompiler.makeClass(CodeCompiler.java:229)
at org.apache.drill.exec.compile.CodeCompiler.access$300(CodeCompiler.java:41)
at org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:212)
at org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:209)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
... 29 more
2018-05-28 22:48:17,564 ERROR [24f3408d-9cd2-f5ee-6bf6-30274b289811:frag:0:0] fragment.FragmentExecutor: SYSTEM ERROR: IllegalStateException: Problem found with implementationClasses
Fragment 0:0
[Error Id: 81d76aa7-51bb-4a7d-88fc-465d8de5734f on charles-mbp.fios-router.home:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IllegalStateException: Problem found with implementationClasses
Fragment 0:0
[Error Id: 81d76aa7-51bb-4a7d-88fc-465d8de5734f on charles-mbp.fios-router.home:31010]
at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:586)
at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:298)
at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:267)
at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.drill.exec.exception.SchemaChangeException: Failure while attempting to load generated class
at org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.getGenerated2Copier(RemovingRecordBatch.java:232)
at org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.setupNewSchema(RemovingRecordBatch.java:70)
at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:82)
at org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:93)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:105)
at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:79)
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:95)
at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:234)
at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:227)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:227)
... 4 more
Caused by: org.apache.drill.exec.exception.ClassTransformationException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalStateException: Problem found with implementationClasses
at org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:197)
at org.apache.drill.exec.compile.CodeCompiler.createInstance(CodeCompiler.java:163)
at org.apache.drill.exec.ops.BaseFragmentContext.getImplementationClass(BaseFragmentContext.java:60)
at org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.getGenerated2Copier(RemovingRecordBatch.java:224)
... 22 more
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalStateException: Problem found with implementationClasses
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)
at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
at org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:186)
... 25 more
Caused by: java.lang.IllegalStateException: Problem found with implementationClasses
at org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:252)
at org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:229)
at org.apache.drill.exec.compile.CodeCompiler$CodeGenCompiler.compile(CodeCompiler.java:79)
at org.apache.drill.exec.compile.CodeCompiler.makeClass(CodeCompiler.java:229)
at org.apache.drill.exec.compile.CodeCompiler.access$300(CodeCompiler.java:41)
at org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:212)
at org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:209)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
... 29 more
Error: SYSTEM ERROR: IllegalStateException: Problem found with implementationClasses
Fragment 0:0
[Error Id: 81d76aa7-51bb-4a7d-88fc-465d8de5734f on charles-mbp.fios-router.home:31010] (state=,code=0)
java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Problem found with implementationClasses
Fragment 0:0
[Error Id: 81d76aa7-51bb-4a7d-88fc-465d8de5734f on charles-mbp.fios-router.home:31010]
at org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:489)
at org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(DrillCursor.java:561)
at org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(DrillResultSetImpl.java:1895)
at org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(DrillResultSetImpl.java:61)
at org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:473)
at org.apache.drill.jdbc.impl.DrillMetaImpl.prepareAndExecute(DrillMetaImpl.java:1100)
at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:477)
at org.apache.drill.jdbc.impl.DrillConnectionImpl.prepareAndExecuteInternal(DrillConnectionImpl.java:191)
at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:110)
at org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:121)
at org.apache.drill.jdbc.impl.DrillStatementImpl.execute(DrillStatementImpl.java:101)
at sqlline.Commands.execute(Commands.java:841)
at sqlline.Commands.sql(Commands.java:751)
at sqlline.SqlLine.dispatch(SqlLine.java:746)
at sqlline.SqlLine.begin(SqlLine.java:621)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: IllegalStateException: Problem found with implementationClasses
Fragment 0:0
[Error Id: 81d76aa7-51bb-4a7d-88fc-465d8de5734f on charles-mbp.fios-router.home:31010]
at org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123)
at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:468)
at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:102)
at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:274)
at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:244)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
at java.lang.Thread.run(Thread.java:745)
```
It seems that some library is conflicting with h2o-genmodel in Drill. Any suggestions?