You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by kylin_liu <al...@163.com> on 2017/09/07 03:29:04 UTC
kylin2.1+spark 2.1.1
我架设了 KYLIN 最新版本的，sample 测试是可以正常使用spark。
我自己也创建了cube 也跑成功了一个，但是后面的建立的cube 就失败了，我从spark 的history 查看日志，错误如下
java.lang.IllegalAccessError: tried to access class
org.apache.kylin.dimension.BooleanDimEnc$BooleanSerializer from class
org.apache.kylin.engine.spark.KylinKryoRegistrator
java.lang.IllegalAccessError: tried to access class
org.apache.kylin.dimension.BooleanDimEnc$BooleanSerializer from class
org.apache.kylin.engine.spark.KylinKryoRegistrator
at
org.apache.kylin.engine.spark.KylinKryoRegistrator.kylinClassByReflection1(KylinKryoRegistrator.java:173)
at
org.apache.kylin.engine.spark.KylinKryoRegistrator.registerClasses(KylinKryoRegistrator.java:85)
at
org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:134)
at
org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:134)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at
org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:134)
at
org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:289)
at
org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:274)
at
org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:184)
at
org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:287)
at
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:225)
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1269)
at
org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:206)
at
org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:66)
at
org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:66)
at
org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)
at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:86)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
请问一下这个错误是 kylin 的配置问题呢 还是 spark 的问题，现在很迷惑。
而且我用spark 执行成功的 cube ,spark 只能用 yarn+client ,用 yarn+clucter 失败
请指教，谢谢！！
--
Sent from: http://apache-kylin.74782.x6.nabble.com/
Re: kylin2.1+spark 2.1.1
Posted by Billy Liu <bi...@apache.org>.
As the javadocs for IllegalAccessError state,
Normally, this error is caught by the compiler; this error can only occur
at run time if the definition of a class has incompatibly changed.
2017-09-07 11:29 GMT+08:00 kylin_liu <al...@163.com>:
> 我架设了 KYLIN 最新版本的，sample 测试是可以正常使用spark。
> 我自己也创建了cube 也跑成功了一个，但是后面的建立的cube 就失败了，我从spark 的history 查看日志，错误如下
> java.lang.IllegalAccessError: tried to access class
> org.apache.kylin.dimension.BooleanDimEnc$BooleanSerializer from class
> org.apache.kylin.engine.spark.KylinKryoRegistrator
>
> java.lang.IllegalAccessError: tried to access class
> org.apache.kylin.dimension.BooleanDimEnc$BooleanSerializer from class
> org.apache.kylin.engine.spark.KylinKryoRegistrator
> at
> org.apache.kylin.engine.spark.KylinKryoRegistrator.
> kylinClassByReflection1(KylinKryoRegistrator.java:173)
> at
> org.apache.kylin.engine.spark.KylinKryoRegistrator.registerClasses(
> KylinKryoRegistrator.java:85)
> at
> org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(
> KryoSerializer.scala:134)
> at
> org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(
> KryoSerializer.scala:134)
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.
> scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(
> ArrayOps.scala:186)
> at
> org.apache.spark.serializer.KryoSerializer.newKryo(
> KryoSerializer.scala:134)
> at
> org.apache.spark.serializer.KryoSerializerInstance.
> borrowKryo(KryoSerializer.scala:289)
> at
> org.apache.spark.serializer.KryoSerializerInstance.<init>(
> KryoSerializer.scala:274)
> at
> org.apache.spark.serializer.KryoSerializer.newInstance(
> KryoSerializer.scala:184)
> at
> org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(
> TorrentBroadcast.scala:287)
> at
> org.apache.spark.broadcast.TorrentBroadcast$$anonfun$
> readBroadcastBlock$1.apply(TorrentBroadcast.scala:225)
> at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1269)
> at
> org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(
> TorrentBroadcast.scala:206)
> at
> org.apache.spark.broadcast.TorrentBroadcast._value$
> lzycompute(TorrentBroadcast.scala:66)
> at
> org.apache.spark.broadcast.TorrentBroadcast._value(
> TorrentBroadcast.scala:66)
> at
> org.apache.spark.broadcast.TorrentBroadcast.getValue(
> TorrentBroadcast.scala:96)
> at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:86)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(
> Executor.scala:322)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> 请问一下这个错误是 kylin 的配置问题呢 还是 spark 的问题，现在很迷惑。
>
> 而且我用spark 执行成功的 cube ,spark 只能用 yarn+client ,用 yarn+clucter 失败
> 请指教，谢谢！！
>
> --
> Sent from: http://apache-kylin.74782.x6.nabble.com/
>