You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by "a773807943@gmail.com" <a7...@gmail.com> on 2020/04/21 06:34:15 UTC

cube on spark livy Invalid response 400 with

hi everyone ： 
    I want to use spark with Apache Livy to build cube， but it not work.

    the kylin.log is :

    2020-04-21 14:18:07,158 ERROR [pool-10-thread-3] threadpool.DefaultScheduler:116 : ExecuteException job:acddba5e-ca93-20ec-a4b2-e56aa0dacd38
org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException: java.lang.RuntimeException: livy execute failed.
Invalid response 400 with url http://cunchu03.ysec.yy.com:8999/batches

    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:194)
    at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kylin.job.exception.ExecuteException: java.lang.RuntimeException: livy execute failed.
Invalid response 400 with url http://cunchu03.ysec.yy.com:8999/batches

    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:194)
    at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
    ... 4 more
Caused by: java.lang.RuntimeException: livy execute failed.
Invalid response 400 with url http://cunchu03.ysec.yy.com:8999/batches

    at org.apache.kylin.common.livy.LivyRestExecutor.execute(LivyRestExecutor.java:76)
    at org.apache.kylin.source.hive.MRHiveDictUtil.runLivySqlJob(MRHiveDictUtil.java:144)
    at org.apache.kylin.source.hive.CreateFlatHiveTableByLivyStep.createFlatHiveTable(CreateFlatHiveTableByLivyStep.java:45)
    at org.apache.kylin.source.hive.CreateFlatHiveTableByLivyStep.doWork(CreateFlatHiveTableByLivyStep.java:60)
    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
    ... 6 more


   the kylin.properties  with livy is 

kylin.engine.livy-conf.livy-enabled=true
kylin.engine.livy-conf.livy-url=http://cunchu03.ysec.yy.com:8999
kylin.engine.livy-conf.livy-key.file=hdfs:///kylinData/jar/kylin-job-3.0.1.jar
kylin.engine.livy-conf.livy-arr.jars=hdfs:///kylinData/hbasejar/hbase-client-1.1.2.2.6.5.1050-37.jar,hdfs:///kylinData/hbasejar/hbase-common-1.1.2.2.6.5.1050-37.jar,hdfs:///kylinData/hbasejar/hbase-hadoop-compat-1.1.2.2.6.5.1050-37.jar,hdfs:///kylinData/hbasejar/hbase-hadoop2-compat-1.1.2.2.6.5.1050-37.jar,hdfs:///kylinData/hbasejar/hbase-server-1.1.2.2.6.5.1050-37.jar,hdfs:///kylinData/hbasejar/htrace-core-3.1.0-incubating.jar,hdfs:///kylinData/hbasejar/metrics-core-2.2.0.jar,hdfs:///kylinData/hbasejar/hbase-protocol-1.1.2.2.6.5.0-292.jar


my hadoop is use HDP-2.6.5.1050

                                                                                                                                                                                                                                                                                                                thanks!    



a773807943@gmail.com