Re: 回复: 安装问题

[JiaTao Tao <ta...@gmail.com>]

Hi
Maybe you can change the execution engine from TEZ to MR and give it a try.
Add this

    <property>
        <name>hive.execution.engine</name>
        <value>mr</value>
    </property>

to kylin_hive_conf.xml or direct change it in hive-site.xml.


-- 


Regards!

Aron Tao

mingwen.pan@analyticservice.net <mi...@analyticservice.net>
于2019年4月19日周五 下午12:43写道：

> HI KYLIN,
>     在hdp 平台，Kylin build cubes时候出错
> INFO : Total jobs = 1 INFO : Launching Job 1 out of 1 INFO : Starting task
> [Stage-1:MAPRED] in serial mode INFO : Subscribed to counters: [] for
> queryId: hive_20190419142859_2332b1f4-f1d8-4c3f-87ab-79800abd3ab4 INFO :
> Tez session hasn't been created yet. Opening session ERROR : Failed to
> execute tez graph. org.apache.tez.dag.api.SessionNotRunning: TezSession has
> already shutdown. Application application_1555566483117_0009 failed 2 times
> due to AM Container for appattempt_1555566483117_0009_000002 exited with
> exitCode: 1 Failing this attempt.Diagnostics: [2019-04-19
> 02:29:06.129]Exception from container-launch. Container id:
> container_e51_1555566483117_0009_02_000001 Exit code: 1 [2019-04-19
> 02:29:06.133]Container exited with a non-zero exit code 1. Error file:
> prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr
> : [2019-04-19 02:29:06.133]Container exited with a non-zero exit code 1.
> Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096
> bytes of stderr : For more detailed output, check the application tracking
> page: http://masternode1:8088/cluster/app/application_1555566483117_0009
> Then click on links to logs of each attempt. . Failing the application. at
> org.apache.tez.client.TezClient.waitTillReady(TezClient.java:1013)
> ~[tez-api-0.9.1.3.1.0.0-78.jar:0.9.1.3.1.0.0-78] at
> org.apache.tez.client.TezClient.waitTillReady(TezClient.java:982)
> ~[tez-api-0.9.1.3.1.0.0-78.jar:0.9.1.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:536)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:451)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.openInternal(TezSessionPoolSession.java:124)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:373)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:372)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:199)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:210)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2711)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2382)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2054)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1752)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1746)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)
> ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226)
> ~[hive-service-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
> ~[hive-service-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:324)
> ~[hive-service-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
> at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112] at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> ~[hadoop-common-3.1.1.3.1.0.0-78.jar:?] at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:342)
> ~[hive-service-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[?:1.8.0_112] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_112] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[?:1.8.0_112] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_112] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[?:1.8.0_112] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[?:1.8.0_112] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] ERROR
> : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask INFO : Completed executing
> command(queryId=hive_20190419142859_2332b1f4-f1d8-4c3f-87ab-79800abd3ab4);
> Time taken: 6.901 seconds Error: Error while processing statement: FAILED:
> Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=1) Closing: 0:
> jdbc:hive2://datanode1:2181,datanode2:2181,datanode3:2181/default;password=root;serviceDiscoveryMode=zooKeeper;user=root;zooKeeperNamespace=hiveserver2
> The command is: hive -e "USE default; DROP TABLE IF EXISTS
> kylin_intermediate_cube_test_4eccf098_dc26_230f_c64a_46872178ef1d; CREATE
> EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_cube_test_4eccf098_dc26_230f_c64a_46872178ef1d (
> REGION_R_NAME string ) STORED AS SEQUENCEFILE LOCATION
> 'hdfs://masternode1:8020/kylin/kylin_metadata/kylin-eb4a4a90-9503-23a3-9aa9-8234d39ee5f6/kylin_intermediate_cube_test_4eccf098_dc26_230f_c64a_46872178ef1d';
> ALTER TABLE
> kylin_intermediate_cube_test_4eccf098_dc26_230f_c64a_46872178ef1d SET
> TBLPROPERTIES('auto.purge'='true'); INSERT OVERWRITE TABLE
> \`kylin_intermediate_cube_test_4eccf098_dc26_230f_c64a_46872178ef1d\`
> SELECT \`REGION\`.\`R_NAME\` as \`REGION_R_NAME\` FROM
> \`DEFAULT\`.\`REGION\` as \`REGION\` WHERE 1=1; " --hiveconf
> hive.merge.mapredfiles=false --hiveconf hive.auto.convert.join=true
> --hiveconf hive.exec.compress.output=true --hiveconf
> hive.auto.convert.join.noconditionaltask=true --hiveconf
> mapreduce.job.split.metainfo.maxsize=-1 --hiveconf
> hive.merge.mapfiles=false --hiveconf
> hive.auto.convert.join.noconditionaltask.size=100000000 --hiveconf
> hive.stats.autogather=true at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:96)
> at
> org.apache.kylin.source.hive.CreateFlatHiveTableStep.createFlatHiveTable(CreateFlatHiveTableStep.java:62)
> at
> org.apache.kylin.source.hive.CreateFlatHiveTableStep.doWork(CreateFlatHiveTableStep.java:99)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> ------------------------------
> mingwen.pan@analyticservice.net
>
>
> *发件人：* mingwen.pan@analyticservice.net
> *发送时间：* 2019-04-19 06:59
> *收件人：* user <us...@kylin.apache.org>; dev <de...@kylin.apache.org>
> *主题：* 回复: 回复: 安装问题
> HI KYLIN,
>
>     下面是安装问题？帮忙看看？谢谢！
>
> ------------------------------
> mingwen.pan@analyticservice.net
>
>
> *发件人：* mingwen.pan@analyticservice.net
> *发送时间：* 2019-04-19 06:57
> *收件人：* dev <de...@kylin.apache.org>; user <us...@kylin.apache.org>
> *主题：* 回复: 安装问题
>
>
> ------------------------------
> mingwen.pan@analyticservice.net
>
>
> *发件人：* mingwen.pan@analyticservice.net
> *发送时间：* 2019-04-19 06:54
> *收件人：* dev <de...@kylin.apache.org>; user <us...@kylin.apache.org>
> *主题：* 安装问题
> HI KYLIN,
>    再HDP 下装了KYLIN2.6.1,出错:
>
> ------------------------------
> mingwen.pan@analyticservice.net
>
>