You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by wanglb <wa...@qq.com> on 2017/10/20 05:30:59 UTC
回复： build cube:Exception from container-launch.
yarn里找不到更详细的日志，不过我参考KYLIN-629 ，随便在kylin_job_conf.xml、kylin_hive_conf.xml、kylin_job_conf_inmem.xml添加了参数
<property>
<name>mapreduce.application.classpath</name>
<value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*, $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
<description>mr classpath</description>
</property>
现在已经执行成功了，谢谢！
------------------ 原始邮件 ------------------
发件人: "Shuangyin Ge"<go...@gmail.com>;
发送时间: 2017年10月20日(星期五) 中午1:10
收件人: "user"<us...@kylin.apache.org>;
主题: Re: build cube:Exception from container-launch.
Hi,
Please check the container log for container_1507516583223_7366_02_000001 on YARN, which should give you more information.
2017-10-20 12:58 GMT+08:00 wanglb <wa...@qq.com>:
BUILD CUBE当执行到第3步时失败，谁能帮忙看看是什么原因，错误详情如下，
#3 Step Name: Extract Fact Table Distinct Columns
User: bdcbas1234
Name: Kylin_Fact_Distinct_Columns_w_cub_source_Step
Application Type: MAPREDUCE
Application Tags:
State: FAILED
FinalStatus: FAILED
Started: 星期五 十月 20 13:07:41 +0800 2017
Elapsed: 8sec
Tracking URL: History
Diagnostics:
Application application_1507516583223_7366 failed 2 times due to AM Container for appattempt_1507516583223_7366_000002 exited with exitCode: 1 due to: Exception from container-launch.
Container id: container_1507516583223_7366_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:290)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Shell output: main : command provided 1
main : user is bdcbas1234
main : requested yarn user is bdcbas1234
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
Re: build cube:Exception from container-launch.
Posted by Li Yang <li...@apache.org>.
Usually user don't have to override "mapreduce.application.classpath" like
you have done. Kylin should automatically set classpath right. You have
work around the issue, but I still recommend to find out the root cause.
2017-10-23 16:52 GMT+08:00 wanglb <wa...@qq.com>:
> 在xml里添加参数不能完全解决问题，
> 最近针对BULID CUBE第三步的报错信息，做出的尝试：
> 在kylin_job_conf.xml、kylin_hive_conf.xml、kylin_job_conf_inmem.xml
> 添加参数：mapreduce.application.classpath
> 然后resume上面报错的BULID CUBE
> BULID CUBE 第三步及后续步骤，可执行成功
> 但是新BULID一个CUBE任务，在第一步失败，报错信息同上面的第三步
>
> 也就是说，在三个xml文件里不添加参数mapreduce.application.classpath时，BULID CUBE在第3步报错
> 添加参数mapreduce.application.classpath时，BULID CUBE在第1步报错
>
> 这个问题很奇怪，这两个地方用这个参数有什么不同吗
> 是我哪里设置的参数有问题？还是bug？
>
> 请求帮助
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "wanglb";<wa...@qq.com>;
> *发送时间:* 2017年10月20日(星期五) 中午1:30
> *收件人:* "user"<us...@kylin.apache.org>;
> *主题:* 回复： build cube:Exception from container-launch.
>
> yarn里找不到更详细的日志，不过
> 我参考KYLIN-629 <https://issues.apache.org/jira/browse/KYLIN-629>
> ，随便在kylin_job_conf.xml、kylin_hive_conf.xml、kylin_job_conf_inmem.xml添加了参数
> <property>
> <name>mapreduce.application.classpath</name>
> <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,
> $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
> <description>mr classpath</description>
> </property>
>
> 现在已经执行成功了，谢谢！
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Shuangyin Ge"<go...@gmail.com>;
> *发送时间:* 2017年10月20日(星期五) 中午1:10
> *收件人:* "user"<us...@kylin.apache.org>;
> *主题:* Re: build cube:Exception from container-launch.
>
> Hi,
>
> Please check the container log for container_1507516583223_7366_02_000001
> on YARN, which should give you more information.
>
> 2017-10-20 12:58 GMT+08:00 wanglb <wa...@qq.com>:
>
>> BUILD CUBE当执行到第3步时失败，谁能帮忙看看是什么原因，错误详情如下，
>> #3 Step Name: Extract Fact Table Distinct Columns
>>
>> User: bdcbas1234
>> Name: Kylin_Fact_Distinct_Columns_w_cub_source_Step
>> Application Type: MAPREDUCE
>> Application Tags:
>> State: FAILED
>> FinalStatus: FAILED
>> Started: 星期五 十月 20 13:07:41 +0800 2017
>> Elapsed: 8sec
>> Tracking URL: History
>> Diagnostics:
>> Application application_1507516583223_7366 failed 2 times due to AM
>> Container for appattempt_1507516583223_7366_000002 exited with exitCode:
>> 1 due to: Exception from container-launch.
>> Container id: container_1507516583223_7366_02_000001
>> Exit code: 1
>> Stack trace: ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Sh
>> ell.java:702)
>> at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExec
>> utor.launchContainer(LinuxContainerExecutor.java:290)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.l
>> auncher.ContainerLaunch.call(ContainerLaunch.java:299)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.l
>> auncher.ContainerLaunch.call(ContainerLaunch.java:81)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1145)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Shell output: main : command provided 1
>> main : user is bdcbas1234
>> main : requested yarn user is bdcbas1234
>> Container exited with a non-zero exit code 1
>> .Failing this attempt.. Failing the application.
>>
>>
>
回复： build cube:Exception from container-launch.
Posted by wanglb <wa...@qq.com>.
在xml里添加参数不能完全解决问题，
最近针对BULID CUBE第三步的报错信息，做出的尝试：
在kylin_job_conf.xml、kylin_hive_conf.xml、kylin_job_conf_inmem.xml
添加参数：mapreduce.application.classpath
然后resume上面报错的BULID CUBE
BULID CUBE 第三步及后续步骤，可执行成功
但是新BULID一个CUBE任务，在第一步失败，报错信息同上面的第三步
也就是说，在三个xml文件里不添加参数mapreduce.application.classpath时，BULID CUBE在第3步报错
添加参数mapreduce.application.classpath时，BULID CUBE在第1步报错
这个问题很奇怪，这两个地方用这个参数有什么不同吗
是我哪里设置的参数有问题？还是bug？
请求帮助
------------------ 原始邮件 ------------------
发件人: "wanglb";<wa...@qq.com>;
发送时间: 2017年10月20日(星期五) 中午1:30
收件人: "user"<us...@kylin.apache.org>;
主题: 回复： build cube:Exception from container-launch.
我参考KYLIN-629 ，随便在kylin_job_conf.xml、kylin_hive_conf.xml、kylin_job_conf_inmem.xml添加了参数
<property>
<name>mapreduce.application.classpath</name>
<value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*, $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
<description>mr classpath</description>
</property>
现在已经执行成功了，谢谢！
------------------ 原始邮件 ------------------
发件人: "Shuangyin Ge"<go...@gmail.com>;
发送时间: 2017年10月20日(星期五) 中午1:10
收件人: "user"<us...@kylin.apache.org>;
主题: Re: build cube:Exception from container-launch.
Hi,
Please check the container log for container_1507516583223_7366_02_000001 on YARN, which should give you more information.
2017-10-20 12:58 GMT+08:00 wanglb <wa...@qq.com>:
BUILD CUBE当执行到第3步时失败，谁能帮忙看看是什么原因，错误详情如下，
#3 Step Name: Extract Fact Table Distinct Columns
User: bdcbas1234
Name: Kylin_Fact_Distinct_Columns_w_cub_source_Step
Application Type: MAPREDUCE
Application Tags:
State: FAILED
FinalStatus: FAILED
Started: 星期五 十月 20 13:07:41 +0800 2017
Elapsed: 8sec
Tracking URL: History
Diagnostics:
Application application_1507516583223_7366 failed 2 times due to AM Container for appattempt_1507516583223_7366_000002 exited with exitCode: 1 due to: Exception from container-launch.
Container id: container_1507516583223_7366_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:290)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Shell output: main : command provided 1
main : user is bdcbas1234
main : requested yarn user is bdcbas1234
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.