You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "kangjunxiang (Jira)" <ji...@apache.org> on 2020/03/18 08:58:00 UTC
[jira] [Commented] (KYLIN-4431) Use the official sample action
"Quick Start with Streaming Sample Cube" build Error
[ https://issues.apache.org/jira/browse/KYLIN-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17061518#comment-17061518 ]
kangjunxiang commented on KYLIN-4431:
-------------------------------------
issue:
3970
https://issues.apache.org/jira/browse/KYLIN-3970?jql=text%20~%20%22partition.assignment.strategy%22
A new error occurred after using "try-2",The following information：
--------------------------------------------------------------------------------------
org.apache.kylin.engine.mr.exception.MapReduceException: Counters: 12
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=3
Rack-local map tasks=1
Total time spent by all maps in occupied slots (ms)=286636
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=71659
Total vcore-milliseconds taken by all map tasks=71659
Total megabyte-milliseconds taken by all map tasks=97742876
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Job Diagnostics:Task failed task_1584424619465_0008_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
Failure task Diagnostics:
Error: java.lang.ClassNotFoundException: org.apache.kafka.clients.consumer.Consumer
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at org.apache.kylin.source.kafka.hadoop.KafkaInputFormat.createRecordReader(KafkaInputFormat.java:107)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:527)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:223)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at org.apache.kyl
> Use the official sample action "Quick Start with Streaming Sample Cube" build Error
> -----------------------------------------------------------------------------------
>
> Key: KYLIN-4431
> URL: https://issues.apache.org/jira/browse/KYLIN-4431
> Project: Kylin
> Issue Type: Bug
> Components: NRT Streaming
> Affects Versions: v3.0.1
> Environment: HDP-3.1.0.0
> Apache kylin 3.0.1
> Kafka 2.0.0
> Reporter: kangjunxiang
> Priority: Major
>
> h2. Use the official sample action "Quick Start with Streaming Sample Cube" {color:#333333}build {color}{color:#333333}Error，The error message is as follows:{color}
>
> ---------------------------error logs---------------------------
> 2020-03-17 15:37:13,389 ERROR [Scheduler 775260112 Job 94b82185-70b0-2a4d-e83e-e5bbdecefa3f-430] execution.AbstractExecutable:181 : error running Executable: CubingJob\{id=94b82185-70b0-2a4d-e83e-e5bbdecefa3f, name=BUILD CUBE - kylin_streaming_cube - 0_2128 - CST 2020-03-17 15:35:57, state=RUNNING}
> 2020-03-17 15:37:13,395 DEBUG [pool-7-thread-1] cachesync.Broadcaster:119 : Servers in the cluster: [localhost:7070]
> 2020-03-17 15:37:13,396 DEBUG [pool-7-thread-1] cachesync.Broadcaster:129 : Announcing new broadcast to all: BroadcastEvent\{entity=execute_output, event=update, cacheKey=94b82185-70b0-2a4d-e83e-e5bbdecefa3f}
> 2020-03-17 15:37:13,401 DEBUG [http-nio-7070-exec-4] cachesync.Broadcaster:267 : Broadcasting UPDATE, execute_output, 94b82185-70b0-2a4d-e83e-e5bbdecefa3f
> 2020-03-17 15:37:13,402 INFO [Scheduler 775260112 Job 94b82185-70b0-2a4d-e83e-e5bbdecefa3f-430] execution.ExecutableManager:478 : job id:94b82185-70b0-2a4d-e83e-e5bbdecefa3f from RUNNING to ERROR
> 2020-03-17 15:37:13,402 DEBUG [Scheduler 775260112 Job 94b82185-70b0-2a4d-e83e-e5bbdecefa3f-430] execution.AbstractExecutable:332 : no need to send email, user list is empty
> 2020-03-17 15:37:13,403 DEBUG [pool-7-thread-1] cachesync.Broadcaster:119 : Servers in the cluster: [localhost:7070]
> 2020-03-17 15:37:13,403 DEBUG [pool-7-thread-1] cachesync.Broadcaster:129 : Announcing new broadcast to all: BroadcastEvent\{entity=execute_output, event=update, cacheKey=94b82185-70b0-2a4d-e83e-e5bbdecefa3f}
> 2020-03-17 15:37:13,403 DEBUG [http-nio-7070-exec-4] cachesync.Broadcaster:301 : Done broadcasting UPDATE, execute_output, 94b82185-70b0-2a4d-e83e-e5bbdecefa3f
> 2020-03-17 15:37:13,404 ERROR [pool-11-thread-10] threadpool.DefaultScheduler:116 : ExecuteException job:94b82185-70b0-2a4d-e83e-e5bbdecefa3f
> org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.engine.mr.exception.MapReduceException: Counters: 12
> Job Counters
> Failed map tasks=4
> Launched map tasks=4
> Other local map tasks=3
> Rack-local map tasks=1
> Total time spent by all maps in occupied slots (ms)=109572
> Total time spent by all reduces in occupied slots (ms)=0
> Total time spent by all map tasks (ms)=27393
> Total vcore-milliseconds taken by all map tasks=27393
> Total megabyte-milliseconds taken by all map tasks=37364052
> Map-Reduce Framework
> CPU time spent (ms)=0
> Physical memory (bytes) snapshot=0
> Virtual memory (bytes) snapshot=0
> Job Diagnostics:Task failed task_1584424619465_0006_m_000000
> Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
>
> Failure task Diagnostics:
> Error: org.apache.kafka.clients.consumer.ConsumerConfig.configNames()Ljava/util/Set;
>
> at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:194)
> at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.engine.mr.exception.MapReduceException: Counters: 12
> Job Counters
> Failed map tasks=4
> Launched map tasks=4
> Other local map tasks=3
> Rack-local map tasks=1
> Total time spent by all maps in occupied slots (ms)=109572
> Total time spent by all reduces in occupied slots (ms)=0
> Total time spent by all map tasks (ms)=27393
> Total vcore-milliseconds taken by all map tasks=27393
> Total megabyte-milliseconds taken by all map tasks=37364052
> Map-Reduce Framework
> CPU time spent (ms)=0
> Physical memory (bytes) snapshot=0
> Virtual memory (bytes) snapshot=0
> Job Diagnostics:Task failed task_1584424619465_0006_m_000000
> Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
>
> ---------------------------error logs---------------------------
>
> *Help, thanks!*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)