You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "XiaoXiang Yu (JIRA)" <ji...@apache.org> on 2019/04/23 06:30:00 UTC
[jira] [Commented] (KYLIN-3970) NRT Streaming cannot work at HDP3.0
[ https://issues.apache.org/jira/browse/KYLIN-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16823735#comment-16823735 ]
XiaoXiang Yu commented on KYLIN-3970:
-------------------------------------
h3. *First problem:* *Kafka jar conflict*
org.apache.kylin.engine.mr.exception.MapReduceException: no counters for job job_1555653800318_0004Job Diagnostics:Task failed task_1555653800318_0004_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
Failure task Diagnostics:
Error: org.apache.kafka.clients.consumer.ConsumerConfig.configNames()Ljava/util/Set;
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
h3. *Second problem: Hive null value*
org.apache.kylin.engine.mr.exception.MapReduceException: no counters for job job_1555653800318_0007Job Diagnostics:Task failed task_1555653800318_0007_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
Failure task Diagnostics:
Error: java.lang.IllegalArgumentException: there is no valid date pattern for:\N
at org.apache.kylin.common.util.DateFormat.stringToMillis(DateFormat.java:124)
at org.apache.kylin.metadata.datatype.DataTypeOrder$DateTimeOrder.toComparable(DataTypeOrder.java:154)
at org.apache.kylin.metadata.datatype.DataTypeOrder$DateTimeOrder.toComparable(DataTypeOrder.java:148)
at org.apache.kylin.metadata.datatype.DataTypeOrder.compare(DataTypeOrder.java:95)
at org.apache.kylin.metadata.datatype.DataTypeOrder.max(DataTypeOrder.java:90)
at org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper.doMap(FactDistinctColumnsMapper.java:194)
at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
> NRT Streaming cannot work at HDP3.0
> -----------------------------------
>
> Key: KYLIN-3970
> URL: https://issues.apache.org/jira/browse/KYLIN-3970
> Project: Kylin
> Issue Type: Bug
> Components: NRT Streaming
> Affects Versions: v2.6.1
> Reporter: XiaoXiang Yu
> Assignee: XiaoXiang Yu
> Priority: Major
> Labels: HDP, Hadoop3
> Fix For: v3.0.0, v2.6.2
>
> Attachments: image-2019-04-23-14-25-51-066.png
>
>
> We found that when using [apache-kylin-2.6.1-bin-hadoop3.tar.gz|https://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.6.1/apache-kylin-2.6.1-bin-hadoop3.tar.gz], NRT streaming cannot work in serval step for jar conflict problem.
>
> !image-2019-04-23-14-25-51-066.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)