[jira] [Commented] (KYLIN-2142) error while building the Streaming Cube.

["Shaofeng SHI (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KYLIN-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15624123#comment-15624123 ] 

Shaofeng SHI commented on KYLIN-2142:
-------------------------------------

Kylin 1.5.x streaming is compiled with Kafka 0.8 API, which doesn't work with Kafka 0.9;

Now the 1.5.x streaming is deprecated, and the new streaming (compiled with Kafka 0.10 API, is back compitable with 0.9) will be released soon, which is under beta testing now, please download Kylin 1.6.0-SNAPSHOT and check this blog (https://kylin.apache.org/blog/2016/10/18/new-nrt-streaming/) 

> error while building the Streaming Cube.
> ----------------------------------------
>
>                 Key: KYLIN-2142
>                 URL: https://issues.apache.org/jira/browse/KYLIN-2142
>             Project: Kylin
>          Issue Type: Bug
>          Components: streaming
>    Affects Versions: v1.5.4.1
>         Environment: HDP 2.4, Kafka 0.9, kylin : 1.5.4.1
>            Reporter: Bhanuprakash
>
> hi all,
> error while building the Streaming Cube.
> Environmental Details 
> *****HDP 2.4 sandbox
> *****Kylin : 1.5.4.2
> *****kafka : 0.9
> i have followed the URL for building the streaming cube in Kylin 
> 1) i have successfully created the kafka topic.
> 2) i have successfully integrated the kafka topic with kylin streaming data.
> 3) i am able to view the kafka broker sending the messages per 2 seconds and can see the data .
> 4) i have created the Streaming DB and took Order_time as partitioner.
> 5) i have executed the build command for streaming cube.
> 6) i have already google for this issue and i found below URL nd i tried copying the kafka client jars to Kylin lib folder and started to build the new streaming cube but still it has failed with the same issue.
> ( https://www.mail-archive.com/issues@kylin.apache.org/msg04708.html )
> when i see the log i see below error log
> 2016-10-29 16:55:30,894 INFO  [main TimedJsonStreamParser:97]: TimedJsonStreamParser with formatTs false tsColName order_time
> Exception in thread "main" java.lang.NoSuchMethodError: kafka.cluster.Broker.<init>(ILjava/lang/String;I)V
>         at org.apache.kylin.source.kafka.config.KafkaClusterConfig$1.apply(KafkaClusterConfig.java:70)
>         at org.apache.kylin.source.kafka.config.KafkaClusterConfig$1.apply(KafkaClusterConfig.java:66)
>         at com.google.common.collect.Lists$TransformingRandomAccessList.get(Lists.java:575)
>         at java.util.AbstractList$Itr.next(AbstractList.java:358)
>         at org.apache.kylin.source.kafka.util.KafkaRequester.getKafkaTopicMeta(KafkaRequester.java:100)
>         at org.apache.kylin.source.kafka.KafkaStreamingInput.getBatchWithTimeWindow(KafkaStreamingInput.java:100)
>         at org.apache.kylin.engine.streaming.OneOffStreamingBuilder$1.run(OneOffStreamingBuilder.java:75)
>         at org.apache.kylin.engine.streaming.cli.StreamingCLI.startOneOffCubeStreaming(StreamingCLI.java:123)
>         at org.apache.kylin.engine.streaming.cli.StreamingCLI.main(StreamingCLI.java:97)
> 2016-10-29 16:55:31,125 INFO  [Thread-0 ConnectionManager$HConnectionImplementation:1683]: Closing zookeeper sessionid=0x1580fec18c0002f
> 2016-10-29 16:55:31,181 INFO  [Thread-0 ZooKeeper:684]: Session: 0x1580fec18c0002f closed
> 2016-10-29 16:55:31,185 INFO  [main-EventThread ClientCnxn:524]: EventThread shut down



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)