You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Katte <li...@ccfan.com.cn> on 2019/08/26 06:44:23 UTC
答复: kafka构建 重置offset
谢谢!
我发现通过下面命令可以进行构建了,当我指定"sourceOffsetStart": -1,kylin会从offset:0的位置开始消费;如果指定"sourceOffsetStart": 0,kylin会从上次消费的位置开始读取,我这样理解是对的吧。
curl -X PUT --user ADMIN:KYLIN -H "Content-Type: application/json;charset=utf-8" -d '{ "sourceOffsetStart": -1, "sourceOffsetEnd": 9223372036854775807, "buildType": "BUILD"}' http://localhost:7070/kylin/api/cubes/CUBE_T2_0_4/build2
发送自 Windows 10 版邮件应用
发件人: Xiaoxiang Yu
发送时间: 2019年8月24日 18:56
收件人: user@kylin.apache.org; liu.zhen@ccfan.com.cn
主题: Re: kafka构建 重置offset
Dear Katte:
As far as I can see, I think Kylin has the ability of resume the consumption in the right position(right behind the latest segment's offset), even after you restart your kafka and kylin.
I guess you may need to check the state of your kafka topic, you may use tools provided at $KAFKA_HOME/bin to check current offset of your topic partitions, and make sure you can consume the latest message by $KAFKA_HOME/bin/kafka-console-consumer.sh . If you find you Kafka is in right state and the error is caused by Kylin, please provide more detail like kylin version, exception trace in kylin.log and your metadata for further analysis.
----------------
Best wishes,
Xiaoxiang Yu
发件人: Katte <li...@ccfan.com.cn>
答复: "user@kylin.apache.org" <us...@kylin.apache.org>
日期: 2019年8月23日 星期五 19:31
收件人: "user@kylin.apache.org" <us...@kylin.apache.org>
主题: kafka构建 重置offset
你好!
我通过kafka构建了一个 Cube,构建成功后最后的offset位置是10050.
因为实际需要,我们重启了kylin与kafka。之后我再次向指定的topic发送了3条数据。
之后,我再次build cube时,会提示说当前offset是10050,不能构建。
请问一下应该如何处理呢?谢谢!
因为实际情况,我们有可能不定期 重的启kylin与kafka。
发送自 Windows 10 版邮件应用