Re: deltastreamer group.id Noeffectaftersetting

[Vinoth Chandar <vi...@apache.org>]

DeltaStreamer actually just uses the same mechanism as Spark Streaming to
manage offsets. So wondering if you see the same behavior with a plain
spark streaming job. ?

It manages the offset checkpoints manually by itself within the hoodie
commit metadata, to do exactly once ingestion of data..

On Wed, Mar 18, 2020 at 3:07 AM 965147871@qq.com <96...@qq.com> wrote:

>
> hello, all When using deltastreamer to use kafka data, I want to specify
> group.id, but the problem encountered is that after specifying it, I
> cannot find it on the kafka side. I found that there are no groups under my
> theme. why is it like this? I also manually set enable.auto.commit = true
> at the same time, but it didn't seem to work. In kafkaUtils.scala
> fixKafkaParams this method,
>    kafkaParams.put (ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false:
> java.lang.Boolean),
> which is forced to be rewritten as false. I think it is one of the reasons
> why the group cannot be found, but when it is not automatically submitted,
> it is usually chosen to be submitted manually. , I did not feel this
> phenomenon.
>
>
> Please help answer
> thanks
> liujinhui
>
>
> 965147871@qq.com
>