Upgrade to flink 1.14.2 and using new Data Source and Sink API

[Daniel Peled <da...@gmail.com>]

Hi everyone,

We have upgraded our flink version from 1.13.5 to 1.14.2
We are using the new kafkaSource and KafkaSink (instead of FlinkKafkaConsumer
and FlinkKafkaProducer)
After the upgrade, we *keep seeing* these log messages in TM logs
Is this OK ?
Are we doing something wrong ?

BR,
Danny

[image: image.png]

Re: Upgrade to flink 1.14.2 and using new Data Source and Sink API

[Mika Naylor <ma...@autophagy.io>]

Hi Daniel,

These logs look pretty normal. As for the -1 epochs, depending on which version
you're using, I think that this might apply:

"For a producer which is being initialized for the first time, the producerId and
epoch will be set to -1. For a producer which is reinitializing, a positive valued
producerId and epoch must be provided."

(From: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89068820#:~:text=For%20a%20producer%20which%20is,and%20epoch%20must%20be%20provided)

I think that these logs are being created when a producer is being
initialized, likely because the logging mode is INFO, which is quite
verbose.

Kind regards,
Mika

On 13.01.2022 13:40, Daniel Peled wrote:
>Hi everyone,
>
>We have upgraded our flink version from 1.13.5 to 1.14.2
>We are using the new kafkaSource and KafkaSink (instead of FlinkKafkaConsumer
>and FlinkKafkaProducer)
>After the upgrade, we *keep seeing* these log messages in TM logs
>Is this OK ?
>Are we doing something wrong ?
>
>BR,
>Danny
>
>[image: image.png]



Mika Naylor
https://autophagy.io