You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by eef hhj <zz...@gmail.com> on 2021/03/18 11:50:46 UTC

Parameter to config read frequency in Kafka SQL connector

Hi team,

We are in a situatoin that we want to reduce the read frequency of Kafka
SQL connector. I did some investigation on the properties of Kafka client,
while it seems it does not have such options. Athough I found the batch
size config('properties.max.partition.fetch.bytes') among the config
options.

https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html

Any suggestions on this?

Re: Parameter to config read frequency in Kafka SQL connector

Posted by eef hhj <zz...@gmail.com>.

Hi Dawid,

Thank you for the reply, I do not find such property in Kafka connector's
config, while it's easy to customize with the passed in properties
parameter.

*-- Best wishes*
*Kai*

On Fri, Mar 19, 2021 at 5:21 PM Dawid Wysakowicz <dw...@apache.org>
wrote:

> Hi,
>
> Unfortunately I have no experience with this. You can pass any Kafka
> client parameters through the properties.* option[1] and see if the
> setting works for you.
>
> Best,
>
> Dawid
>
> [1]
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connectors/kafka.html#properties
>
> On 18/03/2021 12:50, eef hhj wrote:
> > Hi team,
> >
> > We are in a situatoin that we want to reduce the read frequency of
> > Kafka SQL connector. I did some investigation on the properties of
> > Kafka client, while it seems it does not have such options. Athough I
> > found the batch size config('properties.max.partition.fetch.bytes')
> > among the config options.
> >
> >
> https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html
> > <
> https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html
> >
> >
> > Any suggestions on this?
> >
>
>

-- 
*Best regards,*
*- Kai*

Re: Parameter to config read frequency in Kafka SQL connector

Posted by Dawid Wysakowicz <dw...@apache.org>.

Hi,

Unfortunately I have no experience with this. You can pass any Kafka
client parameters through the properties.* option[1] and see if the
setting works for you.

Best,

Dawid

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connectors/kafka.html#properties

On 18/03/2021 12:50, eef hhj wrote:
> Hi team,
>
> We are in a situatoin that we want to reduce the read frequency of
> Kafka SQL connector. I did some investigation on the properties of
> Kafka client, while it seems it does not have such options. Athough I
> found the batch size config('properties.max.partition.fetch.bytes')
> among the config options.
>
> https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html
> <https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html>
>
> Any suggestions on this?
>