Latency requirements for Kafka connect workers

[ashish sood <as...@gmail.com>]

Hi Team,

As per my understanding, the distributed Kafka connect workers communicate
with each other over the rest interface. Is there any specific latency
requirements i.e. there should be a maximum latency of 'x' ms to allow
effective functioning of the Kafka connect distributed worker cluster.

Regards
Ashish

Re: Latency requirements for Kafka connect workers

[Malcolm McFarland <mm...@cavulus.com>]

Hey Ashish,

If I understand it correctly, the workers are actually using connect groups
via Kafka to coordinate (I believe via the configured
status.storage.topic and the Kafka consumer group.id). See here:
https://docs.confluent.io/5.5.0/connect/userguide.html#connect-userguide-distributed-config

Adding/removing connectors _is_ done via REST though:
https://docs.confluent.io/5.5.0/connect/references/restapi.html#connect-userguide-rest

Cheers,
Malcolm McFarland
Cavulus


This correspondence is from HealthPlanCRM, LLC, d/b/a Cavulus. Any
unauthorized or improper disclosure, copying, distribution, or use of the
contents of this message is prohibited. The information contained in this
message is intended only for the personal and confidential use of the
recipient(s) named above. If you have received this message in error,
please notify the sender immediately and delete the original message.

Malcolm McFarland
Cavulus


This correspondence is from HealthPlanCRM, LLC, d/b/a Cavulus. Any
unauthorized or improper disclosure, copying, distribution, or use of the
contents of this message is prohibited. The information contained in this
message is intended only for the personal and confidential use of the
recipient(s) named above. If you have received this message in error,
please notify the sender immediately and delete the original message.


On Fri, Jan 15, 2021 at 12:03 AM ashish sood <as...@gmail.com> wrote:

> Hi Team,
>
> As per my understanding, the distributed Kafka connect workers communicate
> with each other over the rest interface. Is there any specific latency
> requirements i.e. there should be a maximum latency of 'x' ms to allow
> effective functioning of the Kafka connect distributed worker cluster.
>
> Regards
> Ashish
>