[jira] [Updated] (KAFKA-3106) After PUT a connector config from REST API, GET a connector config will fail

["Ewen Cheslack-Postava (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ewen Cheslack-Postava updated KAFKA-3106:
-----------------------------------------
    Priority: Minor  (was: Major)

There are a few other ways we could handle this besides blocking on this:

1. Accept that actually getting the connector "committed" and visible across the cluster is relatively expensive and return 202 ACCEPTED instead of 201 CREATED. This indicates we've accepted the request but it may not have fully been processed. It's a bit weird, however, since we *will* have written the connector and so we are guaranteed it will eventually propagate.
2. Change the way connector lookup works so it can work with the latest snapshot of data available, even if its not the one currently active as the group's consistent snapshot. This is acknowledging that we may have new config info available, but not yet have "committed" to it in the herder since that requires the rebalance and consensus among workers.

I'm a bit wary of making the change to block until the rebalance occurs as that can take a long time. And I think we can get it in the DistributedHerder's snapshot of config state but not have finished rebalancing into an active state, in which case the connector will still effectively be in limbo anyway (not all nodes may be in sync, so one could potentially have old config state). Given that we'd normally expect it to be ok to put all the workers behind a load balancer since we handle routing internally, anything that doesn't behave ok by making a second request to a different node seems problematic.

> After  PUT a connector config from REST API, GET a connector config will fail
> -----------------------------------------------------------------------------
>
>                 Key: KAFKA-3106
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3106
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>            Reporter: jin xing
>            Assignee: jin xing
>            Priority: Minor
>
> If there is already a connector in Connect, and we PUT a connector config by REST API, the assignment.offset of DistributedHerder will below the configStat.offset, thus GET connector config though REST API will fail because of failed to pass "checkConfigSynced";
> The failed message is "Cannot get config data because config is not in sync and this is not the leader";
> There need to be a rebalance process for  PUT to update the assignment.offset;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)