Re: [DISCUSS] KIP-477: Add PATCH method for connector config in Connect REST API

[Chris Egerton <ch...@confluent.io.INVALID>]

Hi all,

Know it's been a while for this KIP but in my personal experience the value
of a PATCH method in the REST API has actually increased over time and I'd
love to have this option for quick, stop-the-bleeding remediation efforts.

One thought that comes to mind is that sometimes it may be useful to
explicitly remove properties from a connector configuration. We might
permit this by allowing users to specify null (the JSON literal, not a
string containing the characters "null") as the value for to-be-removed
properties.

I'd love to see this change if you're still interested in driving it, Ivan.
Hopefully we can give it the attention it deserves in the upcoming months!

Cheers,

Chris

On Fri, Jun 28, 2019 at 4:56 AM Ivan Yurchenko <iv...@gmail.com>
wrote:

> Thank you for your feedback Ryanne!
> These are all surely valid concerns and PATCH isn't really necessary or
> suitable for normal production configuration management. However, there are
> cases where quick patching of the configuration is useful, such as hot
> fixes of production or in development.
>
> Overall, the change itself is really tiny and if the cost-benefit balance
> is positive, I'd still like to drive it further.
>
> Ivan
>
> On Wed, 26 Jun 2019 at 17:45, Ryanne Dolan <ry...@gmail.com> wrote:
>
> > Ivan, I looked at adding PATCH a while ago as well. I decided not to
> pursue
> > the idea for a few reasons:
> >
> > 1) PATCH is still racy. For example, if you want to add a topic to the
> > "topics" property, you still need to read, modify, and write the existing
> > value. To handle this, you'd need to support atomic sub-document
> > operations, which I don't see happening.
> >
> > 2) A common pattern is to store your configurations in git or something,
> > and then apply them via PUT. Throw in some triggers or jenkins etc, and
> you
> > have a more robust solution than PATCH provides.
> >
> > 3) For properties that change a lot, it's possible to use an out-of-band
> > data source, e.g. Kafka or Zookeeper, and then have your Connector
> > subscribe to changes. I've done something like this to enable dynamic
> > reconfiguration of Connectors from command-line tools and dashboards
> > without involving the Connect REST API at all. Moreover, I've done so in
> an
> > atomic, non-racy way.
> >
> > So I don't think PATCH is strictly necessary nor sufficient for atomic
> > partial updates. That said, it doesn't hurt and I'm happy to support the
> > KIP.
> >
> > Ryanne
> >
> > On Tue, Jun 25, 2019 at 12:15 PM Ivan Yurchenko <
> ivan0yurchenko@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Since Kafka 2.3 has just been release and more people may have time to
> > look
> > > at this now, I'd like to bump this discussion.
> > > Thanks.
> > >
> > > Ivan
> > >
> > >
> > > On Thu, 13 Jun 2019 at 17:20, Ivan Yurchenko <ivan0yurchenko@gmail.com
> >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I'd like to start the discussion of KIP-477: Add PATCH method for
> > > > connector config in Connect REST API.
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-477%3A+Add+PATCH+method+for+connector+config+in+Connect+REST+API
> > > >
> > > > There is also a draft PR: https://github.com/apache/kafka/pull/6934.
> > > >
> > > > Thank you.
> > > >
> > > > Ivan
> > > >
> > >
> >
>