Re: [DISCUSS] KAFKA-4680: min.insync.replica can be set > replication factor

[Alexandre Dupriez <al...@gmail.com>]

Thanks Paolo for taking care of this.

I think option (3) would be the closest to iso-functionality. I wonder
how useful it could be to always add the replication factor of the
topic in the error message [1] versus returning a modified one fitted
for this specific case?

[1] https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/cluster/Partition.scala#L971-L975

Le jeu. 27 févr. 2020 à 19:06, Paolo Moriello
<pa...@gmail.com> a écrit :
>
> Hello,
>
> I'd like to take up this Jira ticket
> <https://issues.apache.org/jira/browse/KAFKA-4680>. This is an old ticket,
> marked as a Kafka bug.
>
> Before moving forward, I'd like to open a discussion on what would be the
> best approach to take on when doing the validation, as well as discuss
> about possible edge cases we should consider.
>
> Basically, at the moment, it is possible to specify min.insync.replicas >
> replication factor. When this happens, it is not possible to produce on a
> topic when acks=all as client callback returns NOT_ENOUGH_REPLICAS, and the
> broker logs error messages on each request. As suggested in the Jira, the
> validation should happen much earlier in the process, eg. at topic
> creation/configuration setup.
>
> Regarding the approach to use on validating the configuration; do we want,
> for instance, to:
> 1. print a WARN about the mismatch in the configuration
> 2. make the request FAIL
> 3. or print a more specific message on produce
>
> Options 1 and 2 anticipate the validation on topic creation / configuration
> change. These require to validate the configuration in more than one place:
> at topic creation, at configuration setup/update (both for
> min.insync.replicas and the default.replication.factor), at partition
> reassignment (when reducing replication factor). Don't know about
> consequences
> Option 3 is simpler; it does not anticipate the validation, but at least
> improves the visibility over the issue on the client side.
>
> I'd be in favor of a softer approach, which might include both printing a
> warning on topic creation/configuration-update and eventually a more
> specific message when producing on the topic. On the other end, this does
> not solve the problem, as we would allow anyway the mismatch in the
> configuration. Option 2 would solve the problem with an harder validation
> (eg blocking topic creation or configuration setup), but this requires to
> validate any possible edge case (eg. how do we prevent a change in
> min.insync.replicas if we have already created topic with lower replication
> factor?).
>
> Let me know what's your opinion on this, and if there is any other scenario
> we should consider for the validation (for instance, what's the impact on
> internal topics?).
>
> Thanks,
> Paolo