"version" in kafka-reassign-partitions's JSON input file is not updated by KIP-113

[Attila Sasvari <as...@cloudera.com.INVALID>]

Hi there,

KIP-113 added a new, optional filed to the input JSON file of
kafka-reassign-partitions:

{
  "version" : int,
  "partitions" : [
    {
      "topic" : str,
      "partition" : int,
      "replicas" : [int],
      "log_dirs" : [str]    <-- NEW. A log directory can be either "any",
or a valid absolute path that begins with '/'. This is an optional filed.
It is treated as an array of "any" if this field is not explicitly
specified in the json file.
    },
    ...
  ]
}
{code}

KIP-113 says:
This KIP is a pure addition. So there is no backward compatibility concern.

Is it intentional that "version" remained 1?

Regards,
Attila

Re: "version" in kafka-reassign-partitions's JSON input file is not updated by KIP-113

[Attila Sasvári <as...@apache.org>]

Any ideas why version has not been changed of the reassignment json file?

If you generate it with  kafka-reassign-partitions --zookeeper $(hostname
-f):2181 --topics-to-move-json-file reassign.json --broker-list "26,27,28"
--generate, you can see something like this:

Current partition replica assignment
{"version":1,"partitions":[{"topic":"testTopic","partition":4,"replicas":[26,27,28],"log_dirs":["any","any","any"]},{"topic":"testTopic","partition":2,"replicas":[27,26,28],"log_dirs":["any","any","any"]},{"topic":"testTopic","partition":1,"replicas":[26,28,27],"log_dirs":["any","any","any"]},{"topic":"testTopic","partition":0,"replicas":[28,27,26],"log_dirs":["any","any","any"]},{"topic":"testTopic","partition":3,"replicas":[28,26,27],"log_dirs":["any","any","any"]}]}

Proposed partition reassignment configuration:
{"version":1,"partitions":[{"topic":"testTopic","partition":0,"replicas":[26,27,28],"log_dirs":["any","any","any"]},{"topic":"testTopic","partition":3,"replicas":[26,28,27],"log_dirs":["any","any","any"]},{"topic":"testTopic","partition":2,"replicas":[28,26,27],"log_dirs":["any","any","any"]},{"topic":"testTopic","partition":4,"replicas":[27,26,28],"log_dirs":["any","any","any"]},{"topic":"testTopic","partition":1,"replicas":[27,28,26],"log_dirs":["any","any","any"]}]}

Using an earlier Kafka version, it looked like this:
{"version":1,"partitions":[{"topic":"testTopic","partition":4,"replicas":[26,27,28]},{"topic":"testTopic","partition":2,"replicas":[27,26,28]},{"topic":"testTopic",
...
{"topic":"testTopic","partition":3,"replicas":[28,26,27]}]}

What do you think?


On Tue, Nov 20, 2018 at 4:48 PM Attila Sasvari
<as...@cloudera.com.invalid> wrote:
>
> Hi there,
>
> KIP-113 added a new, optional filed to the input JSON file of
> kafka-reassign-partitions:
>
> {
>   "version" : int,
>   "partitions" : [
>     {
>       "topic" : str,
>       "partition" : int,
>       "replicas" : [int],
>       "log_dirs" : [str]    <-- NEW. A log directory can be either "any",
> or a valid absolute path that begins with '/'. This is an optional filed.
> It is treated as an array of "any" if this field is not explicitly
> specified in the json file.
>     },
>     ...
>   ]
> }
> {code}
>
> KIP-113 says:
> This KIP is a pure addition. So there is no backward compatibility
concern.
>
> Is it intentional that "version" remained 1?
>
> Regards,
> Attila