You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by tinawenqiao <31...@qq.com> on 2019/01/03 02:33:51 UTC
Use KafkaAdminClient to get broker dynamic config “leader.replication.throttled.rate” and “follower.replication.throttled.rate” ,but returned null
Hi, everyone: I found KafkaAdminClient.describeConfigs can't return the dynamic broker config value.
First I use kafka-reassign-partitions.sh to reassign partitions, and at the same time I add throttle as follows. And then use describeConfig to get the config value, but I can't get the result as I expected.
Is it a bug or the design is just right like this?
Step 1: Reassign with throttle parameter
bin/kafka-reassign-partitions.sh --zookeeper localhost:2183 --reassignment-json-file reassign.json --execute --throttle 3000 --replica-alter-log-dirs-throttle 2000 --bootstrap-server localhost:19092
Current partition replica assignment
{"version":1,"partitions":[{"topic":"test-topic3","partition":0,"replicas":[115],"log_dirs":["any"]}]}
Save this to use as the --reassignment-json-file option during rollback
Warning: You must run Verify periodically, until the reassignment completes, to ensure the throttle is removed. You can also alter the throttle by rerunning the Execute command passing a new value.
The inter-broker throttle limit was set to 3000 B/s
The replica-alter-dir throttle limit was set to 2000 B/s
Successfully started reassignment of partitions.
Step 2: I can get broker dynamic config on zk.
[zk: localhost:2183(CONNECTED) 47] get /config/brokers/111
{"version":1,"config":{"replica.alter.log.dirs.io.max.bytes.per.second":"2000","leader.replication.throttled.rate":"3000","follower.replication.throttled.rate":"3000"}}
cZxid = 0x1a99
ctime = Fri Dec 21 13:59:20 CST 2018
mZxid = 0x2ebb
mtime = Thu Jan 03 10:06:56 CST 2019
pZxid = 0x1a99
cversion = 0
dataVersion = 19
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 168
numChildren = 0
Step3:Use org.apache.kafka.clients.admin.KafkaAdminClient describe broker 111's config, the results are all null
leader.replication.throttled.rate = null, isReadOnly:true, source:DYNAMIC_BROKER_CONFIG
follower.replication.throttled.rate = null, isReadOnly:true, source:DYNAMIC_BROKER_CONFIG
replica.alter.log.dirs.io.max.bytes.per.second = null, isReadOnly:true, source:DYNAMIC_BROKER_CONFIG
Re: Use KafkaAdminClient to get broker dynamic config “leader.replication.throttled.rate” and “follower.replication.throttled.rate” ,but returned null
Posted by Rajini Sivaram <ra...@gmail.com>.
KafkaAdminClent#describeConfigs currently only returns values of
non-sensitive configs that are defined in `KafkaConfig`. Since `
leader.replication.throttled.rate` is not defined in `KafkaConfig`, it is
handled as a custom config and hence the value is not returned. We could
however change this behaviour for the three configs defined in
kafka.server.DynamicConfig since we know that these are not sensitive
values. Can you open a JIRA?
Thanks,
Rajini
On Thu, Jan 3, 2019 at 2:34 AM tinawenqiao <31...@qq.com> wrote:
> Hi, everyone: I found KafkaAdminClient.describeConfigs can't return the
> dynamic broker config value.
> First I use kafka-reassign-partitions.sh to reassign partitions, and
> at the same time I add throttle as follows. And then use describeConfig to
> get the config value, but I can't get the result as I expected.
> Is it a bug or the design is just right like this?
>
>
> Step 1: Reassign with throttle parameter
> bin/kafka-reassign-partitions.sh --zookeeper localhost:2183
> --reassignment-json-file reassign.json --execute --throttle 3000
> --replica-alter-log-dirs-throttle 2000 --bootstrap-server localhost:19092
> Current partition replica assignment
>
> {"version":1,"partitions":[{"topic":"test-topic3","partition":0,"replicas":[115],"log_dirs":["any"]}]}
> Save this to use as the --reassignment-json-file option during rollback
> Warning: You must run Verify periodically, until the reassignment
> completes, to ensure the throttle is removed. You can also alter the
> throttle by rerunning the Execute command passing a new value.
> The inter-broker throttle limit was set to 3000 B/s
> The replica-alter-dir throttle limit was set to 2000 B/s
> Successfully started reassignment of partitions.
>
>
>
> Step 2: I can get broker dynamic config on zk.
> [zk: localhost:2183(CONNECTED) 47] get /config/brokers/111
>
> {"version":1,"config":{"replica.alter.log.dirs.io.max.bytes.per.second":"2000","leader.replication.throttled.rate":"3000","follower.replication.throttled.rate":"3000"}}
> cZxid = 0x1a99
> ctime = Fri Dec 21 13:59:20 CST 2018
> mZxid = 0x2ebb
> mtime = Thu Jan 03 10:06:56 CST 2019
> pZxid = 0x1a99
> cversion = 0
> dataVersion = 19
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 168
> numChildren = 0
>
>
>
>
> Step3:Use org.apache.kafka.clients.admin.KafkaAdminClient describe broker
> 111's config, the results are all null
> leader.replication.throttled.rate = null, isReadOnly:true,
> source:DYNAMIC_BROKER_CONFIG
>
> follower.replication.throttled.rate = null, isReadOnly:true,
> source:DYNAMIC_BROKER_CONFIG
>
> replica.alter.log.dirs.io.max.bytes.per.second = null, isReadOnly:true,
> source:DYNAMIC_BROKER_CONFIG