[jira] [Resolved] (KAFKA-20973) Unable to dynamically update non string configs with config providers

["Chia-Ping Tsai (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-20973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chia-Ping Tsai resolved KAFKA-20973.
------------------------------------
    Fix Version/s:     (was: 4.5.0)
       Resolution: Won't Fix

> Unable to dynamically update non string configs with config providers
> ---------------------------------------------------------------------
>
>                 Key: KAFKA-20973
>                 URL: https://issues.apache.org/jira/browse/KAFKA-20973
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Mickael Maison
>            Assignee: Ming-Yen Chung
>            Priority: Major
>         Attachments: DynamicConfigWithConfigProviderTest.java
>
>
> When using config.providers in a dynamic broker config update (via IncrementalAlterConfigs), provider references like ${file:/path:key} work for STRING and PASSWORD typed configs but silently fail for other types
>   (INT, LONG, BOOLEAN, LIST with validators, etc.).
> The root cause is in DynamicBrokerConfig.fromPersistentProps (line 327), which calls removeInvalidConfigs and validateConfigTypes on the raw persisted values before resolveVariableConfigs is called. For non-STRING
>   types, the literal placeholder string ${file:/path:key} fails type parsing (e.g., "Not a number of type INT" for num.io.threads), and the config entry is silently removed from the props.
> There is a unit test, DynamicBrokerReconfigurationTest.testUpdatesUsingConfigProvider which is expected to demonstrate this work but the test is flawed because it only tests custom and per listener configurations which are handle differently. If we try to update a config like num.io.threads, the test fails with:
> {noformat}
> ERROR Dynamic broker config contains invalid values in: Set(num.io.threads), these configs will be ignored (kafka.server.DynamicBrokerConfig:63)
> org.apache.kafka.common.config.ConfigException: Invalid value ${file:num.io.threads:threads} for configuration num.io.threads: Not a number of type INT
> 	at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:789)
> 	at org.apache.kafka.server.config.DynamicConfig$Broker.validate(DynamicConfig.java:72)
> 	at org.apache.kafka.server.config.DynamicBrokerConfig.validateConfigTypes(DynamicBrokerConfig.java:151)
> 	at org.apache.kafka.server.config.DynamicBrokerConfig.validateConfigTypes(DynamicBrokerConfig.java:145)
> 	at kafka.server.DynamicBrokerConfig.removeInvalidConfigs(DynamicBrokerConfig.scala:371)
> 	at kafka.server.DynamicBrokerConfig.fromPersistentProps(DynamicBrokerConfig.scala:327)
> 	at kafka.server.DynamicBrokerConfig.$anonfun$updateBrokerConfig$1(DynamicBrokerConfig.scala:279)
> 	at org.apache.kafka.server.util.LockUtils.inLock(LockUtils.java:96)
> {noformat}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)