[GitHub] [kafka] hachikuji commented on a change in pull request #10005: MINOR: Add ConfigRepository, use in Partition and KafkaApis

[GitBox <gi...@apache.org>]

hachikuji commented on a change in pull request #10005:
URL: https://github.com/apache/kafka/pull/10005#discussion_r569770342



##########
File path: core/src/main/scala/kafka/cluster/Partition.scala
##########
@@ -89,8 +88,7 @@ object Partition extends KafkaMetricsGroup {
 
     val configProvider = new TopicConfigFetcher {
       override def fetch(): Properties = {
-        val adminZkClient = new AdminZkClient(replicaManager.zkClient)
-        adminZkClient.fetchEntityConfig(ConfigType.Topic, topicPartition.topic)
+        replicaManager.configRepository.topicConfigs(topicPartition.topic())

Review comment:
       Hmm.. It seems like `ConfigRepository` can just replace `TopicConfigFetcher`?

##########
File path: core/src/main/scala/kafka/server/metadata/LocalConfigRepository.scala
##########
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server.metadata
+
+import java.util
+import java.util.Properties
+import java.util.concurrent.ConcurrentHashMap
+import java.util.function.BiFunction
+
+import org.apache.kafka.common.config.ConfigResource
+import org.apache.kafka.common.config.ConfigResource.Type
+
+import scala.jdk.CollectionConverters._
+
+/**
+ * A ConfigRepository that stores configurations locally.
+ */
+class LocalConfigRepository extends ConfigRepository {
+  val configMap = new ConcurrentHashMap[ConfigResource, util.HashMap[String, String]]
+
+  def setTopicConfig(topic: String, key: String, value: String): Unit = {

Review comment:
       Useful to add some documentation about the contract. For example, it looks like we expect value=null to indicate deletion (and on that note, it might be clearer to have an explicit `removeTopicConfig`).

##########
File path: core/src/main/scala/kafka/server/metadata/ConfigRepository.scala
##########
@@ -0,0 +1,26 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server.metadata
+
+import java.util.Properties
+
+trait ConfigRepository {
+  def topicConfigs(name: String): Properties

Review comment:
       nit: can we be explicit and rename the args to `topicName` and `brokerId`?
   
   Also, it would be useful to document the fact that the returned value is a copy of current configuration and will not reflect future changes.

##########
File path: core/src/main/scala/kafka/server/metadata/LocalConfigRepository.scala
##########
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server.metadata
+
+import java.util
+import java.util.Properties
+import java.util.concurrent.ConcurrentHashMap
+import java.util.function.BiFunction
+
+import org.apache.kafka.common.config.ConfigResource
+import org.apache.kafka.common.config.ConfigResource.Type
+
+import scala.jdk.CollectionConverters._
+
+/**
+ * A ConfigRepository that stores configurations locally.
+ */
+class LocalConfigRepository extends ConfigRepository {
+  val configMap = new ConcurrentHashMap[ConfigResource, util.HashMap[String, String]]
+
+  def setTopicConfig(topic: String, key: String, value: String): Unit = {
+    setConfig(new ConfigResource(Type.TOPIC, topic), key, value)
+  }
+
+  def setBrokerConfig(id: Int, key: String, value: String): Unit = {
+    setConfig(new ConfigResource(Type.BROKER, id.toString()), key, value)
+  }
+
+  def setConfig(configResource: ConfigResource, key: String, value: String): Unit = {
+    configMap.compute(configResource, new BiFunction[ConfigResource, util.HashMap[String, String], util.HashMap[String, String]] {
+      override def apply(resource: ConfigResource,
+                         curConfigs: util.HashMap[String, String]): util.HashMap[String, String] = {
+        if (value == null) {
+          if (curConfigs == null) {
+            null
+          } else {
+            val newConfigs = new util.HashMap[String, String](curConfigs)
+            newConfigs.remove(key)
+            if (newConfigs.isEmpty) {
+              null
+            } else {
+              newConfigs
+            }
+          }
+        } else {
+          if (curConfigs == null) {
+            val newConfigs = new util.HashMap[String, String](1)
+            newConfigs.put(key, value)
+            newConfigs
+          } else {
+            val newConfigs = new util.HashMap[String, String](curConfigs.size() + 1)
+            newConfigs.putAll(curConfigs)
+            newConfigs.put(key, value)
+            newConfigs
+          }
+        }
+      }
+    })
+  }
+
+  def config(configResource: ConfigResource): Properties = {

Review comment:
       nit: can we be consistent on pluralization? I think I tend to see `config` as plural already, so maybe we can use `topicConfig` and `brokerConfig`.
   
   Really we could even pull this up to `ConfigRepository`. Maybe something like this:
   
   ```scala
   trait ConfigRepository {
     def config(configResource: ConfigResource): Properties
     
     def topicConfig(topic: String): Properties = {
       config(new ConfigResource(Type.TOPIC, topic))
     }
   
     override def brokerConfig(id: Int): Properties = {
       config(new ConfigResource(Type.BROKER, id.toString()))
     }
   }
   ```

##########
File path: core/src/main/scala/kafka/server/KafkaApis.scala
##########
@@ -120,6 +120,7 @@ class KafkaApis(val requestChannel: RequestChannel,
   type FetchResponseStats = Map[TopicPartition, RecordConversionStats]
   this.logIdent = "[KafkaApi-%d] ".format(brokerId)
   val adminZkClient = new AdminZkClient(zkClient)
+  val configHelper = new ConfigHelper(metadataCache, config, replicaManager.configRepository)

Review comment:
       nit: it's a little ugly to reach into `ReplicaManager` in order to get the dependency we need. This also makes the mocking more difficult. Can we instead add `ConfigRepository` as a formal parameter?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org