You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/10/19 15:24:54 UTC
[GitHub] [kafka] cmccabe opened a new pull request #11416: MINOR: Improve createTopics and incrementalAlterConfigs in KRaft
cmccabe opened a new pull request #11416:
URL: https://github.com/apache/kafka/pull/11416
For CreateTopics, fix a bug where if one createTopics in a batch failed, they would all fail with
the same error code. Make the error message for TOPIC_ALREADY_EXISTS consistent with the ZK-based
code by including the topic name.
For IncrementalAlterConfigs, before we allow topic configurations to be set, we should check that
they are valid. (This also applies to newly created topics.) IncrementalAlterConfigs should ignore
non-null payloads for DELETE operations. Previously we would return an error in these cases.
However, this is not compatible with the old ZK-based code, which ignores the payload in these
cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cmccabe commented on a change in pull request #11416: KAFKA-13490: Improve createTopics and incrementalAlterConfigs in KRaft
Posted by GitBox <gi...@apache.org>.
cmccabe commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r761494119
##########
File path: metadata/src/test/java/org/apache/kafka/controller/ConfigurationControlManagerTest.java
##########
@@ -72,6 +73,14 @@
define("ghi", ConfigDef.Type.BOOLEAN, true, ConfigDef.Importance.HIGH, "ghi"));
}
+ static class ExampleConfigurationValidator implements ConfigurationValidator {
+ static final ExampleConfigurationValidator INSTANCE = new ExampleConfigurationValidator();
+
+ @Override
+ public void validate(ConfigResource resource, Map<String, String> config) {
+ }
Review comment:
ok, I'll do that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cmccabe commented on a change in pull request #11416: KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode
Posted by GitBox <gi...@apache.org>.
cmccabe commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r761588451
##########
File path: core/src/main/scala/kafka/server/ControllerConfigurationValidator.scala
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import java.util
+import java.util.Properties
+
+import kafka.log.LogConfig
+import org.apache.kafka.common.config.ConfigResource
+import org.apache.kafka.common.config.ConfigResource.Type.{BROKER, TOPIC}
+import org.apache.kafka.controller.ConfigurationValidator
+import org.apache.kafka.common.errors.InvalidRequestException
+
+import scala.collection.mutable
+
+class ControllerConfigurationValidator extends ConfigurationValidator {
+ override def validate(resource: ConfigResource, config: util.Map[String, String]): Unit = {
+ resource.`type`() match {
+ case TOPIC =>
+ val properties = new Properties()
+ val nullTopicConfigs = new mutable.ArrayBuffer[String]()
+ config.entrySet().forEach(e => {
+ if (e.getValue() == null) {
+ nullTopicConfigs += e.getKey()
+ } else {
+ properties.setProperty(e.getKey(), e.getValue())
+ }
+ })
+ if (!nullTopicConfigs.isEmpty) {
+ throw new InvalidRequestException("Null value not supported for topic configs : " +
+ nullTopicConfigs.mkString(","))
+ }
+ LogConfig.validate(properties)
+ case BROKER =>
+ // TODO: add broker configuration validation
Review comment:
filed https://issues.apache.org/jira/browse/KAFKA-13503 for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cmccabe commented on a change in pull request #11416: KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode
Posted by GitBox <gi...@apache.org>.
cmccabe commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r761580433
##########
File path: metadata/src/main/java/org/apache/kafka/controller/ConfigurationControlManager.java
##########
@@ -122,19 +126,13 @@ private void incrementalAlterConfigResource(ConfigResource configResource,
newValue = opValue;
break;
case DELETE:
- if (opValue != null) {
Review comment:
There are existing integration tests that pass a non-null value. This PR doesn't convert those tests over but a future one will.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a change in pull request #11416: MINOR: Improve createTopics and incrementalAlterConfigs in KRaft
Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r734967664
##########
File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java
##########
@@ -472,11 +478,7 @@ private ApiError createTopic(CreatableTopic topic,
if (error.isFailure()) return error;
} else if (topic.replicationFactor() < -1 || topic.replicationFactor() == 0) {
return new ApiError(Errors.INVALID_REPLICATION_FACTOR,
- "Replication factor was set to an invalid non-positive value.");
- } else if (!topic.assignments().isEmpty()) {
- return new ApiError(INVALID_REQUEST,
- "Replication factor was not set to -1 but a manual partition " +
- "assignment was specified.");
+ "Number of partitions must be larger than 0.");
Review comment:
typo: "number of partitions" -> "Replication factor"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] hachikuji commented on a change in pull request #11416: KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode
Posted by GitBox <gi...@apache.org>.
hachikuji commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r761564392
##########
File path: metadata/src/main/java/org/apache/kafka/controller/ConfigurationControlManager.java
##########
@@ -122,19 +126,13 @@ private void incrementalAlterConfigResource(ConfigResource configResource,
newValue = opValue;
break;
case DELETE:
- if (opValue != null) {
Review comment:
Do we have any cases where a value is provided for a DELETE operation or was this just to make existing tests work?
##########
File path: core/src/main/scala/kafka/server/ControllerConfigurationValidator.scala
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import java.util
+import java.util.Properties
+
+import kafka.log.LogConfig
+import org.apache.kafka.common.config.ConfigResource
+import org.apache.kafka.common.config.ConfigResource.Type.{BROKER, TOPIC}
+import org.apache.kafka.controller.ConfigurationValidator
+import org.apache.kafka.common.errors.InvalidRequestException
+
+import scala.collection.mutable
+
+class ControllerConfigurationValidator extends ConfigurationValidator {
+ override def validate(resource: ConfigResource, config: util.Map[String, String]): Unit = {
+ resource.`type`() match {
+ case TOPIC =>
+ val properties = new Properties()
+ val nullTopicConfigs = new mutable.ArrayBuffer[String]()
+ config.entrySet().forEach(e => {
+ if (e.getValue() == null) {
+ nullTopicConfigs += e.getKey()
+ } else {
+ properties.setProperty(e.getKey(), e.getValue())
+ }
+ })
+ if (!nullTopicConfigs.isEmpty) {
+ throw new InvalidRequestException("Null value not supported for topic configs : " +
+ nullTopicConfigs.mkString(","))
+ }
+ LogConfig.validate(properties)
+ case BROKER =>
+ // TODO: add broker configuration validation
Review comment:
Do we have a JIRA for this?
##########
File path: core/src/main/scala/kafka/server/ControllerConfigurationValidator.scala
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import java.util
+import java.util.Properties
+
+import kafka.log.LogConfig
+import org.apache.kafka.common.config.ConfigResource
+import org.apache.kafka.common.config.ConfigResource.Type.{BROKER, TOPIC}
+import org.apache.kafka.controller.ConfigurationValidator
+import org.apache.kafka.common.errors.InvalidRequestException
+
+import scala.collection.mutable
+
+class ControllerConfigurationValidator extends ConfigurationValidator {
+ override def validate(resource: ConfigResource, config: util.Map[String, String]): Unit = {
+ resource.`type`() match {
+ case TOPIC =>
+ val properties = new Properties()
+ val nullTopicConfigs = new mutable.ArrayBuffer[String]()
+ config.entrySet().forEach(e => {
+ if (e.getValue() == null) {
+ nullTopicConfigs += e.getKey()
+ } else {
+ properties.setProperty(e.getKey(), e.getValue())
+ }
+ })
+ if (!nullTopicConfigs.isEmpty) {
+ throw new InvalidRequestException("Null value not supported for topic configs : " +
+ nullTopicConfigs.mkString(","))
+ }
+ LogConfig.validate(properties)
+ case BROKER =>
+ // TODO: add broker configuration validation
+ case _ =>
Review comment:
Another type that we need to support is `BROKER_LOGGER`. Probably worth a separate JIRA.
##########
File path: metadata/src/test/java/org/apache/kafka/controller/ConfigurationControlManagerTest.java
##########
@@ -72,6 +73,7 @@
define("ghi", ConfigDef.Type.BOOLEAN, true, ConfigDef.Importance.HIGH, "ghi"));
}
+ private final static ConfigurationValidator NOOP_VALIDATOR = (__, ___) -> { };
Review comment:
nit: maybe consider moving this so that we can use `ConfigurationValidator.NOOP` or something like that. Then it could also be used in the quorum controller builder then.
##########
File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java
##########
@@ -472,11 +478,7 @@ private ApiError createTopic(CreatableTopic topic,
if (error.isFailure()) return error;
} else if (topic.replicationFactor() < -1 || topic.replicationFactor() == 0) {
return new ApiError(Errors.INVALID_REPLICATION_FACTOR,
- "Replication factor was set to an invalid non-positive value.");
- } else if (!topic.assignments().isEmpty()) {
- return new ApiError(INVALID_REQUEST,
- "Replication factor was not set to -1 but a manual partition " +
- "assignment was specified.");
+ "Replication factor must be larger than 0.");
Review comment:
Must be larger than 0 or -1 to indicate the default replication factor?
##########
File path: core/src/main/scala/kafka/server/ControllerConfigurationValidator.scala
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import java.util
+import java.util.Properties
+
+import kafka.log.LogConfig
+import org.apache.kafka.common.config.ConfigResource
+import org.apache.kafka.common.config.ConfigResource.Type.{BROKER, TOPIC}
+import org.apache.kafka.controller.ConfigurationValidator
+import org.apache.kafka.common.errors.InvalidRequestException
+
+import scala.collection.mutable
+
+class ControllerConfigurationValidator extends ConfigurationValidator {
+ override def validate(resource: ConfigResource, config: util.Map[String, String]): Unit = {
+ resource.`type`() match {
+ case TOPIC =>
+ val properties = new Properties()
+ val nullTopicConfigs = new mutable.ArrayBuffer[String]()
+ config.entrySet().forEach(e => {
+ if (e.getValue() == null) {
+ nullTopicConfigs += e.getKey()
+ } else {
+ properties.setProperty(e.getKey(), e.getValue())
+ }
+ })
+ if (!nullTopicConfigs.isEmpty) {
Review comment:
nit: use `nonEmpty`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cmccabe commented on a change in pull request #11416: KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode
Posted by GitBox <gi...@apache.org>.
cmccabe commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r761580186
##########
File path: core/src/main/scala/kafka/server/ControllerConfigurationValidator.scala
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import java.util
+import java.util.Properties
+
+import kafka.log.LogConfig
+import org.apache.kafka.common.config.ConfigResource
+import org.apache.kafka.common.config.ConfigResource.Type.{BROKER, TOPIC}
+import org.apache.kafka.controller.ConfigurationValidator
+import org.apache.kafka.common.errors.InvalidRequestException
+
+import scala.collection.mutable
+
+class ControllerConfigurationValidator extends ConfigurationValidator {
+ override def validate(resource: ConfigResource, config: util.Map[String, String]): Unit = {
+ resource.`type`() match {
+ case TOPIC =>
+ val properties = new Properties()
+ val nullTopicConfigs = new mutable.ArrayBuffer[String]()
+ config.entrySet().forEach(e => {
+ if (e.getValue() == null) {
+ nullTopicConfigs += e.getKey()
+ } else {
+ properties.setProperty(e.getKey(), e.getValue())
+ }
+ })
+ if (!nullTopicConfigs.isEmpty) {
+ throw new InvalidRequestException("Null value not supported for topic configs : " +
+ nullTopicConfigs.mkString(","))
+ }
+ LogConfig.validate(properties)
+ case BROKER =>
+ // TODO: add broker configuration validation
+ case _ =>
Review comment:
`BROKER_LOGGER` is handled by individual brokers and not persisted anywhere in the metadata.
One thing we should do, though, is allow people to use `BROKER_LOGGER` on controller-only nodes. Filed https://issues.apache.org/jira/browse/KAFKA-13502 for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jsancio commented on a change in pull request #11416: MINOR: Improve createTopics and incrementalAlterConfigs in KRaft
Posted by GitBox <gi...@apache.org>.
jsancio commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r744007754
##########
File path: metadata/src/test/java/org/apache/kafka/controller/ConfigurationControlManagerTest.java
##########
@@ -72,6 +73,14 @@
define("ghi", ConfigDef.Type.BOOLEAN, true, ConfigDef.Importance.HIGH, "ghi"));
}
+ static class ExampleConfigurationValidator implements ConfigurationValidator {
+ static final ExampleConfigurationValidator INSTANCE = new ExampleConfigurationValidator();
+
+ @Override
+ public void validate(ConfigResource resource, Map<String, String> config) {
+ }
Review comment:
Did you plan to implement this? If you not, you can use the trick you are doing in other places. E.g.
```java
static ConfigurationValidator NOOP_VALIDATOR = (_, __) -> {};
```
##########
File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java
##########
@@ -215,6 +216,11 @@ public Builder setAlterConfigPolicy(Optional<AlterConfigPolicy> alterConfigPolic
return this;
}
+ public Builder setConfigurationValidator(ConfigurationValidator configurationValidator) {
+ this.configurationValidator = configurationValidator;
+ return this;
+ }
Review comment:
This method is never called in this PR. Are we missing changes?
##########
File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java
##########
@@ -388,7 +389,12 @@ public void replay(RemoveTopicRecord record) {
Map<String, CreatableTopicResult> successes = new HashMap<>();
for (CreatableTopic topic : request.topics()) {
if (topicErrors.containsKey(topic.name())) continue;
- ApiError error = createTopic(topic, records, successes);
+ ApiError error;
+ try {
+ error = createTopic(topic, records, successes);
Review comment:
I find it strange that `createTopic` can return an `ApiError` or throw an `ApiException`. I think it should do one or the other but not both. What is now throwing that requires a `try ... catch ...`?
##########
File path: metadata/src/main/java/org/apache/kafka/controller/ConfigurationValidator.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kafka.controller;
+
+import org.apache.kafka.common.config.ConfigResource;
+
+import java.util.Map;
+
+
+public interface ConfigurationValidator {
+ /**
+ * Throws an ApiException if a configuration is invalid for the given resource.
+ *
+ * @param resource The configuration resource.
+ * @param config The new configuration.
+ */
+ void validate(ConfigResource resource, Map<String, String> config);
Review comment:
I couldn't find a class in `src/main` that implements this interface. Are we missing files in this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] hachikuji commented on a change in pull request #11416: KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode
Posted by GitBox <gi...@apache.org>.
hachikuji commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r762101999
##########
File path: core/src/main/scala/kafka/server/ControllerConfigurationValidator.scala
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import java.util
+import java.util.Properties
+
+import kafka.log.LogConfig
+import org.apache.kafka.common.config.ConfigResource
+import org.apache.kafka.common.config.ConfigResource.Type.{BROKER, TOPIC}
+import org.apache.kafka.controller.ConfigurationValidator
+import org.apache.kafka.common.errors.InvalidRequestException
+
+import scala.collection.mutable
+
+class ControllerConfigurationValidator extends ConfigurationValidator {
+ override def validate(resource: ConfigResource, config: util.Map[String, String]): Unit = {
+ resource.`type`() match {
+ case TOPIC =>
+ val properties = new Properties()
+ val nullTopicConfigs = new mutable.ArrayBuffer[String]()
+ config.entrySet().forEach(e => {
+ if (e.getValue() == null) {
+ nullTopicConfigs += e.getKey()
+ } else {
+ properties.setProperty(e.getKey(), e.getValue())
+ }
+ })
+ if (!nullTopicConfigs.isEmpty) {
+ throw new InvalidRequestException("Null value not supported for topic configs : " +
+ nullTopicConfigs.mkString(","))
+ }
+ LogConfig.validate(properties)
+ case BROKER =>
+ // TODO: add broker configuration validation
+ case _ =>
Review comment:
Yeah, logging for the controller is what I had in mind.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cmccabe commented on pull request #11416: KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode
Posted by GitBox <gi...@apache.org>.
cmccabe commented on pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#issuecomment-986337146
Committed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cmccabe closed pull request #11416: KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode
Posted by GitBox <gi...@apache.org>.
cmccabe closed pull request #11416:
URL: https://github.com/apache/kafka/pull/11416
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cmccabe commented on a change in pull request #11416: KAFKA-13490: Improve createTopics and incrementalAlterConfigs in KRaft
Posted by GitBox <gi...@apache.org>.
cmccabe commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r761500264
##########
File path: metadata/src/main/java/org/apache/kafka/controller/ConfigurationValidator.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kafka.controller;
+
+import org.apache.kafka.common.config.ConfigResource;
+
+import java.util.Map;
+
+
+public interface ConfigurationValidator {
+ /**
+ * Throws an ApiException if a configuration is invalid for the given resource.
+ *
+ * @param resource The configuration resource.
+ * @param config The new configuration.
+ */
+ void validate(ConfigResource resource, Map<String, String> config);
Review comment:
Yes, there was supposed to be a ControllerConfigurationValidator file. Added.
##########
File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java
##########
@@ -215,6 +216,11 @@ public Builder setAlterConfigPolicy(Optional<AlterConfigPolicy> alterConfigPolic
return this;
}
+ public Builder setConfigurationValidator(ConfigurationValidator configurationValidator) {
+ this.configurationValidator = configurationValidator;
+ return this;
+ }
Review comment:
Yes, there was supposed to be a ControllerConfigurationValidator file. Added.
##########
File path: metadata/src/test/java/org/apache/kafka/controller/ConfigurationControlManagerTest.java
##########
@@ -72,6 +73,14 @@
define("ghi", ConfigDef.Type.BOOLEAN, true, ConfigDef.Importance.HIGH, "ghi"));
}
+ static class ExampleConfigurationValidator implements ConfigurationValidator {
+ static final ExampleConfigurationValidator INSTANCE = new ExampleConfigurationValidator();
+
+ @Override
+ public void validate(ConfigResource resource, Map<String, String> config) {
+ }
Review comment:
good idea, I'll do that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cmccabe commented on a change in pull request #11416: KAFKA-13490: Improve createTopics and incrementalAlterConfigs in KRaft
Posted by GitBox <gi...@apache.org>.
cmccabe commented on a change in pull request #11416:
URL: https://github.com/apache/kafka/pull/11416#discussion_r761495515
##########
File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java
##########
@@ -388,7 +389,12 @@ public void replay(RemoveTopicRecord record) {
Map<String, CreatableTopicResult> successes = new HashMap<>();
for (CreatableTopic topic : request.topics()) {
if (topicErrors.containsKey(topic.name())) continue;
- ApiError error = createTopic(topic, records, successes);
+ ApiError error;
+ try {
+ error = createTopic(topic, records, successes);
Review comment:
The try...catch is needed for cases where an unexpected exception is thrown. I agree that this could be refactored to just be exception-based. But let's do that in a follow-on since this change is already big enough...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org