You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "RivenSun (Jira)" <ji...@apache.org> on 2022/04/03 08:32:00 UTC
[jira] [Comment Edited] (KAFKA-13793) Add validators for serialization and deserialization related configuration
[ https://issues.apache.org/jira/browse/KAFKA-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516472#comment-17516472 ]
RivenSun edited comment on KAFKA-13793 at 4/3/22 8:31 AM:
----------------------------------------------------------
I combed through all the configs, and the following configs are missing the validator.
KafkaProducer&KafkaConsumer&KafkaStreams&Connector side:
{code:java}
security.protocol
sasl.mechanism{code}
KafkaServer side:
{code:java}
security.inter.broker.protocol
sasl.mechanism.controller.protocol
sasl.mechanism.inter.broker.protocol {code}
For these parameters above, I think it can be fixed together with the serialization/deserialization configuration.
[~guozhang] [~showuon] what do you think?
In particular, for the parameter {*}compression.type{*}, there is already a validator at the topic level, but missing validators at the producer and KafkaServer levels.
Before creating this jira, I submitted a PR (11985) to fix the `compression.type` issue.
Thanks.
was (Author: rivensun):
I combed through all the configs, and the following configs are missing the validator.
KafkaProducer&KafkaConsumer&KafkaStreams&Connector side:
{code:java}
security.protocol
sasl.mechanism{code}
KafkaServer side:
{code:java}
security.inter.broker.protocol
sasl.mechanism.controller.protocol
sasl.mechanism.inter.broker.protocol {code}
For these parameters above, I think it can be fixed together with the serialization/deserialization configuration.
[~guozhang] [~showuon] what do you think?
In particular, for the parameter {*}compression.type{*}, there is already a validator at the topic level, but missing validators at the producer and KafkaServer levels.
Before creating this jira, I submitted a PR (11985) to fix the `compression.type` issue.
Thanks.
> Add validators for serialization and deserialization related configuration
> --------------------------------------------------------------------------
>
> Key: KAFKA-13793
> URL: https://issues.apache.org/jira/browse/KAFKA-13793
> Project: Kafka
> Issue Type: Improvement
> Components: clients, config
> Reporter: RivenSun
> Assignee: RivenSun
> Priority: Major
>
> These configurations of producer and consumer have the same problem.
> {code:java}
> key.serializer, value.serializer, key.deserializer, value.deserializer{code}
>
> Take the `key.serializer` configuration as an example:
> {code:java}
> Map<String,Object> props = new HashMap<>();
> props.put("key.serializer", null);{code}
> It is expected that this abnormal configuration can be verified during the startup process of kafkaProducer, but the actual startup result:
> {code:java}
> Exception in thread "main" org.apache.kafka.common.KafkaException: Failed to construct kafka producer
> at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:440)
> at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:291)
> at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:274)
> at us.zoom.mq.server.adapter.kafka.ProducerTest.main(ProducerTest.java:139)
> Caused by: java.lang.NullPointerException
> at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:368)
> ... 3 more {code}
> There was a line of code that threw a null pointer, causing KafkaProducer initialization to fail.
> I think we should be able to find this bad configuration during the validation of all the configuration i.e. execute the *ConfigDef.parseValue(ConfigKey key, Object value, boolean isSet) method* and throw a *ConfigException* instead of NullPointerException.
> Solution:
> Add *NonNullValidator* to these configurations.
> For example, when ProducerConfig defines `key.serializer` configuration, add Validator:
> {code:java}
> .define(KEY_SERIALIZER_CLASS_CONFIG,
> Type.CLASS,
> ConfigDef.NO_DEFAULT_VALUE,
> new ConfigDef.NonNullValidator(),
> Importance.HIGH,
> KEY_SERIALIZER_CLASS_DOC) {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)