You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Jhen-Yung Hsu (Jira)" <ji...@apache.org> on 2024/12/06 08:56:00 UTC
[jira] [Comment Edited] (KAFKA-10043) Some parameters will be overwritten which was configured in consumer.config where running "ConsumerPerformance.scala"
[ https://issues.apache.org/jira/browse/KAFKA-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17903584#comment-17903584 ]
Jhen-Yung Hsu edited comment on KAFKA-10043 at 12/6/24 8:55 AM:
----------------------------------------------------------------
The *priority of loading properties* is chaotic and inconsistent across different files, as [~chia7712] mentioned in [https://github.com/apache/kafka/pull/9790#issuecomment-752856869].
For example:
* In ConsoleConsumer.java/ConsumerPerformance.java [https://github.com/apache/kafka/blob/22f4c4a173368785ce5e65c97043b8bc579a559c/tools/src/main/java/org/apache/kafka/tools/ConsumerPerformance.java#L339-L355], it uses its own way to load properties.
* In ConsoleProducer.java [https://github.com/apache/kafka/blob/22f4c4a173368785ce5e65c97043b8bc579a559c/tools/src/main/java/org/apache/kafka/tools/ConsoleProducer.java#L315-L347], it uses the method defined in [CommandLineUtils.maybeMergeOptions|https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java#L183-L199] to load properties.
Currently, the priority of loading properties heavily depends on implementation details, making it confusing and unclear to users. I think we should creating a *KIP* to define a consistent priority for loading properties. The *preferred priority* should be:
# Command line
# Config file
# Default value
This approach closely aligns with the precedence defined and used in [CommandLineUtils.maybeMergeOptions|https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java#L183-L199]. We could consider using this method consistently across all files.
The following files, which I have noticed so far, need to have their property-loading priority aligned:
* {{ConsoleConsumer.java}} / {{ConsumerPerformance.java}}
* {{ConsoleProducer.java}}
* {{ConsoleShareConsumer.java}} / {{ConsoleShareConsumerOptions.java}}
was (Author: JIRAUSER307316):
The *priority of loading properties* is chaotic and inconsistent across different files, as [~chia7712] mentioned in [https://github.com/apache/kafka/pull/9790#issuecomment-752856869].
For example:
* In ConsoleConsumer.java/ConsumerPerformance.java [https://github.com/apache/kafka/blob/22f4c4a173368785ce5e65c97043b8bc579a559c/tools/src/main/java/org/apache/kafka/tools/ConsumerPerformance.java#L339-L355], it uses its own way to load properties.
* In ConsoleProducer.java [https://github.com/apache/kafka/blob/22f4c4a173368785ce5e65c97043b8bc579a559c/tools/src/main/java/org/apache/kafka/tools/ConsoleProducer.java#L315-L347], it uses the method defined in [CommandLineUtils.maybeMergeOptions|https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java#L183-L199] to load properties.
Currently, the priority of loading properties heavily depends on implementation details, making it confusing and unclear to users. I think we should creating a *KIP* to define a consistent priority for loading properties. The *preferred priority* should be:
# Command line
# Config file
# Default value
This approach closely aligns with the precedence defined and used in [CommandLineUtils.maybeMergeOptions|https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java#L183-L199].
The following files, which I have noticed so far, need to have their property-loading priority aligned:
* {{ConsoleConsumer.java}} / {{ConsumerPerformance.java}}
* {{ConsoleProducer.java}}
* {{ConsoleShareConsumer.java}} / {{ConsoleShareConsumerOptions.java}}
> Some parameters will be overwritten which was configured in consumer.config where running "ConsumerPerformance.scala"
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-10043
> URL: https://issues.apache.org/jira/browse/KAFKA-10043
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 2.1.1
> Reporter: tigertan
> Assignee: Jhen-Yung Hsu
> Priority: Minor
> Labels: perfomance
> Fix For: 2.6.0
>
> Attachments: bug-.png
>
>
> When running the script of "kafka-consumer-perf-test.sh", some parameters which was set through consumer.config would be overwritten.Such as "group.id".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)