You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2022/04/22 08:00:59 UTC
[GitHub] [skywalking] wyt opened a new pull request, #8929: Fix the kafka consumer configuration override bug.
wyt opened a new pull request, #8929:
URL: https://github.com/apache/skywalking/pull/8929
<!--
⚠️ Please make sure to read this template first, pull requests that don't accord with this template
maybe closed without notice.
Texts surrounded by `<` and `>` are meant to be replaced by you, e.g. <framework name>, <issue number>.
Put an `x` in the `[ ]` to mark the item as CHECKED. `[x]`
-->
<!-- ==== 🐛 Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist 👇 ====
### Fix <bug description or the bug issue number or bug issue link>
- [] Add a unit test to verify that the fix works.
- [ ] Explain briefly why the bug exists and how to fix it.
==== 🐛 Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist 👆 ==== -->
<!-- ==== 📈 Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist 👇 ====
### Improve the performance of <class or module or ...>
- [ ] Add a benchmark for the improvement, refer to [the existing ones](https://github.com/apache/skywalking/blob/master/apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java)
- [ ] The benchmark result.
```text
<Paste the benchmark results here>
```
- [ ] Links/URLs to the theory proof or discussion articles/blogs. <links/URLs here>
==== 📈 Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist 👆 ==== -->
<!-- ==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👇 ====
### <Feature description>
- [ ] If this is non-trivial feature, paste the links/URLs to the design doc.
- [ ] Update the documentation to include this new feature.
- [ ] Tests(including UT, IT, E2E) are added to verify the new feature.
- [ ] If it's UI related, attach the screenshots below.
==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👆 ==== -->
- [ ] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #<issue number>.
- [x] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/changelog/docs/en/changes/changes.md).
When I try the configuration below to set different consumer group for oap server, it doesn't work.
Still the default group in the program.
```yaml
kafka-fetcher:
selector: ${SW_KAFKA_FETCHER:default}
default:
bootstrapServers: ${SW_KAFKA_FETCHER_SERVERS:192.168.101.1:9092}
......
kafkaConsumerConfig:
enable.auto.commit: true
group.id: test-group-mysql
```
The reason is that there is an order problem when setting in the code:
```java
properties = new Properties();
properties.putAll(config.getKafkaConsumerConfig());
properties.setProperty(ConsumerConfig.GROUP_ID_CONFIG, config.getGroupId());
properties.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, config.getBootstrapServers());
```
I don't know if this is by design or a mistake in itself.
Please confirm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] dmsolr commented on pull request #8929: Fix the kafka consumer configuration override bug.
Posted by GitBox <gi...@apache.org>.
dmsolr commented on PR #8929:
URL: https://github.com/apache/skywalking/pull/8929#issuecomment-1106535040
In fact, it totally is not a bug. To simplify configuration, we prefect users to configure as follows:
```yaml
kafka-fetcher:
selector: ${SW_KAFKA_FETCHER:default}
default:
bootstrapServers: ${SW_KAFKA_FETCHER_SERVERS:192.168.101.1:9092}
groupId: ${SW_KAFKA_FETCHER_GROUP_ID: skywalking}
```
Not to configure `kafkaConsumerConfig`, this is for advance configure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #8929: Fix the kafka consumer configuration override bug.
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #8929:
URL: https://github.com/apache/skywalking/pull/8929#issuecomment-1106542595
@dmsolr I checked the codes, it seems we really provide this `kafkaConsumerConfig`. Once we did, I think the changed behavior is at least more sensible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] dmsolr commented on pull request #8929: Fix the kafka consumer configuration override bug.
Posted by GitBox <gi...@apache.org>.
dmsolr commented on PR #8929:
URL: https://github.com/apache/skywalking/pull/8929#issuecomment-1106554783
make sense.
Yes, users can configure `BOOTSTRAP_SERVERS_CONFIG` and `GROUP_ID_CONFIG ` by KafkaFetchConfig(base) and KafkaConsumerConfig(advance).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng merged pull request #8929: Fix the kafka consumer configuration override bug.
Posted by GitBox <gi...@apache.org>.
wu-sheng merged PR #8929:
URL: https://github.com/apache/skywalking/pull/8929
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #8929: Fix the kafka consumer configuration override bug.
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #8929:
URL: https://github.com/apache/skywalking/pull/8929#issuecomment-1106564124
Yes, once they could be override, make `kafkaConsumerConfig` to override single configuration key seems a better pratice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org