You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "xiangfu0 (via GitHub)" <gi...@apache.org> on 2024/05/02 17:01:12 UTC
[PR] Handle backward compatibility for kafka common package references in KafkaStreamConfig [pinot]
xiangfu0 opened a new pull request, #13056:
URL: https://github.com/apache/pinot/pull/13056
Due to the recent shade plugin change, `org.apache.kafka` package is always relocated to `org.apache.pinot.shaded.org.apache.kafka.common`.
Try to amend any reference in `org.apache.kafka.common` package to `org.apache.pinot.shaded.org.apache.kafka.common` if the referenced class doesn't exist.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Handle backward compatibility for kafka common package references in KafkaStreamConfig [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #13056:
URL: https://github.com/apache/pinot/pull/13056#discussion_r1588126539
##########
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaConfigBackwardCompatibleUtils.java:
##########
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.stream.kafka20;
+
+import java.util.Map;
+import org.apache.pinot.spi.stream.StreamConfig;
+
+
+public class KafkaConfigBackwardCompatibleUtils {
+ private KafkaConfigBackwardCompatibleUtils() {
+ }
+
+ public static final String KAFKA_COMMON_PACKAGE_PREFIX = "org.apache.kafka.common";
+ public static final String PINOT_SHADED_PACKAGE_PREFIX = "org.apache.pinot.shaded.";
+
+ /**
+ * Handle the stream config to replace the Kafka common package with the shaded version if needed.
+ */
+ public static void handleStreamConfig(StreamConfig streamConfig) {
+ Map<String, String> streamConfigMap = streamConfig.getStreamConfigsMap();
+ for (Map.Entry<String, String> entry : streamConfigMap.entrySet()) {
+ String[] valueParts = entry.getValue().split(" ");
Review Comment:
What is the example value here? Why do we need to split it?
(minor) Use `StringUtils.split(entry.getValue(), ' ')`
##########
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaConfigBackwardCompatibleUtils.java:
##########
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.stream.kafka20;
+
+import java.util.Map;
+import org.apache.pinot.spi.stream.StreamConfig;
+
+
+public class KafkaConfigBackwardCompatibleUtils {
+ private KafkaConfigBackwardCompatibleUtils() {
+ }
+
+ public static final String KAFKA_COMMON_PACKAGE_PREFIX = "org.apache.kafka.common";
+ public static final String PINOT_SHADED_PACKAGE_PREFIX = "org.apache.pinot.shaded.";
+
+ /**
+ * Handle the stream config to replace the Kafka common package with the shaded version if needed.
+ */
+ public static void handleStreamConfig(StreamConfig streamConfig) {
+ Map<String, String> streamConfigMap = streamConfig.getStreamConfigsMap();
+ for (Map.Entry<String, String> entry : streamConfigMap.entrySet()) {
+ String[] valueParts = entry.getValue().split(" ");
+ boolean updated = false;
+ for (int i = 0; i < valueParts.length; i++) {
+ if (valueParts[i].startsWith(KAFKA_COMMON_PACKAGE_PREFIX)) {
+ try {
+ Class.forName(valueParts[i]);
+ } catch (ClassNotFoundException e) {
+ // If not, replace the class with the shaded version
+ valueParts[i] = PINOT_SHADED_PACKAGE_PREFIX + valueParts[i];
Review Comment:
Let's check if the shaded one can be found here
##########
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaConfigBackwardCompatibleUtils.java:
##########
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.stream.kafka20;
+
+import java.util.Map;
+import org.apache.pinot.spi.stream.StreamConfig;
+
+
+public class KafkaConfigBackwardCompatibleUtils {
+ private KafkaConfigBackwardCompatibleUtils() {
+ }
+
+ public static final String KAFKA_COMMON_PACKAGE_PREFIX = "org.apache.kafka.common";
+ public static final String PINOT_SHADED_PACKAGE_PREFIX = "org.apache.pinot.shaded.";
+
+ /**
+ * Handle the stream config to replace the Kafka common package with the shaded version if needed.
+ */
+ public static void handleStreamConfig(StreamConfig streamConfig) {
+ Map<String, String> streamConfigMap = streamConfig.getStreamConfigsMap();
+ for (Map.Entry<String, String> entry : streamConfigMap.entrySet()) {
+ String[] valueParts = entry.getValue().split(" ");
+ boolean updated = false;
+ for (int i = 0; i < valueParts.length; i++) {
+ if (valueParts[i].startsWith(KAFKA_COMMON_PACKAGE_PREFIX)) {
+ try {
+ Class.forName(valueParts[i]);
+ } catch (ClassNotFoundException e) {
+ // If not, replace the class with the shaded version
+ valueParts[i] = PINOT_SHADED_PACKAGE_PREFIX + valueParts[i];
+ updated = true;
+ }
+ }
+ }
+ if (updated) {
+ streamConfigMap.put(entry.getKey(), String.join(" ", valueParts));
Review Comment:
You may directly use `entry.setValue()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Handle backward compatibility for kafka common package references in KafkaStreamConfig [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 merged PR #13056:
URL: https://github.com/apache/pinot/pull/13056
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Handle backward compatibility for kafka common package references in KafkaStreamConfig [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #13056:
URL: https://github.com/apache/pinot/pull/13056#issuecomment-2091152543
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13056?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `0%` with `16 lines` in your changes are missing coverage. Please review.
> Project coverage is 34.99%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`159c844`)](https://app.codecov.io/gh/apache/pinot/pull/13056?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 393 commits behind head on master.
| [Files](https://app.codecov.io/gh/apache/pinot/pull/13056?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...am/kafka20/KafkaConfigBackwardCompatibleUtils.java](https://app.codecov.io/gh/apache/pinot/pull/13056?src=pr&el=tree&filepath=pinot-plugins%2Fpinot-stream-ingestion%2Fpinot-kafka-2.0%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fplugin%2Fstream%2Fkafka20%2FKafkaConfigBackwardCompatibleUtils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1zdHJlYW0taW5nZXN0aW9uL3Bpbm90LWthZmthLTIuMC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcGx1Z2luL3N0cmVhbS9rYWZrYTIwL0thZmthQ29uZmlnQmFja3dhcmRDb21wYXRpYmxlVXRpbHMuamF2YQ==) | 0.00% | [15 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/13056?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ot/plugin/stream/kafka20/KafkaConsumerFactory.java](https://app.codecov.io/gh/apache/pinot/pull/13056?src=pr&el=tree&filepath=pinot-plugins%2Fpinot-stream-ingestion%2Fpinot-kafka-2.0%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fplugin%2Fstream%2Fkafka20%2FKafkaConsumerFactory.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1zdHJlYW0taW5nZXN0aW9uL3Bpbm90LWthZmthLTIuMC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcGx1Z2luL3N0cmVhbS9rYWZrYTIwL0thZmthQ29uc3VtZXJGYWN0b3J5LmphdmE=) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/13056?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #13056 +/- ##
=============================================
- Coverage 61.75% 34.99% -26.76%
+ Complexity 207 6 -201
=============================================
Files 2436 2428 -8
Lines 133233 133030 -203
Branches 20636 20632 -4
=============================================
- Hits 82274 46550 -35724
- Misses 44911 83080 +38169
+ Partials 6048 3400 -2648
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-0.01%)` | :arrow_down: |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-0.01%)` | :arrow_down: |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (ø)` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-61.71%)` | :arrow_down: |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.99% <0.00%> (-26.64%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-61.75%)` | :arrow_down: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.99% <0.00%> (+7.26%)` | :arrow_up: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.99% <0.00%> (-26.76%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `46.49% <ø> (-15.26%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `46.49% <ø> (-0.40%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/13056/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/13056?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Handle backward compatibility for kafka common package references in KafkaStreamConfig [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on code in PR #13056:
URL: https://github.com/apache/pinot/pull/13056#discussion_r1588176478
##########
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaConfigBackwardCompatibleUtils.java:
##########
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.stream.kafka20;
+
+import java.util.Map;
+import org.apache.pinot.spi.stream.StreamConfig;
+
+
+public class KafkaConfigBackwardCompatibleUtils {
+ private KafkaConfigBackwardCompatibleUtils() {
+ }
+
+ public static final String KAFKA_COMMON_PACKAGE_PREFIX = "org.apache.kafka.common";
+ public static final String PINOT_SHADED_PACKAGE_PREFIX = "org.apache.pinot.shaded.";
+
+ /**
+ * Handle the stream config to replace the Kafka common package with the shaded version if needed.
+ */
+ public static void handleStreamConfig(StreamConfig streamConfig) {
+ Map<String, String> streamConfigMap = streamConfig.getStreamConfigsMap();
+ for (Map.Entry<String, String> entry : streamConfigMap.entrySet()) {
+ String[] valueParts = entry.getValue().split(" ");
+ boolean updated = false;
+ for (int i = 0; i < valueParts.length; i++) {
+ if (valueParts[i].startsWith(KAFKA_COMMON_PACKAGE_PREFIX)) {
+ try {
+ Class.forName(valueParts[i]);
+ } catch (ClassNotFoundException e) {
+ // If not, replace the class with the shaded version
+ valueParts[i] = PINOT_SHADED_PACKAGE_PREFIX + valueParts[i];
+ updated = true;
+ }
+ }
+ }
+ if (updated) {
+ streamConfigMap.put(entry.getKey(), String.join(" ", valueParts));
Review Comment:
done
##########
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaConfigBackwardCompatibleUtils.java:
##########
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.stream.kafka20;
+
+import java.util.Map;
+import org.apache.pinot.spi.stream.StreamConfig;
+
+
+public class KafkaConfigBackwardCompatibleUtils {
+ private KafkaConfigBackwardCompatibleUtils() {
+ }
+
+ public static final String KAFKA_COMMON_PACKAGE_PREFIX = "org.apache.kafka.common";
+ public static final String PINOT_SHADED_PACKAGE_PREFIX = "org.apache.pinot.shaded.";
+
+ /**
+ * Handle the stream config to replace the Kafka common package with the shaded version if needed.
+ */
+ public static void handleStreamConfig(StreamConfig streamConfig) {
+ Map<String, String> streamConfigMap = streamConfig.getStreamConfigsMap();
+ for (Map.Entry<String, String> entry : streamConfigMap.entrySet()) {
+ String[] valueParts = entry.getValue().split(" ");
Review Comment:
done
##########
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaConfigBackwardCompatibleUtils.java:
##########
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.stream.kafka20;
+
+import java.util.Map;
+import org.apache.pinot.spi.stream.StreamConfig;
+
+
+public class KafkaConfigBackwardCompatibleUtils {
+ private KafkaConfigBackwardCompatibleUtils() {
+ }
+
+ public static final String KAFKA_COMMON_PACKAGE_PREFIX = "org.apache.kafka.common";
+ public static final String PINOT_SHADED_PACKAGE_PREFIX = "org.apache.pinot.shaded.";
+
+ /**
+ * Handle the stream config to replace the Kafka common package with the shaded version if needed.
+ */
+ public static void handleStreamConfig(StreamConfig streamConfig) {
+ Map<String, String> streamConfigMap = streamConfig.getStreamConfigsMap();
+ for (Map.Entry<String, String> entry : streamConfigMap.entrySet()) {
+ String[] valueParts = entry.getValue().split(" ");
+ boolean updated = false;
+ for (int i = 0; i < valueParts.length; i++) {
+ if (valueParts[i].startsWith(KAFKA_COMMON_PACKAGE_PREFIX)) {
+ try {
+ Class.forName(valueParts[i]);
+ } catch (ClassNotFoundException e) {
+ // If not, replace the class with the shaded version
+ valueParts[i] = PINOT_SHADED_PACKAGE_PREFIX + valueParts[i];
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org