You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2021/08/15 13:56:48 UTC
[skywalking] branch master updated: Fix kafka-reporter-plugin shade
package conflict (#7463)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new ce45397 Fix kafka-reporter-plugin shade package conflict (#7463)
ce45397 is described below
commit ce45397c5673673ccabec994cb4afe58ce3bb241
Author: hailin0 <ha...@yeah.net>
AuthorDate: Sun Aug 15 21:55:45 2021 +0800
Fix kafka-reporter-plugin shade package conflict (#7463)
* Fix kafka-reporter-plugin shade package conflict
Don't shade lz4, snappy or zstd package because they call native library files.
---
CHANGES.md | 1 +
.../kafka-reporter-plugin/pom.xml | 75 +++++++++++++---------
.../java-agent/How-to-enable-kafka-reporter.md | 2 +
test/e2e/e2e-test/docker/kafka/Dockerfile.provider | 2 +
4 files changed, 48 insertions(+), 32 deletions(-)
diff --git a/CHANGES.md b/CHANGES.md
index efb4f49..8f7c729 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -15,6 +15,7 @@ Release Notes.
* Support Multiple DNS period resolving mechanism
* Modify `Tags.STATUS_CODE` field name to `Tags.HTTP_RESPONSE_STATUS_CODE` and type from `StringTag` to `IntegerTag`, add `Tags.RPC_RESPONSE_STATUS_CODE` field to hold rpc response code value.
+* Fix kafka-reporter-plugin shade package conflict
#### OAP-Backend
diff --git a/apm-sniffer/optional-reporter-plugins/kafka-reporter-plugin/pom.xml b/apm-sniffer/optional-reporter-plugins/kafka-reporter-plugin/pom.xml
index 6b1a1c6..c60ac20 100644
--- a/apm-sniffer/optional-reporter-plugins/kafka-reporter-plugin/pom.xml
+++ b/apm-sniffer/optional-reporter-plugins/kafka-reporter-plugin/pom.xml
@@ -52,6 +52,13 @@
<createDependencyReducedPom>true</createDependencyReducedPom>
<createSourcesJar>true</createSourcesJar>
<shadeSourcesContent>true</shadeSourcesContent>
+ <artifactSet>
+ <excludes>
+ <exclude>org.xerial.snappy:snappy-java</exclude>
+ <exclude>org.lz4:lz4-java</exclude>
+ <exclude>com.github.luben:zstd-jni</exclude>
+ </excludes>
+ </artifactSet>
<relocations>
<relocation>
<pattern>org.apache.kafka</pattern>
@@ -69,38 +76,6 @@
<pattern>org.slf4j</pattern>
<shadedPattern>${shade.package}/org.slf4j</shadedPattern>
</relocation>
- <relocation>
- <pattern>org.xerial</pattern>
- <shadedPattern>${shade.package}/org.xerial</shadedPattern>
- </relocation>
- <relocation>
- <pattern>net.jpountz</pattern>
- <shadedPattern>${shade.package}/net.jpountz</shadedPattern>
- </relocation>
- <relocation>
- <pattern>aix.ppc64</pattern>
- <shadedPattern>${shade.package}/aix.ppc64</shadedPattern>
- </relocation>
- <relocation>
- <pattern>com.github.luben</pattern>
- <shadedPattern>${shade.package}/com.github.luben</shadedPattern>
- </relocation>
- <relocation>
- <pattern>darwin</pattern>
- <shadedPattern>${shade.package}/darwin</shadedPattern>
- </relocation>
- <relocation>
- <pattern>include</pattern>
- <shadedPattern>${shade.package}/include</shadedPattern>
- </relocation>
- <relocation>
- <pattern>linux</pattern>
- <shadedPattern>${shade.package}/linux</shadedPattern>
- </relocation>
- <relocation>
- <pattern>win</pattern>
- <shadedPattern>${shade.package}/win</shadedPattern>
- </relocation>
</relocations>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
@@ -150,6 +125,42 @@
</dependency>
</dependencies>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <version>${maven-dependency-plugin.version}</version>
+ <executions>
+ <execution>
+ <id>copy</id>
+ <phase>package</phase>
+ <goals>
+ <goal>copy</goal>
+ </goals>
+ <configuration>
+ <artifactItems>
+ <artifactItem>
+ <groupId>org.xerial.snappy</groupId>
+ <artifactId>snappy-java</artifactId>
+ <version>1.1.7.3</version>
+ <outputDirectory>${optional.reporter.plugins.dest.dir}</outputDirectory>
+ </artifactItem>
+ <artifactItem>
+ <groupId>org.lz4</groupId>
+ <artifactId>lz4-java</artifactId>
+ <version>1.6.0</version>
+ <outputDirectory>${optional.reporter.plugins.dest.dir}</outputDirectory>
+ </artifactItem>
+ <artifactItem>
+ <groupId>com.github.luben</groupId>
+ <artifactId>zstd-jni</artifactId>
+ <version>1.4.3-1</version>
+ <outputDirectory>${optional.reporter.plugins.dest.dir}</outputDirectory>
+ </artifactItem>
+ </artifactItems>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
</plugins>
</build>
diff --git a/docs/en/setup/service-agent/java-agent/How-to-enable-kafka-reporter.md b/docs/en/setup/service-agent/java-agent/How-to-enable-kafka-reporter.md
index 7e7b824..baf9c69 100644
--- a/docs/en/setup/service-agent/java-agent/How-to-enable-kafka-reporter.md
+++ b/docs/en/setup/service-agent/java-agent/How-to-enable-kafka-reporter.md
@@ -2,6 +2,8 @@
The Kafka reporter plugin support report traces, JVM metrics, Instance Properties, and profiled snapshots to Kafka cluster, which is disabled in default. Move the jar of the plugin, `kafka-reporter-plugin-x.y.z.jar`, from `agent/optional-reporter-plugins` to `agent/plugins` for activating.
+If you configure to use `compression.type` such as `lz4`, `zstd`, `snappy`, etc., you also need to move the jar of the plugin, `lz4-java-x.y.z.jar` or `zstd-jni-x.y.z.jar` or `snappy-java.x.y.z.jar`, from `agent/optional-reporter-plugins` to `agent/plugins`.
+
Notice, currently, the agent still needs to configure GRPC receiver for delivering the task of profiling. In other words, the following configure cannot be omitted.
```properties
diff --git a/test/e2e/e2e-test/docker/kafka/Dockerfile.provider b/test/e2e/e2e-test/docker/kafka/Dockerfile.provider
index 3e61dc0..916a9b2 100644
--- a/test/e2e/e2e-test/docker/kafka/Dockerfile.provider
+++ b/test/e2e/e2e-test/docker/kafka/Dockerfile.provider
@@ -27,6 +27,7 @@ RUN mv apache-skywalking-apm-bin/* . || true
RUN sed -i 's/# plugin.kafka.bootstrap_servers/plugin.kafka.bootstrap_servers/g' agent/config/agent.config
RUN echo 'plugin.kafka.get_topic_timeout=${SW_GET_TOPIC_TIMEOUT:60}' >> agent/config/agent.config
+RUN echo 'plugin.kafka.producer_config[compression.type]=snappy' >> agent/config/agent.config
# add log reporter configuration
RUN echo 'plugin.toolkit.log.grpc.reporter.server_host=${SW_GRPC_LOG_SERVER_HOST:oap}' >> agent/config/agent.config
@@ -34,6 +35,7 @@ RUN echo 'plugin.toolkit.log.grpc.reporter.server_port=${SW_GRPC_LOG_SERVER_PORT
# activate kafka-reporter plugin
RUN mv agent/optional-reporter-plugins/kafka-reporter-plugin-*.jar agent/plugins/ || true
+RUN mv agent/optional-reporter-plugins/snappy-java-*.jar agent/plugins/ || true
VOLUME /services