You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2021/10/21 16:36:46 UTC
[GitHub] [skywalking] CalvinKirs opened a new pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
CalvinKirs opened a new pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985
- [ ]Closes #7910
- [ ] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/CHANGES.md).
Writing the benchmark itself is not hard, but getting it right is. Thankfully, the JMH suite provides helpful annotations and features to mitigate most of them. To get started, you need to make your benchmark extend the AbstractMicrobenchmark, which makes sure the test gets run through JUnit and configures some defaults:
```
@BenchmarkMode({Mode.Throughput})
public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
}
```
For most benchmark tests, you can write it as a normal JUnit.
You can also choose Customizing Runtime Conditions.
At present, you can run all JMH Tests by running the jar directly after packaging.
But running all JMH directly through mvn and showing that the overall report still has some problems, you can first see if it is reasonable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-964665143
I re-write the contributing document for you, generally 2 things
1. Don't use `you could` or `you` repeatedly in the document. The document is more like an instruction, rather than command.
2. Upper case and some other grammar issues still exist, including very long sentences.
Is there anything we should add this PR? If not, please remove the `[WIP]` in the title and update changes.md in the root.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962535033
For benchmark, I am feeling do we really need to run them in a global command? Or should we only allow them runnable in specific module only?
I can't see who will run all the benchmarks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746293895
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ <uberjar.name>benchmarks</uberjar.name>
+ <maven-shade-plugin.version>3.1.1</maven-shade-plugin.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>${maven-shade-plugin.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <finalName>${uberjar.name}</finalName>
+ <transformers>
+ <transformer
+ implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </transformer>
+ </transformers>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>**/Log4j2Plugins.dat</exclude>
+ <exclude>**/EndpointGrouping4OpenapiBenchmark.*</exclude>
Review comment:
Files could be loaded through classpath or in the uber jar. You shouldn't exclude meaningful things out of packages.
What is the issue exactly here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746185489
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ <uberjar.name>benchmarks</uberjar.name>
+ <maven-shade-plugin.version>3.1.1</maven-shade-plugin.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>${maven-shade-plugin.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <finalName>${uberjar.name}</finalName>
+ <transformers>
+ <transformer
+ implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </transformer>
+ </transformers>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>**/Log4j2Plugins.dat</exclude>
+ <exclude>**/EndpointGrouping4OpenapiBenchmark.*</exclude>
Review comment:
Resource cannot be obtained when running in jar mode,
Therefore `ResourceUtils.class.getClassLoader().getResource(directoryPath)` will be empty. So I excluded this Test when packaging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744254405
##########
File path: oap-server/server-library/library-datacarrier-queue/pom.xml
##########
@@ -29,8 +29,9 @@
<dependencies>
<dependency>
- <groupId>org.openjdk.jmh</groupId>
- <artifactId>jmh-generator-annprocess</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-testing</artifactId>
+ <version>${project.version}</version>
Review comment:
Same requests for other repositories. There should be no `pom.xml` change about `jmh`->`server-testing`, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744277964
##########
File path: oap-server/microbench/src/main/java/org/apache/skywalking/oap/server/microbench/library/datacarrier/SampleData.java
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.microbench.library.datacarrier;
+
+public class SampleData {
+ private int intValue;
+
+ private String name;
+
+ public int getIntValue() {
+ return intValue;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public SampleData setIntValue(int intValue) {
+ this.intValue = intValue;
+ return this;
+ }
+
+ public SampleData setName(String name) {
+ this.name = name;
+ return this;
+ }
Review comment:
These methods seem to be useless, let's remove these or if they are needed, let's use lombok `@Getter` / `@Setter` annotation to eliminate the boilerplate codes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-951008124
> `oap-server` is already almost the top besides the legacy protocol module.
>
> `server-testing` includes class for backend modularization core only. Is there something you need to test in protocol module?
I need to add the JMH base class, so that later contributors can write the JMH Test with less generic code by inheriting the base class.
base class like this:
class.microbench/src/main/java/org/apache/skywalking/microbench/base/AbstractMicrobenchmark.java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs edited a comment on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs edited a comment on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-950987510
> > Moving the benchmark classes into another separate module makes me feeling super strange. Could you explain why? This module could be excluded(in default) from dist binary, and provide as `test` scope dependency for other modules.
> > I also mentioned `server-testing`, which is providing this. Could you explain why you can't provide the basic codes into there?
>
> Thank you for your review, it is still in progress, so I hope you can roughly see if it is reasonable, I understand what you mean, I will modify it, and I will tell you when I am done.
I'm currently trying to do this, but I'm confused. I see that the module `server testing` is under the `oap-server` module, but JMH should be required for all backend modules, and I want to move the `server-testing` module to the top-level directory ,so that it looks more appropriate. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-965991393
Thanks ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r737489391
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
I found a problem. If we want to do JMH Test on the code in library-util, then it will appear: server-testing depends on `library-module`, but library-module depends on `library-util`, and `library-util` depends on `server-testing`. This creates a circular dependency.
If server-testing wants to add JMH-related code, it should be used as the bottom-most dependency, that is, it does not depend on any module itself.
Currently, there are other temporary solutions. Only
```
org.apache.skywalking.oap.server.library.util.CollectionUtils#isNotEmpty(T[])
```
in `library-util` is used in `library-module`. We can implement this method in this module Logic so that `library-module` will be the lowest-level module, but this is not the most ideal.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r742583566
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,102 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>exec-maven-plugin</artifactId>
+ <version>${exec-maven-plugin.version}</version>
+ <configuration>
+ <executable>java</executable>
+ <arguments>
Review comment:
Thanks for the reminder, I'll fix it later ：）
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746300432
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ <uberjar.name>benchmarks</uberjar.name>
+ <maven-shade-plugin.version>3.1.1</maven-shade-plugin.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>${maven-shade-plugin.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <finalName>${uberjar.name}</finalName>
+ <transformers>
+ <transformer
+ implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </transformer>
+ </transformers>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>**/Log4j2Plugins.dat</exclude>
+ <exclude>**/EndpointGrouping4OpenapiBenchmark.*</exclude>
Review comment:
The problem is that `EndpointGrouping4OpenapiBenchmark` read files from class path but benchmark tests generate those tests outside of the class path, I'm helping @CalvinKirs to resolve this and fix some other problems in that benchmark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746261655
##########
File path: oap-server/microbench/src/main/java/org/apache/skywalking/oap/server/microbench/base/AbstractMicrobenchmark.java
##########
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.microbench.base;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.profile.GCProfiler;
+import org.openjdk.jmh.results.format.ResultFormatType;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import lombok.extern.slf4j.Slf4j;
+
+/**
+ * All JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+ * customize runtime conditions (Measurement, Fork, Warmup, etc.)
+ * <p>
+ * You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+ * -jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
+ */
+@Warmup(iterations = AbstractMicrobenchmark.DEFAULT_WARMUP_ITERATIONS)
+@Measurement(iterations = AbstractMicrobenchmark.DEFAULT_MEASURE_ITERATIONS)
+@Fork(AbstractMicrobenchmark.DEFAULT_FORKS)
+@State(Scope.Thread)
+@Slf4j
+public abstract class AbstractMicrobenchmark {
+ static final int DEFAULT_WARMUP_ITERATIONS = 10;
+
+ static final int DEFAULT_MEASURE_ITERATIONS = 10;
+
+ static final int DEFAULT_FORKS = 2;
+
+ public static class JmhThreadExecutor extends ThreadPoolExecutor {
+ public JmhThreadExecutor(int size, String name) {
+ super(size, size, 10, TimeUnit.SECONDS, new LinkedBlockingQueue<>(), Executors.defaultThreadFactory());
+ }
+ }
+
+ private ChainedOptionsBuilder newOptionsBuilder() {
+
+ String className = getClass().getSimpleName();
+
+ ChainedOptionsBuilder optBuilder = new OptionsBuilder()
+ // set benchmark class name
+ .include(".*" + className + ".*")
+ // add GC profiler
+ .addProfiler(GCProfiler.class)
+ //set jvm args
+ .jvmArgsAppend("-Xmx512m", "-Xms512m", "-XX:MaxDirectMemorySize=512m",
+ "-XX:BiasedLockingStartupDelay=0",
+ "-Djmh.executor=CUSTOM",
+ "-Djmh.executor.class=org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark$JmhThreadExecutor"
+ );
+
+ String output = getReportDir();
+ if (output != null) {
+ boolean writeFileStatus;
+ String filePath = getReportDir() + className + ".json";
+ File file = new File(filePath);
+
+ if (file.exists()) {
+ writeFileStatus = file.delete();
+ } else {
+ writeFileStatus = file.getParentFile().mkdirs();
+ try {
+ writeFileStatus = file.createNewFile();
+ } catch (IOException e) {
+ log.warn("jmh test create file error", e);
+ }
+ }
+ if (writeFileStatus) {
+ optBuilder.resultFormat(ResultFormatType.JSON)
+ .result(filePath);
+ }
+ }
+ return optBuilder;
+ }
+
+ @Test
+ public void run() throws Exception {
+ new Runner(newOptionsBuilder().build()).run();
+ }
+
+ private static String getReportDir() {
+ return System.getProperty("perfReportDir");
Review comment:
This parameter needs to be added if the user uses IDE to run it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746262691
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ <uberjar.name>benchmarks</uberjar.name>
+ <maven-shade-plugin.version>3.1.1</maven-shade-plugin.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>${maven-shade-plugin.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <finalName>${uberjar.name}</finalName>
+ <transformers>
+ <transformer
+ implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </transformer>
+ </transformers>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>**/Log4j2Plugins.dat</exclude>
+ <exclude>**/EndpointGrouping4OpenapiBenchmark.*</exclude>
Review comment:
hi, thanks for your review, because this needs to read the resource, but it cannot get the resource when running through the jar. That is what I mentioned above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r745703977
##########
File path: docs/en/guides/README.md
##########
@@ -44,6 +44,21 @@ and if you would like to run all the ITs, simply run `./mvnw -Pall,CI-with-IT cl
Please be advised that if you're writing integration tests, name it with the pattern `IT*` so they would only run with the `CI-with-IT` profile.
+### Java Microbenchmark Harness (JMH)
+JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM.
+
+We provide the `microbench` module, which provides `org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark`,
+
+all JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+customize runtime conditions (Measurement, Fork, Warmup, etc.)
+
+You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+-jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
+
+if you want to visualize your results, you can choose '-jar benchmark.jar -rf json' to export the report results in json format, there are many websites that publicly provide report analysis, you can find it yourself.
+
Review comment:
Another blank line.
##########
File path: docs/en/guides/README.md
##########
@@ -44,6 +44,21 @@ and if you would like to run all the ITs, simply run `./mvnw -Pall,CI-with-IT cl
Please be advised that if you're writing integration tests, name it with the pattern `IT*` so they would only run with the `CI-with-IT` profile.
+### Java Microbenchmark Harness (JMH)
+JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM.
+
+We provide the `microbench` module, which provides `org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark`,
+
+all JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+customize runtime conditions (Measurement, Fork, Warmup, etc.)
+
+You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+-jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
Review comment:
I think `java -jar` is not enough. Because there are dependencies in other module(depended), we need to package a uber jar for this module, right?
##########
File path: docs/en/guides/README.md
##########
@@ -44,6 +44,21 @@ and if you would like to run all the ITs, simply run `./mvnw -Pall,CI-with-IT cl
Please be advised that if you're writing integration tests, name it with the pattern `IT*` so they would only run with the `CI-with-IT` profile.
+### Java Microbenchmark Harness (JMH)
+JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM.
+
+We provide the `microbench` module, which provides `org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark`,
+
+all JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+customize runtime conditions (Measurement, Fork, Warmup, etc.)
+
+You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+-jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
+
+if you want to visualize your results, you can choose '-jar benchmark.jar -rf json' to export the report results in json format, there are many websites that publicly provide report analysis, you can find it yourself.
Review comment:
Please make sure your sentences are complete, and the first letter of sentence should be upper case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962537544
> It may be more precise, but I can't customize a certain test, so I can only run all of them.
That's because you bundle the jar as an executable binary, if you treat it as a plain jar and run the specific class with that jar as a class path, it might be possible to run a single test, like `java -cp benchmark.jar org.apache.skywalking.SampleDataBenchmarkTest`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r735721333
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
OK, then please submit another new pull request first which move the codes in `apm-commons` into `server-library/library-datacarrier-queue` and `server-library/library-util`.
This code structure is legacy before agent splitting out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r737571335
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
Maybe set up a `library-testing` as a new module?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r738346673
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
I can't see the case yet. So, if you want to move all benchmark into one module, which depends on all necessary modules, feel free to do. Just make sure all benchmarks wouldn't run in `mvn test` in default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744253806
##########
File path: oap-server/microbench/src/main/java/org/apache/skywalking/oap/server/microbench/base/AbstractMicrobenchmark.java
##########
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.microbench.base;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.profile.GCProfiler;
+import org.openjdk.jmh.results.format.ResultFormatType;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import lombok.extern.slf4j.Slf4j;
+
+@Warmup(iterations = AbstractMicrobenchmark.DEFAULT_WARMUP_ITERATIONS)
+@Measurement(iterations = AbstractMicrobenchmark.DEFAULT_MEASURE_ITERATIONS)
+@Fork(AbstractMicrobenchmark.DEFAULT_FORKS)
+@State(Scope.Thread)
+@Slf4j
+public abstract class AbstractMicrobenchmark {
+ static final int DEFAULT_WARMUP_ITERATIONS = 10;
+
+ static final int DEFAULT_MEASURE_ITERATIONS = 10;
+
+ static final int DEFAULT_FORKS = 2;
+
+ public static class JmhThreadExecutor extends ThreadPoolExecutor {
+ public JmhThreadExecutor(int size, String name) {
+ super(size, size, 10, TimeUnit.SECONDS, new LinkedBlockingQueue<>(), Executors.defaultThreadFactory());
+ }
+ }
+
+ private ChainedOptionsBuilder newOptionsBuilder() {
+
+ String className = getClass().getSimpleName();
+
+ ChainedOptionsBuilder optBuilder = new OptionsBuilder()
+ // set benchmark class name
+ .include(".*" + className + ".*")
+ // add GC profiler
+ .addProfiler(GCProfiler.class)
+ //set jvm args
+ .jvmArgsAppend("-Xmx512m", "-Xms512m", "-XX:MaxDirectMemorySize=512m",
+ "-XX:BiasedLockingStartupDelay=0",
+ "-Djmh.executor=CUSTOM", "-Djmh.executor.class=org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark$JmhThreadExecutor");
+
+ String output = getReportDir();
+ if (output != null) {
+ boolean writeFileStatus;
+ String filePath = getReportDir() + className + ".json";
+ File file = new File(filePath);
+
+ if (file.exists()) {
+ writeFileStatus = file.delete();
+ } else {
+ writeFileStatus = file.getParentFile().mkdirs();
+ try {
+ writeFileStatus = file.createNewFile();
+ } catch (IOException e) {
+ log.warn("jmh test create file error" + e);
Review comment:
```suggestion
log.warn("jmh test create file error", e);
```
##########
File path: oap-server/microbench/src/main/java/org/apache/skywalking/oap/server/microbench/base/AbstractMicrobenchmark.java
##########
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.microbench.base;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.profile.GCProfiler;
+import org.openjdk.jmh.results.format.ResultFormatType;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import lombok.extern.slf4j.Slf4j;
+
+@Warmup(iterations = AbstractMicrobenchmark.DEFAULT_WARMUP_ITERATIONS)
+@Measurement(iterations = AbstractMicrobenchmark.DEFAULT_MEASURE_ITERATIONS)
+@Fork(AbstractMicrobenchmark.DEFAULT_FORKS)
+@State(Scope.Thread)
+@Slf4j
+public abstract class AbstractMicrobenchmark {
+ static final int DEFAULT_WARMUP_ITERATIONS = 10;
+
+ static final int DEFAULT_MEASURE_ITERATIONS = 10;
+
+ static final int DEFAULT_FORKS = 2;
+
+ public static class JmhThreadExecutor extends ThreadPoolExecutor {
+ public JmhThreadExecutor(int size, String name) {
+ super(size, size, 10, TimeUnit.SECONDS, new LinkedBlockingQueue<>(), Executors.defaultThreadFactory());
+ }
+ }
+
+ private ChainedOptionsBuilder newOptionsBuilder() {
+
+ String className = getClass().getSimpleName();
+
+ ChainedOptionsBuilder optBuilder = new OptionsBuilder()
+ // set benchmark class name
+ .include(".*" + className + ".*")
Review comment:
This is not needed
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744278991
##########
File path: tools/dependencies/check-LICENSE.sh
##########
@@ -22,7 +22,7 @@ tar -zxf dist/apache-skywalking-apm-bin.tar.gz -C dist
# List all modules(jars) that belong to the SkyWalking itself, these will be ignored
# when checking the dependency licenses
-./mvnw --batch-mode -Pbackend -Dexec.executable='echo' -Dexec.args='${project.artifactId}-${project.version}.jar' exec:exec -q > self-modules.txt
+./mvnw --batch-mode -Pbackend -Dbenchmark.skip=true -Dexec.executable='echo' -Dexec.args='${project.artifactId}-${project.version}.jar' exec:exec -q > self-modules.txt
Review comment:
yup, thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746256036
##########
File path: oap-server/microbench/src/main/java/org/apache/skywalking/oap/server/microbench/base/AbstractMicrobenchmark.java
##########
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.microbench.base;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.profile.GCProfiler;
+import org.openjdk.jmh.results.format.ResultFormatType;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import lombok.extern.slf4j.Slf4j;
+
+/**
+ * All JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+ * customize runtime conditions (Measurement, Fork, Warmup, etc.)
+ * <p>
+ * You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+ * -jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
+ */
+@Warmup(iterations = AbstractMicrobenchmark.DEFAULT_WARMUP_ITERATIONS)
+@Measurement(iterations = AbstractMicrobenchmark.DEFAULT_MEASURE_ITERATIONS)
+@Fork(AbstractMicrobenchmark.DEFAULT_FORKS)
+@State(Scope.Thread)
+@Slf4j
+public abstract class AbstractMicrobenchmark {
+ static final int DEFAULT_WARMUP_ITERATIONS = 10;
+
+ static final int DEFAULT_MEASURE_ITERATIONS = 10;
+
+ static final int DEFAULT_FORKS = 2;
+
+ public static class JmhThreadExecutor extends ThreadPoolExecutor {
+ public JmhThreadExecutor(int size, String name) {
+ super(size, size, 10, TimeUnit.SECONDS, new LinkedBlockingQueue<>(), Executors.defaultThreadFactory());
+ }
+ }
+
+ private ChainedOptionsBuilder newOptionsBuilder() {
+
+ String className = getClass().getSimpleName();
+
+ ChainedOptionsBuilder optBuilder = new OptionsBuilder()
+ // set benchmark class name
+ .include(".*" + className + ".*")
+ // add GC profiler
+ .addProfiler(GCProfiler.class)
+ //set jvm args
+ .jvmArgsAppend("-Xmx512m", "-Xms512m", "-XX:MaxDirectMemorySize=512m",
+ "-XX:BiasedLockingStartupDelay=0",
+ "-Djmh.executor=CUSTOM",
+ "-Djmh.executor.class=org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark$JmhThreadExecutor"
+ );
+
+ String output = getReportDir();
+ if (output != null) {
+ boolean writeFileStatus;
+ String filePath = getReportDir() + className + ".json";
+ File file = new File(filePath);
+
+ if (file.exists()) {
+ writeFileStatus = file.delete();
+ } else {
+ writeFileStatus = file.getParentFile().mkdirs();
+ try {
+ writeFileStatus = file.createNewFile();
+ } catch (IOException e) {
+ log.warn("jmh test create file error", e);
+ }
+ }
+ if (writeFileStatus) {
+ optBuilder.resultFormat(ResultFormatType.JSON)
+ .result(filePath);
+ }
+ }
+ return optBuilder;
+ }
+
+ @Test
+ public void run() throws Exception {
+ new Runner(newOptionsBuilder().build()).run();
+ }
+
+ private static String getReportDir() {
+ return System.getProperty("perfReportDir");
Review comment:
Sorry to find that this is a built-in feature (`-rff`) of JMH so we'd better not to replicate the feature.
```
java -jar oap-server/microbench/target/benchmarks.jar -rf json -rff /tmp/result.json AtomicRangeIntegerBenchmark
```
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ <uberjar.name>benchmarks</uberjar.name>
+ <maven-shade-plugin.version>3.1.1</maven-shade-plugin.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>${maven-shade-plugin.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <finalName>${uberjar.name}</finalName>
+ <transformers>
+ <transformer
+ implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </transformer>
+ </transformers>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>**/Log4j2Plugins.dat</exclude>
+ <exclude>**/EndpointGrouping4OpenapiBenchmark.*</exclude>
Review comment:
I meant this one `EndpointGrouping4OpenapiBenchmark`, why this benchmark is excluded? If I want to run this benchmark, it throws NCDF
```
java -jar oap-server/microbench/target/benchmarks.jar EndpointGrouping4OpenapiBenchmark
# JMH version: 1.25
# VM version: JDK 1.8.0_292, OpenJDK 64-Bit Server VM, 25.292-b10
# VM invoker: /Users/kezhenxu94/.asdf/installs/java/adoptopenjdk-8.0.292+10/jre/bin/java
# VM options: <none>
# Warmup: 10 iterations, 10 s each
# Measurement: 10 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 4 threads, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.skywalking.oap.server.microbench.core.config.group.openapi.EndpointGrouping4OpenapiBenchmark.formatEndpointNameMatchedPaths20
# Run progress: 0.00% complete, ETA 00:20:00
# Fork: 1 of 2
<failure>
java.lang.NoClassDefFoundError: org/apache/skywalking/oap/server/microbench/core/config/group/openapi/EndpointGrouping4OpenapiBenchmark
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng merged pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng merged pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746170196
##########
File path: oap-server/microbench/src/main/java/org/apache/skywalking/oap/server/microbench/base/AbstractMicrobenchmark.java
##########
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.microbench.base;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.profile.GCProfiler;
+import org.openjdk.jmh.results.format.ResultFormatType;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import lombok.extern.slf4j.Slf4j;
+
+/**
+ * All JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+ * customize runtime conditions (Measurement, Fork, Warmup, etc.)
+ * <p>
+ * You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+ * -jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
+ */
+@Warmup(iterations = AbstractMicrobenchmark.DEFAULT_WARMUP_ITERATIONS)
+@Measurement(iterations = AbstractMicrobenchmark.DEFAULT_MEASURE_ITERATIONS)
+@Fork(AbstractMicrobenchmark.DEFAULT_FORKS)
+@State(Scope.Thread)
+@Slf4j
+public abstract class AbstractMicrobenchmark {
+ static final int DEFAULT_WARMUP_ITERATIONS = 10;
+
+ static final int DEFAULT_MEASURE_ITERATIONS = 10;
+
+ static final int DEFAULT_FORKS = 2;
+
+ public static class JmhThreadExecutor extends ThreadPoolExecutor {
+ public JmhThreadExecutor(int size, String name) {
+ super(size, size, 10, TimeUnit.SECONDS, new LinkedBlockingQueue<>(), Executors.defaultThreadFactory());
+ }
+ }
+
+ private ChainedOptionsBuilder newOptionsBuilder() {
+
+ String className = getClass().getSimpleName();
+
+ ChainedOptionsBuilder optBuilder = new OptionsBuilder()
+ // set benchmark class name
+ .include(".*" + className + ".*")
+ // add GC profiler
+ .addProfiler(GCProfiler.class)
+ //set jvm args
+ .jvmArgsAppend("-Xmx512m", "-Xms512m", "-XX:MaxDirectMemorySize=512m",
+ "-XX:BiasedLockingStartupDelay=0",
+ "-Djmh.executor=CUSTOM",
+ "-Djmh.executor.class=org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark$JmhThreadExecutor"
+ );
+
+ String output = getReportDir();
+ if (output != null) {
+ boolean writeFileStatus;
+ String filePath = getReportDir() + className + ".json";
+ File file = new File(filePath);
+
+ if (file.exists()) {
+ writeFileStatus = file.delete();
+ } else {
+ writeFileStatus = file.getParentFile().mkdirs();
+ try {
+ writeFileStatus = file.createNewFile();
+ } catch (IOException e) {
+ log.warn("jmh test create file error", e);
+ }
+ }
+ if (writeFileStatus) {
+ optBuilder.resultFormat(ResultFormatType.JSON)
+ .result(filePath);
+ }
+ }
+ return optBuilder;
+ }
+
+ @Test
+ public void run() throws Exception {
+ new Runner(newOptionsBuilder().build()).run();
+ }
+
+ private static String getReportDir() {
+ return System.getProperty("perfReportDir");
Review comment:
This is not documented.
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ <uberjar.name>benchmarks</uberjar.name>
+ <maven-shade-plugin.version>3.1.1</maven-shade-plugin.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>${maven-shade-plugin.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <finalName>${uberjar.name}</finalName>
+ <transformers>
+ <transformer
+ implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </transformer>
+ </transformers>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>**/Log4j2Plugins.dat</exclude>
+ <exclude>**/EndpointGrouping4OpenapiBenchmark.*</exclude>
Review comment:
Why exclude this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-951003223
`oap-server` is already almost the top besides the legacy protocol module.
`server-testing` includes class for backend modularization core only. Is there something you need to test in protocol module?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-950987510
> > Moving the benchmark classes into another separate module makes me feeling super strange. Could you explain why? This module could be excluded(in default) from dist binary, and provide as `test` scope dependency for other modules.
> > I also mentioned `server-testing`, which is providing this. Could you explain why you can't provide the basic codes into there?
>
> Thank you for your review, it is still in progress, so I hope you can roughly see if it is reasonable, I understand what you mean, I will modify it, and I will tell you when I am done.
I'm currently trying to do this, but I'm confused. I see that the module "server testing" is under the `oap-server` module, but JMH should be required for all backend modules, and I want to move the `server-testing` module to the top-level directory ,so that it looks more appropriate. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r742522212
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,102 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>exec-maven-plugin</artifactId>
+ <version>${exec-maven-plugin.version}</version>
+ <configuration>
+ <executable>java</executable>
+ <arguments>
Review comment:
Let's add a property to disable this task, as we use the same plugin when checking license dependencies, which will run this unexpectedly
```xml
<properties>
<benchmark.skip>false</benchmark.skip>
</properties>
```
```suggestion
<skip>${benchmark.skip}</skip>
<arguments>
```
and disable it in https://github.com/apache/skywalking/blob/26b81a2e5991ecdd2bde0f7071264cb80bb235cf/tools/dependencies/check-LICENSE.sh#L25
```diff
- ./mvnw --batch-mode -Pbackend -Dexec.executable='echo' -Dexec.args='${project.artifactId}-${project.version}.jar' exec:exec -q > self-modules.txt
+ ./mvnw --batch-mode -Pbackend -Dbenchmark.skip=true -Dexec.executable='echo' -Dexec.args='${project.artifactId}-${project.version}.jar' exec:exec -q > self-modules.txt
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r745759765
##########
File path: docs/en/guides/README.md
##########
@@ -44,6 +44,21 @@ and if you would like to run all the ITs, simply run `./mvnw -Pall,CI-with-IT cl
Please be advised that if you're writing integration tests, name it with the pattern `IT*` so they would only run with the `CI-with-IT` profile.
+### Java Microbenchmark Harness (JMH)
+JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM.
+
+We provide the `microbench` module, which provides `org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark`,
+
+all JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+customize runtime conditions (Measurement, Fork, Warmup, etc.)
+
+You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+-jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
+
+if you want to visualize your results, you can choose '-jar benchmark.jar -rf json' to export the report results in json format, there are many websites that publicly provide report analysis, you can find it yourself.
Review comment:
Thank you for your comment, please take a look again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r738341902
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
> If we move all benchmarks into this module, then this module has to depend on all relative modules, rather than reverse.
Yes, so it is only suitable for benchmarking.
>
> > or get all the benchmark test results through a simple mvn run test
>
> This should not be as an option. Benchmarks are not recommended to run in any case, they are just archived into the source codes.
What I think is whether it is possible to write related benchmark tests for certain core link layers, such as our lightweight queue. I am not sure whether there is a demand for this, what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r745707861
##########
File path: docs/en/guides/README.md
##########
@@ -44,6 +44,21 @@ and if you would like to run all the ITs, simply run `./mvnw -Pall,CI-with-IT cl
Please be advised that if you're writing integration tests, name it with the pattern `IT*` so they would only run with the `CI-with-IT` profile.
+### Java Microbenchmark Harness (JMH)
+JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM.
+
+We provide the `microbench` module, which provides `org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark`,
+
+all JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+customize runtime conditions (Measurement, Fork, Warmup, etc.)
+
+You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+-jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
Review comment:
Yes, the benchmark is the final uber package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746262691
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ <uberjar.name>benchmarks</uberjar.name>
+ <maven-shade-plugin.version>3.1.1</maven-shade-plugin.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>${maven-shade-plugin.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <finalName>${uberjar.name}</finalName>
+ <transformers>
+ <transformer
+ implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </transformer>
+ </transformers>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>**/Log4j2Plugins.dat</exclude>
+ <exclude>**/EndpointGrouping4OpenapiBenchmark.*</exclude>
Review comment:
hi, thanks for your review, It needs to read the resource because it needs it, but it cannot get the resource when running through the jar. That is what I mentioned above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r738341902
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
> If we move all benchmarks into this module, then this module has to depend on all relative modules, rather than reverse.
Yes, so it is only suitable for benchmarking.
>
> > or get all the benchmark test results through a simple mvn run test
>
> This should not be as an option. Benchmarks are not recommended to run in any case, they are just archived into the source codes.
What I think is whether it is possible to write related benchmark tests for certain core link layers, such as our lightweight queue. I am not sure whether there is a demand for this, what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r738298887
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
I thought about doing this, but I want to use this module only for benchmarking.
All benchmark test codes are moved to this module separately so that they can be well isolated from ordinary UT. Contributors can choose to run the ones they are interested in, or get all the benchmark test results through a simple `mvn run test`. Wouldn't it be better?
It seems to have returned to the earliest topic.：）
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r738327897
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
If we move all benchmarks into this module, then this module has to depend on all relative modules, rather than reverse.
> or get all the benchmark test results through a simple mvn run test
This should not be as an option. Benchmarks are not recommended to run in any case, they are just archived into the source codes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744279256
##########
File path: oap-server/microbench/src/main/java/org/apache/skywalking/oap/server/microbench/library/datacarrier/SampleData.java
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.microbench.library.datacarrier;
+
+public class SampleData {
+ private int intValue;
+
+ private String name;
+
+ public int getIntValue() {
+ return intValue;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public SampleData setIntValue(int intValue) {
+ this.intValue = intValue;
+ return this;
+ }
+
+ public SampleData setName(String name) {
+ this.name = name;
+ return this;
+ }
Review comment:
Yes, I have deleted its get&set. In fact, its parameters are not used, let's keep its parameters for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746239711
##########
File path: oap-server/microbench/src/main/java/org/apache/skywalking/oap/server/microbench/base/AbstractMicrobenchmark.java
##########
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.microbench.base;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.profile.GCProfiler;
+import org.openjdk.jmh.results.format.ResultFormatType;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import lombok.extern.slf4j.Slf4j;
+
+/**
+ * All JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+ * customize runtime conditions (Measurement, Fork, Warmup, etc.)
+ * <p>
+ * You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+ * -jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
+ */
+@Warmup(iterations = AbstractMicrobenchmark.DEFAULT_WARMUP_ITERATIONS)
+@Measurement(iterations = AbstractMicrobenchmark.DEFAULT_MEASURE_ITERATIONS)
+@Fork(AbstractMicrobenchmark.DEFAULT_FORKS)
+@State(Scope.Thread)
+@Slf4j
+public abstract class AbstractMicrobenchmark {
+ static final int DEFAULT_WARMUP_ITERATIONS = 10;
+
+ static final int DEFAULT_MEASURE_ITERATIONS = 10;
+
+ static final int DEFAULT_FORKS = 2;
+
+ public static class JmhThreadExecutor extends ThreadPoolExecutor {
+ public JmhThreadExecutor(int size, String name) {
+ super(size, size, 10, TimeUnit.SECONDS, new LinkedBlockingQueue<>(), Executors.defaultThreadFactory());
+ }
+ }
+
+ private ChainedOptionsBuilder newOptionsBuilder() {
+
+ String className = getClass().getSimpleName();
+
+ ChainedOptionsBuilder optBuilder = new OptionsBuilder()
+ // set benchmark class name
+ .include(".*" + className + ".*")
+ // add GC profiler
+ .addProfiler(GCProfiler.class)
+ //set jvm args
+ .jvmArgsAppend("-Xmx512m", "-Xms512m", "-XX:MaxDirectMemorySize=512m",
+ "-XX:BiasedLockingStartupDelay=0",
+ "-Djmh.executor=CUSTOM",
+ "-Djmh.executor.class=org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark$JmhThreadExecutor"
+ );
+
+ String output = getReportDir();
+ if (output != null) {
+ boolean writeFileStatus;
+ String filePath = getReportDir() + className + ".json";
+ File file = new File(filePath);
+
+ if (file.exists()) {
+ writeFileStatus = file.delete();
+ } else {
+ writeFileStatus = file.getParentFile().mkdirs();
+ try {
+ writeFileStatus = file.createNewFile();
+ } catch (IOException e) {
+ log.warn("jmh test create file error", e);
+ }
+ }
+ if (writeFileStatus) {
+ optBuilder.resultFormat(ResultFormatType.JSON)
+ .result(filePath);
+ }
+ }
+ return optBuilder;
+ }
+
+ @Test
+ public void run() throws Exception {
+ new Runner(newOptionsBuilder().build()).run();
+ }
+
+ private static String getReportDir() {
+ return System.getProperty("perfReportDir");
Review comment:
Thank you for your reminder, I have explained it in the document, thank you again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r746262691
##########
File path: oap-server/microbench/pom.xml
##########
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>oap-server</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>microbench</artifactId>
+
+ <properties>
+ <jmh.version>1.25</jmh.version>
+ <slf4j.version>1.7.30</slf4j.version>
+ <uberjar.name>benchmarks</uberjar.name>
+ <maven-shade-plugin.version>3.1.1</maven-shade-plugin.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>${maven-shade-plugin.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <finalName>${uberjar.name}</finalName>
+ <transformers>
+ <transformer
+ implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </transformer>
+ </transformers>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>**/Log4j2Plugins.dat</exclude>
+ <exclude>**/EndpointGrouping4OpenapiBenchmark.*</exclude>
Review comment:
hi, thanks for your review, It needs to read the resource, but it cannot get the resource when running through the jar. That is what I mentioned above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744344580
##########
File path: oap-server/server-receiver-plugin/skywalking-zabbix-receiver-plugin/pom.xml
##########
@@ -35,14 +35,15 @@
</dependency>
<dependency>
<groupId>org.apache.skywalking</groupId>
- <artifactId>meter-analyzer</artifactId>
+ <artifactId>server-testing</artifactId>
<version>${project.version}</version>
+ <scope>test</scope>
</dependency>
<dependency>
- <groupId>org.openjdk.jmh</groupId>
- <artifactId>jmh-generator-annprocess</artifactId>
- <scope>test</scope>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>meter-analyzer</artifactId>
+ <version>${project.version}</version>
Review comment:
Why do we add `server-testing` when remove `jmh-generator-annprocess`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744344320
##########
File path: oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/config/group/openapi/EndpointGroupingRule4Openapi.java
##########
@@ -34,7 +34,7 @@ void addDirectLookup(String serviceName, String endpointName, String endpointGro
endpointNameLookup.put(endpointName, endpointGroupName);
}
- void addGroupedRule(String serviceName, String endpointGroupName, String ruleRegex) {
+ public void addGroupedRule(String serviceName, String endpointGroupName, String ruleRegex) {
Review comment:
Exposing new method is not recommended usually. I think this method is only used in benchmark initialization stage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-964707109
> I re-write the contributing document for you, generally 2 things
>
> 1. Don't use `you could` or `you` repeatedly in the document. The document is more like an instruction, rather than command.
> 2. Upper case and some other grammar issues still exist, including very long sentences.
>
> Is there anything we should add this PR? If not, please remove the `[WIP]` in the title and update changes.md in the root.
Thank you very much
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r745707861
##########
File path: docs/en/guides/README.md
##########
@@ -44,6 +44,21 @@ and if you would like to run all the ITs, simply run `./mvnw -Pall,CI-with-IT cl
Please be advised that if you're writing integration tests, name it with the pattern `IT*` so they would only run with the `CI-with-IT` profile.
+### Java Microbenchmark Harness (JMH)
+JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM.
+
+We provide the `microbench` module, which provides `org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark`,
+
+all JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
+customize runtime conditions (Measurement, Fork, Warmup, etc.)
+
+You can run any of the JMH tests as a normal UT, or you can package it and get all the reported results via `java
+-jar benchmark.jar`, or get the results of a particular Test via `java -jar /benchmarks.jar exampleClassName`.
Review comment:
Yes, the `benchmark.jar` is the final uber package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r745701788
##########
File path: docs/en/guides/README.md
##########
@@ -44,6 +44,21 @@ and if you would like to run all the ITs, simply run `./mvnw -Pall,CI-with-IT cl
Please be advised that if you're writing integration tests, name it with the pattern `IT*` so they would only run with the `CI-with-IT` profile.
+### Java Microbenchmark Harness (JMH)
+JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM.
+
+We provide the `microbench` module, which provides `org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark`,
+
+all JMH tests need to extend this class to make it easier for you to complete JMHTest, you can also choose to
Review comment:
What is this blank line for?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744277409
##########
File path: tools/dependencies/check-LICENSE.sh
##########
@@ -22,7 +22,7 @@ tar -zxf dist/apache-skywalking-apm-bin.tar.gz -C dist
# List all modules(jars) that belong to the SkyWalking itself, these will be ignored
# when checking the dependency licenses
-./mvnw --batch-mode -Pbackend -Dexec.executable='echo' -Dexec.args='${project.artifactId}-${project.version}.jar' exec:exec -q > self-modules.txt
+./mvnw --batch-mode -Pbackend -Dbenchmark.skip=true -Dexec.executable='echo' -Dexec.args='${project.artifactId}-${project.version}.jar' exec:exec -q > self-modules.txt
Review comment:
We don't need this now, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962566350
> > It may be more precise, but I can't customize a certain test, so I can only run all of them.
>
> That's because you bundle the jar as an executable binary, if you treat it as a plain jar and run the specific class with that jar as a class path, it might be possible to run a single test, like `java -cp benchmark.jar org.apache.skywalking.SampleDataBenchmarkTest`.
deeply thanks for your suggestion, let me finish it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744254269
##########
File path: oap-server/server-library/library-datacarrier-queue/pom.xml
##########
@@ -29,8 +29,9 @@
<dependencies>
<dependency>
- <groupId>org.openjdk.jmh</groupId>
- <artifactId>jmh-generator-annprocess</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-testing</artifactId>
+ <version>${project.version}</version>
Review comment:
I think `LinkedArrayBenchmark` has been moved into `microbench` module already. We don't remove this dependency, and existing test case should be removed too, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-963861192
@CalvinKirs Any feedback?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962477785
Now, after packaging, we can execute JMHtest through `java -jar benchmark.jar`, and if we want to write a JMHtest, we only need to inherit `AbstractMicrobenchmark` without paying too much attention to the many details of JMH.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962478272
> Now, after packaging, we can execute JMHtest through `java -jar benchmark.jar`, and if we want to write a JMHtest, we only need to inherit `AbstractMicrobenchmark` without paying too much attention to the many details of JMH.
If we have multiple benchmark tests in benchmark module (finally `benchmark.jar`), can we choose to run only a specific class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-949480441
> Moving the benchmark classes into another separate module makes me feeling super strange. Could you explain why? This module could be excluded(in default) from dist binary, and provide as `test` scope dependency for other modules.
>
> I also mentioned `server-testing`, which is providing this. Could you explain why you can't provide the basic codes into there?
Thank you for your review, it is still in progress, so I hope you can roughly see if it is reasonable, I understand what you mean, I will modify it, and I will tell you when I am done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r735692830
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
@wu-sheng Like this, when custom parameters are needed in special cases, such as forks, warm-up interactions, etc... He can add annotations to override them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r734401302
##########
File path: microbench/pom.xml
##########
@@ -0,0 +1,129 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>apm</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>8.9.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>microbench</artifactId>
+ <properties>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <jmh.version>1.21</jmh.version>
+ <javac.target>11</javac.target>
+ <slf4j.version>1.7.30</slf4j.version>
+ <uberjar.name>benchmarks</uberjar.name>
+ </properties>
+
+ <dependencies>
+ <!--JMH-->
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!--SLF4j-->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!--JUNIT-->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>compile</scope>
+ </dependency>
+ <!--SkyWalking Module-->
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>apm-util</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>${maven-compiler-plugin.version}</version>
+ <configuration>
+ <compilerVersion>${javac.target}</compilerVersion>
+ <source>${javac.target}</source>
+ <target>${javac.target}</target>
+ <useIncrementalCompilation>false</useIncrementalCompilation>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-assembly-plugin</artifactId>
+ <version>${maven-assembly-plugin.version}</version>
+ <configuration>
+ <descriptorRefs>
+ <descriptorRef>jar-with-dependencies</descriptorRef>
+ </descriptorRefs>
+ <archive>
+ <manifest>
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </manifest>
+ </archive>
+ </configuration>
+ <executions>
+ <execution>
+ <id>make-assembly</id>
+ <phase>package</phase>
+ <goals>
+ <!-- <goal>assembly</goal> -->
+ <goal>single</goal>
Review comment:
What does this mean?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-949471788
Moving the benchmark classes into another separate module makes me feeling super strange. Could you explain why? This module could be excluded(in default) from dist binary, and provide as `test` scope dependency for other modules.
I also mentioned `server-testing`, which is providing this. Could you explain why you can't provide the basic codes into there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r736097364
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
Okay, I'm on it.：）
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r738349184
##########
File path: apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java
##########
@@ -35,7 +37,7 @@
* ISSUE-3064
*/
@BenchmarkMode({Mode.Throughput})
-public class LinkedArrayBenchmark {
+public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
@Benchmark
Review comment:
Well, thank you very much, I will do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-963885300
> @CalvinKirs Any feedback?
Sorry, I will update as soon as possible, thanks for your review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962480727
> > Now, after packaging, we can execute JMHtest through `java -jar benchmark.jar`, and if we want to write a JMHtest, we only need to inherit `AbstractMicrobenchmark` without paying too much attention to the many details of JMH.
>
> If we have multiple benchmark tests in benchmark module (finally `benchmark.jar`), can we choose to run only a specific class?
Can not do it now :sweat_smile:
If you only want to test a certain test individually, it is more appropriate to do a basic test in the local environment is more appropriate.
You can run it like a normal UT and get the result. At this time, there is usually no need for more warm-up and test iterations. We just want a rough result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962536551
> For benchmark, I am feeling do we really need to run them in a global command? Or should we only allow them runnable in specific module only? I can't see who will run all the benchmarks.
What I think is that if we optimize on a certain core link, we can show our results through JMH. Often the local execution of the development environment can only get a rough result. At this time, the result is executed on a specific server in the form of jar. It may be more precise, but I can't customize a certain test, so I can only run all of them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962604332
Turns out that benchmark already allows specifying the class names/patterns to execute, so we can simply use `java -jar oap-server/microbench/target/benchmarks.jar LinkedArrayBenchmark` to only run `LinkedArrayBenchmark`.
@CalvinKirs we can document this and we are all set 🎉
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962718509
I can't see document update in the https://github.com/apache/skywalking/blob/master/docs/en/guides/README.md#integration-tests. We should add `benchmark` section after `Integration Tests`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744276307
##########
File path: oap-server/microbench/src/main/java/org/apache/skywalking/oap/server/microbench/base/AbstractMicrobenchmark.java
##########
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.microbench.base;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.profile.GCProfiler;
+import org.openjdk.jmh.results.format.ResultFormatType;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import lombok.extern.slf4j.Slf4j;
+
+@Warmup(iterations = AbstractMicrobenchmark.DEFAULT_WARMUP_ITERATIONS)
+@Measurement(iterations = AbstractMicrobenchmark.DEFAULT_MEASURE_ITERATIONS)
+@Fork(AbstractMicrobenchmark.DEFAULT_FORKS)
+@State(Scope.Thread)
+@Slf4j
+public abstract class AbstractMicrobenchmark {
+ static final int DEFAULT_WARMUP_ITERATIONS = 10;
+
+ static final int DEFAULT_MEASURE_ITERATIONS = 10;
+
+ static final int DEFAULT_FORKS = 2;
+
+ public static class JmhThreadExecutor extends ThreadPoolExecutor {
+ public JmhThreadExecutor(int size, String name) {
+ super(size, size, 10, TimeUnit.SECONDS, new LinkedBlockingQueue<>(), Executors.defaultThreadFactory());
+ }
+ }
+
+ private ChainedOptionsBuilder newOptionsBuilder() {
+
+ String className = getClass().getSimpleName();
+
+ ChainedOptionsBuilder optBuilder = new OptionsBuilder()
+ // set benchmark class name
+ .include(".*" + className + ".*")
+ // add GC profiler
+ .addProfiler(GCProfiler.class)
+ //set jvm args
+ .jvmArgsAppend("-Xmx512m", "-Xms512m", "-XX:MaxDirectMemorySize=512m",
+ "-XX:BiasedLockingStartupDelay=0",
+ "-Djmh.executor=CUSTOM", "-Djmh.executor.class=org.apache.skywalking.oap.server.microbench.base.AbstractMicrobenchmark$JmhThreadExecutor");
+
+ String output = getReportDir();
+ if (output != null) {
+ boolean writeFileStatus;
+ String filePath = getReportDir() + className + ".json";
+ File file = new File(filePath);
+
+ if (file.exists()) {
+ writeFileStatus = file.delete();
+ } else {
+ writeFileStatus = file.getParentFile().mkdirs();
+ try {
+ writeFileStatus = file.createNewFile();
+ } catch (IOException e) {
+ log.warn("jmh test create file error" + e);
Review comment:
Thanks for your checking, I have used the plug-in to check it again and fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on a change in pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on a change in pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#discussion_r744276134
##########
File path: oap-server/server-library/library-datacarrier-queue/pom.xml
##########
@@ -29,8 +29,9 @@
<dependencies>
<dependency>
- <groupId>org.openjdk.jmh</groupId>
- <artifactId>jmh-generator-annprocess</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-testing</artifactId>
+ <version>${project.version}</version>
Review comment:
Yes I have removed this, thank you for checking
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] CalvinKirs commented on pull request #7985: [WIP]Add MicroBench module to make it easier for developers to write JMH Test
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on pull request #7985:
URL: https://github.com/apache/skywalking/pull/7985#issuecomment-962635522
> Just some nits, others look good to me, thanks @CalvinKirs !!
thanks ：）
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org