You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2022/03/22 12:35:12 UTC
[GitHub] [skywalking] kezhenxu94 opened a new pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
kezhenxu94 opened a new pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725
<!--
⚠️ Please make sure to read this template first, pull requests that don't accord with this template
maybe closed without notice.
Texts surrounded by `<` and `>` are meant to be replaced by you, e.g. <framework name>, <issue number>.
Put an `x` in the `[ ]` to mark the item as CHECKED. `[x]`
-->
<!-- ==== 🐛 Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist 👇 ====
### Fix <bug description or the bug issue number or bug issue link>
- [ ] Add a unit test to verify that the fix works.
- [ ] Explain briefly why the bug exists and how to fix it.
==== 🐛 Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist 👆 ==== -->
<!-- ==== 📈 Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist 👇 ====
### Improve the performance of <class or module or ...>
- [ ] Add a benchmark for the improvement, refer to [the existing ones](https://github.com/apache/skywalking/blob/master/apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java)
- [ ] The benchmark result.
```text
<Paste the benchmark results here>
```
- [ ] Links/URLs to the theory proof or discussion articles/blogs. <links/URLs here>
==== 📈 Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist 👆 ==== -->
<!-- ==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👇 ====
### <Feature description>
- [ ] If this is non-trivial feature, paste the links/URLs to the design doc.
- [ ] Update the documentation to include this new feature.
- [ ] Tests(including UT, IT, E2E) are added to verify the new feature.
- [ ] If it's UI related, attach the screenshots below.
==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👆 ==== -->
- [x] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #8697 .
- [x] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/CHANGES.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] hanahmily commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832793189
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
Review comment:
We need a `time window` param here. The generated segments should go into this window.
If we pass the `time window`, the `size` could be 0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832802535
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
+ @RequestObject SegmentRequest request) {
+
+ if (size > 0 && qps > 0) {
+ return HttpResponse.of(
+ HttpStatus.BAD_REQUEST,
+ MediaType.PLAIN_TEXT,
+ "size and qps can't be both set");
+ }
+ log.info("Generate {} mock segments, qps: {}, template: {}", size, qps, request);
+
+ final IntConsumer generator = unused -> {
+ final Segment segment = request.next();
+ log.debug("Generating segment: {}", segment);
+ sourceReceiver.receive(segment);
+ };
+ final String requestId = UUID.randomUUID().toString();
+ final Future<?> future;
+ if (size > 0) {
+ future = eventLoopGroup.submit(() -> IntStream
+ .range(0, size)
+ .forEach(generator));
+ } else {
+ future = eventLoopGroup.scheduleAtFixedRate(() -> IntStream
+ .range(0, qps)
+ .forEach(generator), 0, 1, TimeUnit.SECONDS);
+ }
+
+ futures.put(requestId, future);
+
+ future.addListener(f -> {
+ if (f.isDone()) {
+ futures.remove(requestId);
+ log.info("Generate mock segments finished: {}, requestId: {}", f.isSuccess(),
+ requestId);
+ }
+ if (f.cause() != null) {
+ log.error(f.cause().getMessage(), f.cause());
+ }
+ });
+
+ return HttpResponse.of(MediaType.PLAIN_TEXT, requestId);
+ }
+
+ @Delete("/mock-data/segments/tasks")
+ public HttpResponse cancelRequest(@Param("requestId") String requestId) {
+ final Future<?> future = futures.get(requestId);
+ if (future == null) {
+ return HttpResponse.of(
+ HttpStatus.NOT_FOUND,
+ MediaType.PLAIN_TEXT_UTF_8,
+ "No such request: %s", requestId);
+ }
+ log.info("Cancelling request: {}", requestId);
+ future.cancel(true);
+ return HttpResponse.of(HttpStatus.OK);
+ }
+
+ @ProducesJson
+ @Get("/mock-data/segments/tasks")
Review comment:
> We should add an endpoint to expose the progress of a request. It's about how many tasks are completed, how many errors, etc.
If you specify the `size`, there might be a progress, but if you specify `qps`, there is no progress, it generates the data endlessly until you cancel the task
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r834066978
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/Generator.java
##########
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import com.fasterxml.jackson.annotation.JsonSubTypes;
+import com.fasterxml.jackson.annotation.JsonSubTypes.Type;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+
+@JsonTypeInfo(
+ use = JsonTypeInfo.Id.NAME,
+ include = JsonTypeInfo.As.PROPERTY,
+ property = "type")
+@JsonSubTypes({
+ @Type(value = StringGenerator.class, name = "randomString"),
Review comment:
*BadImport:* Importing nested classes/static methods/static fields with commonly-used names can make code harder to read, because it may not be clear from the context exactly which type is being referred to. Qualifying the name with that of the containing class can make the code clearer. Here we recommend using qualified class: JsonSubTypes. [(details)](https://errorprone.info/bugpattern/BadImport)
```suggestion
@JsonSubTypes.Type(value = StringGenerator.class, name = "randomString"),
```
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r834067159
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/SequenceGenerator.java
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.ThreadLocalRandom;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.google.common.base.Preconditions;
+import lombok.Data;
+import lombok.experimental.Accessors;
+
+@JsonDeserialize(builder = SequenceGenerator.Builder.class)
+public final class SequenceGenerator implements Generator<Long> {
+ private final boolean limitedDomain;
+ private final long min;
+ private final long max;
+ private final long step;
+ private final Integer jitter;
+ private final Integer domainSize;
+ private final ThreadLocalRandom random = ThreadLocalRandom.current();
+ private final Set<Long> domain = new HashSet<>();
+ private volatile long last = 0;
+
+ public SequenceGenerator(Builder builder) {
+ min = builder.min;
+ max = builder.max;
+ step = builder.step;
+ jitter = builder.jitter;
+ domainSize = builder.domainSize;
+ limitedDomain = builder.domainSize != null && builder.domainSize > 0;
+
+ reset();
+ }
+
+ @Override
+ public Long next() {
+ if (!limitedDomain) {
+ return next0();
+ }
+ return domain
+ .stream()
+ .skip(random.nextInt(domain.size()))
+ .findFirst()
+ .orElseThrow(() -> new RuntimeException("Should never happen"));
+ }
+
+ private synchronized long next0() {
+ long next = last + step;
+
+ if (jitter != null) {
+ int j = random.nextInt(jitter);
+ next += random.nextBoolean() ? j : -j;
+ }
+
+ if (next > max) {
+ return max;
+ }
+ if (next < min) {
+ return min;
+ }
+
+ return last = next;
+ }
+
+ @Override
+ public void reset() {
+ last = 0;
+ if (limitedDomain) {
+ domain.clear();
+ while (domain.size() < domainSize) {
+ domain.add(next0());
+ }
+ }
+ }
+
+ @Override
+ public String toString() {
+ return String.valueOf(next());
+ }
+
+ @Data
Review comment:
*MissingSummary:* A summary fragment is required; consider using the value of the @return block as a summary fragment instead. [(details)](https://google.github.io/styleguide/javaguide.html#s7.2-summary-fragment)
```suggestion
Returns {@code this}.
```
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] hanahmily commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832808976
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
+ @RequestObject SegmentRequest request) {
+
+ if (size > 0 && qps > 0) {
+ return HttpResponse.of(
+ HttpStatus.BAD_REQUEST,
+ MediaType.PLAIN_TEXT,
+ "size and qps can't be both set");
+ }
+ log.info("Generate {} mock segments, qps: {}, template: {}", size, qps, request);
+
+ final IntConsumer generator = unused -> {
+ final Segment segment = request.next();
+ log.debug("Generating segment: {}", segment);
+ sourceReceiver.receive(segment);
+ };
+ final String requestId = UUID.randomUUID().toString();
+ final Future<?> future;
+ if (size > 0) {
+ future = eventLoopGroup.submit(() -> IntStream
+ .range(0, size)
+ .forEach(generator));
+ } else {
+ future = eventLoopGroup.scheduleAtFixedRate(() -> IntStream
+ .range(0, qps)
+ .forEach(generator), 0, 1, TimeUnit.SECONDS);
+ }
+
+ futures.put(requestId, future);
+
+ future.addListener(f -> {
+ if (f.isDone()) {
+ futures.remove(requestId);
+ log.info("Generate mock segments finished: {}, requestId: {}", f.isSuccess(),
+ requestId);
+ }
+ if (f.cause() != null) {
+ log.error(f.cause().getMessage(), f.cause());
+ }
+ });
+
+ return HttpResponse.of(MediaType.PLAIN_TEXT, requestId);
+ }
+
+ @Delete("/mock-data/segments/tasks")
+ public HttpResponse cancelRequest(@Param("requestId") String requestId) {
+ final Future<?> future = futures.get(requestId);
+ if (future == null) {
+ return HttpResponse.of(
+ HttpStatus.NOT_FOUND,
+ MediaType.PLAIN_TEXT_UTF_8,
+ "No such request: %s", requestId);
+ }
+ log.info("Cancelling request: {}", requestId);
+ future.cancel(true);
+ return HttpResponse.of(HttpStatus.OK);
+ }
+
+ @ProducesJson
+ @Get("/mock-data/segments/tasks")
Review comment:
The response could be
```json
{
"total":"int"
"running":"int"
"errors": "int"
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r834898179
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/ListGenerator.java
##########
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import lombok.Data;
+
+@JsonDeserialize(builder = ListGenerator.Builder.class)
+public final class ListGenerator<T> implements Generator<List<T>> {
+ private final T item;
+ private final int size;
+
+ public ListGenerator(Builder<T> builder) {
+ item = builder.item;
+ size = builder.size;
+ }
+
+ @Override
+ public List<T> next() {
+ return IntStream
+ .range(0, size)
+ .mapToObj($ -> item)
+ .collect(Collectors.toList());
+ }
+
+ @Override
+ public void reset() {
+ ((Generator<?>) item).reset();
+ }
+
+ @Data
Review comment:
I've recorded this as ignored for this pull request. If you change your mind, just comment `@sonatype-lift unignore`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] hanahmily commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832798384
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
+ @RequestObject SegmentRequest request) {
+
+ if (size > 0 && qps > 0) {
+ return HttpResponse.of(
+ HttpStatus.BAD_REQUEST,
+ MediaType.PLAIN_TEXT,
+ "size and qps can't be both set");
+ }
+ log.info("Generate {} mock segments, qps: {}, template: {}", size, qps, request);
+
+ final IntConsumer generator = unused -> {
+ final Segment segment = request.next();
+ log.debug("Generating segment: {}", segment);
+ sourceReceiver.receive(segment);
+ };
+ final String requestId = UUID.randomUUID().toString();
+ final Future<?> future;
+ if (size > 0) {
+ future = eventLoopGroup.submit(() -> IntStream
+ .range(0, size)
+ .forEach(generator));
+ } else {
+ future = eventLoopGroup.scheduleAtFixedRate(() -> IntStream
+ .range(0, qps)
+ .forEach(generator), 0, 1, TimeUnit.SECONDS);
+ }
+
+ futures.put(requestId, future);
+
+ future.addListener(f -> {
+ if (f.isDone()) {
+ futures.remove(requestId);
+ log.info("Generate mock segments finished: {}, requestId: {}", f.isSuccess(),
+ requestId);
+ }
+ if (f.cause() != null) {
+ log.error(f.cause().getMessage(), f.cause());
+ }
+ });
+
+ return HttpResponse.of(MediaType.PLAIN_TEXT, requestId);
+ }
+
+ @Delete("/mock-data/segments/tasks")
+ public HttpResponse cancelRequest(@Param("requestId") String requestId) {
Review comment:
Can we clear all requests more than sending several requests to this endpoint?
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
Review comment:
We need a time window param here. The generated segments should go into this window.
If we pass the time window, the size could be 0.
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
+ @RequestObject SegmentRequest request) {
+
+ if (size > 0 && qps > 0) {
+ return HttpResponse.of(
+ HttpStatus.BAD_REQUEST,
+ MediaType.PLAIN_TEXT,
+ "size and qps can't be both set");
+ }
+ log.info("Generate {} mock segments, qps: {}, template: {}", size, qps, request);
+
+ final IntConsumer generator = unused -> {
+ final Segment segment = request.next();
+ log.debug("Generating segment: {}", segment);
+ sourceReceiver.receive(segment);
+ };
+ final String requestId = UUID.randomUUID().toString();
+ final Future<?> future;
+ if (size > 0) {
+ future = eventLoopGroup.submit(() -> IntStream
+ .range(0, size)
+ .forEach(generator));
+ } else {
+ future = eventLoopGroup.scheduleAtFixedRate(() -> IntStream
+ .range(0, qps)
+ .forEach(generator), 0, 1, TimeUnit.SECONDS);
+ }
+
+ futures.put(requestId, future);
+
+ future.addListener(f -> {
+ if (f.isDone()) {
+ futures.remove(requestId);
+ log.info("Generate mock segments finished: {}, requestId: {}", f.isSuccess(),
+ requestId);
+ }
+ if (f.cause() != null) {
+ log.error(f.cause().getMessage(), f.cause());
+ }
+ });
+
+ return HttpResponse.of(MediaType.PLAIN_TEXT, requestId);
+ }
+
+ @Delete("/mock-data/segments/tasks")
+ public HttpResponse cancelRequest(@Param("requestId") String requestId) {
+ final Future<?> future = futures.get(requestId);
+ if (future == null) {
+ return HttpResponse.of(
+ HttpStatus.NOT_FOUND,
+ MediaType.PLAIN_TEXT_UTF_8,
+ "No such request: %s", requestId);
+ }
+ log.info("Cancelling request: {}", requestId);
+ future.cancel(true);
+ return HttpResponse.of(HttpStatus.OK);
+ }
+
+ @ProducesJson
+ @Get("/mock-data/segments/tasks")
Review comment:
We should add an endpoint to expose the progress of a request. It's about how many tasks are completed, how many errors, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1075134853
@hanahmily @lujiajing1126 please check whether ^^^ that will be sufficient for our BanyanDB MVP tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1076313160
> Is it possible for us to specify how many spans will share the same trace_id since this is specially treated as a global index in BanyanDB.
I think for BanyanDB, we care more about how many segments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r834898165
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/ListGenerator.java
##########
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import lombok.Data;
+
+@JsonDeserialize(builder = ListGenerator.Builder.class)
+public final class ListGenerator<T> implements Generator<List<T>> {
+ private final T item;
+ private final int size;
+
+ public ListGenerator(Builder<T> builder) {
+ item = builder.item;
+ size = builder.size;
+ }
+
+ @Override
+ public List<T> next() {
+ return IntStream
+ .range(0, size)
+ .mapToObj($ -> item)
+ .collect(Collectors.toList());
+ }
+
+ @Override
+ public void reset() {
+ ((Generator<?>) item).reset();
+ }
+
+ @Data
Review comment:
@sonatype-lift ignore
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1075134164
In this PR I only implement the data generators for Segment and Log, not metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1075155842
Questions
1. Is the ID generated as a random string only? If so, we may face errors in queries as they are not well encoded. How about generating names?
2. Let's add `UUID` and `Java-Trace-ID` generator for trace ID generating, which could make the ID more reasonable.
3. I can see `domainSize` in endpoint and tags, should we add(or added) for service and instance too?
4. `error` should have a error stack attachment, which should be generated inside span's logs.
5. The number of spans(with tags and logs) seems missing.
6. Recommend to write a README.md for this testing-mode tar, including how to use, and examples of traces generated.
> In this PR I only implement the data generators for Segment and Log
Could you make the log command clear?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832801822
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
Review comment:
> We need a time window param here. The generated segments should go into this window.
> If we pass the time window, the size could be 0.
You should set the time window by specifying the time range (`min`, `max`) in the template. I'm not using the current timestamp as the segment/logs timestamp.
```
"startTime": {
"type": "randomInt",
"min": "1647868381000",
"max": "1647878402000"
},
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1075129299
Some examples
<details>
<summary>Submit a task to generate segments</summary>
```shell
### Submit a task to generate mock segment data, 100 segments per seconds
POST http://localhost:12800/mock-data/segments/tasks?qps=100
Content-Type: application/json
{
"traceId": {
"type": "randomString",
"length": "20",
"letters": true,
"numbers": true
},
"serviceInstanceId": {
"type": "randomString",
"length": "10",
"letters": true,
"numbers": true
},
"dataBinary": {
"type": "randomString",
"length": "10",
"prefix": "test",
"letters": true,
"numbers": true
},
"segmentId": {
"type": "randomString",
"length": "20",
"letters": true,
"numbers": true
},
"serviceId": {
"type": "randomString",
"length": "10",
"letters": true,
"numbers": true
},
"endpointId": {
"type": "randomString",
"length": "10",
"prefix": "test_",
"letters": true,
"numbers": true,
"domainSize": 10
},
"startTime": {
"type": "randomInt",
"min": "1647868381000",
"max": "1647878402000",
"domainSize": 10
},
"latency": {
"type": "randomInt",
"min": 100,
"max": 1000
},
"error": {
"type": "randomInt",
"min": "0",
"max": "1"
},
"tags": {
"type": "randomList",
"size": 5,
"item": {
"key": {
"type": "randomString",
"length": "10",
"prefix": "test",
"letters": true,
"numbers": true,
"domainSize": 10
},
"value": {
"type": "randomString",
"length": "10",
"prefix": "test",
"letters": true,
"numbers": true
}
}
}
}
```
</details>
<details>
<summary>Cancel the segment generating task</summary>
```shell
### Cancel the task, with the requestId return from the previous request
DELETE http://localhost:12800/mock-data/segments/tasks?requestId=8bbdaee6-2193-405d-bc0c-a650cd17597e
```
</details>
<details>
<summary>List all segment tasks</summary>
```shell
### List all tasks that are now running
GET http://localhost:12800/mock-data/segments/tasks
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832804395
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
+ @RequestObject SegmentRequest request) {
+
+ if (size > 0 && qps > 0) {
+ return HttpResponse.of(
+ HttpStatus.BAD_REQUEST,
+ MediaType.PLAIN_TEXT,
+ "size and qps can't be both set");
+ }
+ log.info("Generate {} mock segments, qps: {}, template: {}", size, qps, request);
+
+ final IntConsumer generator = unused -> {
+ final Segment segment = request.next();
+ log.debug("Generating segment: {}", segment);
+ sourceReceiver.receive(segment);
+ };
+ final String requestId = UUID.randomUUID().toString();
+ final Future<?> future;
+ if (size > 0) {
+ future = eventLoopGroup.submit(() -> IntStream
+ .range(0, size)
+ .forEach(generator));
+ } else {
+ future = eventLoopGroup.scheduleAtFixedRate(() -> IntStream
+ .range(0, qps)
+ .forEach(generator), 0, 1, TimeUnit.SECONDS);
+ }
+
+ futures.put(requestId, future);
+
+ future.addListener(f -> {
+ if (f.isDone()) {
+ futures.remove(requestId);
+ log.info("Generate mock segments finished: {}, requestId: {}", f.isSuccess(),
+ requestId);
+ }
+ if (f.cause() != null) {
+ log.error(f.cause().getMessage(), f.cause());
+ }
+ });
+
+ return HttpResponse.of(MediaType.PLAIN_TEXT, requestId);
+ }
+
+ @Delete("/mock-data/segments/tasks")
+ public HttpResponse cancelRequest(@Param("requestId") String requestId) {
+ final Future<?> future = futures.get(requestId);
+ if (future == null) {
+ return HttpResponse.of(
+ HttpStatus.NOT_FOUND,
+ MediaType.PLAIN_TEXT_UTF_8,
+ "No such request: %s", requestId);
+ }
+ log.info("Cancelling request: {}", requestId);
+ future.cancel(true);
+ return HttpResponse.of(HttpStatus.OK);
+ }
+
+ @ProducesJson
+ @Get("/mock-data/segments/tasks")
Review comment:
endless could have a progress for now, I think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] hanahmily commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832806922
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
Review comment:
I noticed the type is `randomInt`. Does it mean the timestamps are random data points between min and max? Can you support generating monotonic time? The monotonically increasing timestamp is very important to banyandb's data model.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832173203
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
Review comment:
*SameNameButDifferent:* The name `@Slf4j` refers to [java.lang.SuppressWarnings, lombok.Generated, org.slf4j.Logger, org.slf4j.LoggerFactory] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentRequest.java
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+import org.apache.skywalking.generator.Generator;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import lombok.Data;
+
+@Data
+@JsonTypeInfo(use = JsonTypeInfo.Id.NONE)
+public final class SegmentRequest implements Generator<Segment> {
+ private Generator<String> segmentId;
Review comment:
*SameNameButDifferent:* The name `segmentId;` refers to [java.lang.SuppressWarnings, lombok.Generated, org.apache.skywalking.generator.Generator, java.lang.String] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentRequest.java
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+import org.apache.skywalking.generator.Generator;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import lombok.Data;
+
+@Data
+@JsonTypeInfo(use = JsonTypeInfo.Id.NONE)
+public final class SegmentRequest implements Generator<Segment> {
+ private Generator<String> segmentId;
+ private Generator<String> traceId;
+ private Generator<String> serviceId;
Review comment:
*SameNameButDifferent:* The name `serviceId;` refers to [java.lang.SuppressWarnings, lombok.Generated, org.apache.skywalking.generator.Generator, java.lang.String] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentRequest.java
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+import org.apache.skywalking.generator.Generator;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import lombok.Data;
+
+@Data
+@JsonTypeInfo(use = JsonTypeInfo.Id.NONE)
+public final class SegmentRequest implements Generator<Segment> {
+ private Generator<String> segmentId;
+ private Generator<String> traceId;
Review comment:
*SameNameButDifferent:* The name `traceId;` refers to [java.lang.SuppressWarnings, lombok.Generated, org.apache.skywalking.generator.Generator, java.lang.String] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentRequest.java
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+import org.apache.skywalking.generator.Generator;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import lombok.Data;
+
+@Data
Review comment:
*SameNameButDifferent:* The name `@Data` refers to [java.lang.SuppressWarnings, lombok.Generated, org.apache.skywalking.generator.Generator, java.lang.String, java.lang.Long, java.util.List, org.apache.skywalking.restapi.TagGenerator, java.lang.Override, java.lang.Object] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/pom.xml
##########
@@ -0,0 +1,43 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one
+ ~ or more contributor license agreements. See the NOTICE file
+ ~ distributed with this work for additional information
+ ~ regarding copyright ownership. The ASF licenses this file
+ ~ to you under the Apache License, Version 2.0 (the
+ ~ "License"); you may not use this file except in compliance
+ ~ with the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing,
+ ~ software distributed under the License is distributed on an
+ ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ ~ KIND, either express or implied. See the License for the
+ ~ specific language governing permissions and limitations
+ ~ under the License.
+-->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <artifactId>server-tools</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>9.0.0-SNAPSHOT</version>
+ </parent>
+ <artifactId>data-generator</artifactId>
+ <name>data-generator</name>
+ <description>
+ Module to generate data for testing.
+ TODO: exclude this from the release tar
+ </description>
+
+ <dependencies>
+ <dependency>
Review comment:
*Severe OSS Vulnerability:*
### pkg:maven/org.apache.skywalking/server-starter@9.0.0-SNAPSHOT
0 Critical, 1 Severe, 1 Moderate, 0 Unknown vulnerabilities have been found across 2 dependencies
<details>
<summary><b>Components</b></summary><br/>
<ul>
<details>
<summary><b>pkg:maven/commons-io/commons-io@2.6</b></summary>
<ul>
<details>
<summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [CVE-2021-29425] In Apache Commons IO before 2.7, When invoking the method FileNameUtils.normaliz...
> In Apache Commons IO before 2.7, When invoking the method FileNameUtils.normalize with an improper input string, like "//../foo", or "\\..\foo", the result would be the same value, thus possibly providing access to files in the parent directory, but not further above (thus "limited" path traversal), if the calling code would use the result to construct a path value.
>
> **CVSS Score:** 5.3
>
> **CVSS Vector:** CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:N
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/com.google.guava/guava@28.1-jre</b></summary>
<ul>
<details>
<summary><b>MODERATE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [CVE-2020-8908] A temp directory creation vulnerability exists in all versions of Guava, allowin...
> A temp directory creation vulnerability exists in all versions of Guava, allowing an attacker with access to the machine to potentially access data in a temporary directory created by the Guava API com.google.common.io.Files.createTempDir(). By default, on unix-like systems, the created directory is world-readable (readable by an attacker with access to the system). The method in question has been marked @Deprecated in versions 30.0 and later and should not be used. For Android developers, we recommend choosing a temporary directory API provided by Android, such as context.getCacheDir(). For other Java developers, we recommend migrating to the Java 7 API java.nio.file.Files.createTempDirectory() which explicitly configures permissions of 700, or configuring the Java runtime's java.io.tmpdir system property to point to a location whose permissions are appropriately configured.
>
> **CVSS Score:** 3.3
>
> **CVSS Vector:** CVSS:3.0/AV:L/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:N
</ul>
</details>
</ul>
</details>
</ul>
</details>
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1075838743
Support query is necessary. Because one reason to build this is to verify the query performance in large scale of data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1075837757
@wu-sheng do we need the mock data to be that real and do we need to query the data through OAP? I even wanted to remove the query module from the new starter...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832802347
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
Review comment:
I think documenting this should be done as it is important for time series telemetry data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832810057
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
Review comment:
monotonic as increasing should be more real. But ramdon should still be used for second(same day, hour, minute),because the timestamp of source services are never 100% sync, and segments finished unmonotonic slightly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] hanahmily commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832808198
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
+ @RequestObject SegmentRequest request) {
+
+ if (size > 0 && qps > 0) {
+ return HttpResponse.of(
+ HttpStatus.BAD_REQUEST,
+ MediaType.PLAIN_TEXT,
+ "size and qps can't be both set");
+ }
+ log.info("Generate {} mock segments, qps: {}, template: {}", size, qps, request);
+
+ final IntConsumer generator = unused -> {
+ final Segment segment = request.next();
+ log.debug("Generating segment: {}", segment);
+ sourceReceiver.receive(segment);
+ };
+ final String requestId = UUID.randomUUID().toString();
+ final Future<?> future;
+ if (size > 0) {
+ future = eventLoopGroup.submit(() -> IntStream
+ .range(0, size)
+ .forEach(generator));
+ } else {
+ future = eventLoopGroup.scheduleAtFixedRate(() -> IntStream
+ .range(0, qps)
+ .forEach(generator), 0, 1, TimeUnit.SECONDS);
+ }
+
+ futures.put(requestId, future);
+
+ future.addListener(f -> {
+ if (f.isDone()) {
+ futures.remove(requestId);
+ log.info("Generate mock segments finished: {}, requestId: {}", f.isSuccess(),
+ requestId);
+ }
+ if (f.cause() != null) {
+ log.error(f.cause().getMessage(), f.cause());
+ }
+ });
+
+ return HttpResponse.of(MediaType.PLAIN_TEXT, requestId);
+ }
+
+ @Delete("/mock-data/segments/tasks")
+ public HttpResponse cancelRequest(@Param("requestId") String requestId) {
+ final Future<?> future = futures.get(requestId);
+ if (future == null) {
+ return HttpResponse.of(
+ HttpStatus.NOT_FOUND,
+ MediaType.PLAIN_TEXT_UTF_8,
+ "No such request: %s", requestId);
+ }
+ log.info("Cancelling request: {}", requestId);
+ future.cancel(true);
+ return HttpResponse.of(HttpStatus.OK);
+ }
+
+ @ProducesJson
+ @Get("/mock-data/segments/tasks")
Review comment:
`qps` model could show a success/error ratio instead of a simple progress bar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] lujiajing1126 commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1076322475
> > Is it possible for us to specify how many spans will share the same trace_id since this is specially treated as a global index in BanyanDB.
>
> I think for BanyanDB, we care more about how many segments.
Oh, right, a typo. I meant `Segment` since all `Span`s are encoded as binary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832172925
##########
File path: oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/Segment.java
##########
@@ -22,10 +22,12 @@
import java.util.List;
import lombok.Getter;
import lombok.Setter;
+import lombok.ToString;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SEGMENT;
+@ToString
Review comment:
*SameNameButDifferent:* The name `@ToString` refers to [java.lang.Override, java.lang.SuppressWarnings, lombok.Generated, java.lang.String, java.util.Arrays] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r832804870
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.function.IntConsumer;
+import java.util.stream.IntStream;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.util.EventLoopGroups;
+import com.linecorp.armeria.server.annotation.Default;
+import com.linecorp.armeria.server.annotation.Delete;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.ProducesJson;
+import com.linecorp.armeria.server.annotation.RequestObject;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.source.Segment;
+import org.apache.skywalking.oap.server.core.source.SourceReceiver;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import io.netty.channel.EventLoopGroup;
+import io.netty.util.concurrent.Future;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class SegmentGeneratorHandler {
+ private final SourceReceiver sourceReceiver;
+ private final Map<String, Future<?>> futures = new ConcurrentHashMap<>();
+ private final EventLoopGroup eventLoopGroup = EventLoopGroups.newEventLoopGroup(10);
+
+ public SegmentGeneratorHandler(ModuleManager manager) {
+ sourceReceiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+ }
+
+ @Post("/mock-data/segments/tasks")
+ public HttpResponse generateMockSegments(
+ @Default("0") @Param("size") int size,
+ @Default("0") @Param("qps") int qps,
+ @RequestObject SegmentRequest request) {
+
+ if (size > 0 && qps > 0) {
+ return HttpResponse.of(
+ HttpStatus.BAD_REQUEST,
+ MediaType.PLAIN_TEXT,
+ "size and qps can't be both set");
+ }
+ log.info("Generate {} mock segments, qps: {}, template: {}", size, qps, request);
+
+ final IntConsumer generator = unused -> {
+ final Segment segment = request.next();
+ log.debug("Generating segment: {}", segment);
+ sourceReceiver.receive(segment);
+ };
+ final String requestId = UUID.randomUUID().toString();
+ final Future<?> future;
+ if (size > 0) {
+ future = eventLoopGroup.submit(() -> IntStream
+ .range(0, size)
+ .forEach(generator));
+ } else {
+ future = eventLoopGroup.scheduleAtFixedRate(() -> IntStream
+ .range(0, qps)
+ .forEach(generator), 0, 1, TimeUnit.SECONDS);
+ }
+
+ futures.put(requestId, future);
+
+ future.addListener(f -> {
+ if (f.isDone()) {
+ futures.remove(requestId);
+ log.info("Generate mock segments finished: {}, requestId: {}", f.isSuccess(),
+ requestId);
+ }
+ if (f.cause() != null) {
+ log.error(f.cause().getMessage(), f.cause());
+ }
+ });
+
+ return HttpResponse.of(MediaType.PLAIN_TEXT, requestId);
+ }
+
+ @Delete("/mock-data/segments/tasks")
+ public HttpResponse cancelRequest(@Param("requestId") String requestId) {
+ final Future<?> future = futures.get(requestId);
+ if (future == null) {
+ return HttpResponse.of(
+ HttpStatus.NOT_FOUND,
+ MediaType.PLAIN_TEXT_UTF_8,
+ "No such request: %s", requestId);
+ }
+ log.info("Cancelling request: {}", requestId);
+ future.cancel(true);
+ return HttpResponse.of(HttpStatus.OK);
+ }
+
+ @ProducesJson
+ @Get("/mock-data/segments/tasks")
Review comment:
Progress is just a statistic of generated data, not has to be a completion rate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a change in pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on a change in pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#discussion_r834068057
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/ListGenerator.java
##########
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import lombok.Data;
+
+@JsonDeserialize(builder = ListGenerator.Builder.class)
+public final class ListGenerator<T> implements Generator<List<T>> {
+ private final T item;
+ private final int size;
+
+ public ListGenerator(Builder<T> builder) {
+ item = builder.item;
+ size = builder.size;
+ }
+
+ @Override
+ public List<T> next() {
+ return IntStream
+ .range(0, size)
+ .mapToObj($ -> item)
+ .collect(Collectors.toList());
+ }
+
+ @Override
+ public void reset() {
+ ((Generator<?>) item).reset();
+ }
+
+ @Data
+ public static class Builder<T> {
+ private int size;
+ private T item;
Review comment:
*SameNameButDifferent:* The name `item;` refers to [java.lang.SuppressWarnings, lombok.Generated] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/ListGenerator.java
##########
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import lombok.Data;
+
+@JsonDeserialize(builder = ListGenerator.Builder.class)
+public final class ListGenerator<T> implements Generator<List<T>> {
+ private final T item;
+ private final int size;
+
+ public ListGenerator(Builder<T> builder) {
+ item = builder.item;
+ size = builder.size;
+ }
+
+ @Override
+ public List<T> next() {
+ return IntStream
+ .range(0, size)
+ .mapToObj($ -> item)
+ .collect(Collectors.toList());
+ }
+
+ @Override
+ public void reset() {
+ ((Generator<?>) item).reset();
+ }
+
+ @Data
+ public static class Builder<T> {
+ private int size;
Review comment:
*SameNameButDifferent:* The name `size;` refers to [java.lang.SuppressWarnings, lombok.Generated] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/SequenceGenerator.java
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.ThreadLocalRandom;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.google.common.base.Preconditions;
+import lombok.Data;
+import lombok.experimental.Accessors;
+
+@JsonDeserialize(builder = SequenceGenerator.Builder.class)
+public final class SequenceGenerator implements Generator<Long> {
+ private final boolean limitedDomain;
+ private final long min;
+ private final long max;
+ private final long step;
+ private final Integer jitter;
+ private final Integer domainSize;
+ private final ThreadLocalRandom random = ThreadLocalRandom.current();
+ private final Set<Long> domain = new HashSet<>();
+ private volatile long last = 0;
+
+ public SequenceGenerator(Builder builder) {
+ min = builder.min;
+ max = builder.max;
+ step = builder.step;
+ jitter = builder.jitter;
+ domainSize = builder.domainSize;
+ limitedDomain = builder.domainSize != null && builder.domainSize > 0;
+
+ reset();
+ }
+
+ @Override
+ public Long next() {
+ if (!limitedDomain) {
+ return next0();
+ }
+ return domain
+ .stream()
+ .skip(random.nextInt(domain.size()))
+ .findFirst()
+ .orElseThrow(() -> new RuntimeException("Should never happen"));
+ }
+
+ private synchronized long next0() {
+ long next = last + step;
+
+ if (jitter != null) {
+ int j = random.nextInt(jitter);
+ next += random.nextBoolean() ? j : -j;
+ }
+
+ if (next > max) {
+ return max;
+ }
+ if (next < min) {
+ return min;
+ }
+
+ return last = next;
+ }
+
+ @Override
+ public void reset() {
+ last = 0;
+ if (limitedDomain) {
+ domain.clear();
+ while (domain.size() < domainSize) {
+ domain.add(next0());
+ }
+ }
+ }
+
+ @Override
+ public String toString() {
+ return String.valueOf(next());
+ }
+
+ @Data
+ @Accessors(chain = true)
+ public static class Builder {
+ private long min = Long.MIN_VALUE;
+ private long max = Long.MAX_VALUE;
+ private long step = 1;
+ private Integer domainSize;
Review comment:
*SameNameButDifferent:* The name `domainSize;` refers to [java.lang.SuppressWarnings, lombok.Generated, java.lang.Integer] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/ListGenerator.java
##########
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import lombok.Data;
+
+@JsonDeserialize(builder = ListGenerator.Builder.class)
+public final class ListGenerator<T> implements Generator<List<T>> {
+ private final T item;
+ private final int size;
+
+ public ListGenerator(Builder<T> builder) {
+ item = builder.item;
+ size = builder.size;
+ }
+
+ @Override
+ public List<T> next() {
+ return IntStream
+ .range(0, size)
+ .mapToObj($ -> item)
+ .collect(Collectors.toList());
+ }
+
+ @Override
+ public void reset() {
+ ((Generator<?>) item).reset();
+ }
+
+ @Data
Review comment:
*SameNameButDifferent:* The name `@Data` refers to [java.lang.SuppressWarnings, lombok.Generated, java.lang.Override, java.lang.Object, org.apache.skywalking.generator.ListGenerator.Builder, java.lang.String] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
```suggestion
ListGenerator.@Data
```
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/SequenceGenerator.java
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.ThreadLocalRandom;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.google.common.base.Preconditions;
+import lombok.Data;
+import lombok.experimental.Accessors;
+
+@JsonDeserialize(builder = SequenceGenerator.Builder.class)
+public final class SequenceGenerator implements Generator<Long> {
+ private final boolean limitedDomain;
+ private final long min;
+ private final long max;
+ private final long step;
+ private final Integer jitter;
+ private final Integer domainSize;
+ private final ThreadLocalRandom random = ThreadLocalRandom.current();
+ private final Set<Long> domain = new HashSet<>();
+ private volatile long last = 0;
+
+ public SequenceGenerator(Builder builder) {
+ min = builder.min;
+ max = builder.max;
+ step = builder.step;
+ jitter = builder.jitter;
+ domainSize = builder.domainSize;
+ limitedDomain = builder.domainSize != null && builder.domainSize > 0;
+
+ reset();
+ }
+
+ @Override
+ public Long next() {
+ if (!limitedDomain) {
+ return next0();
+ }
+ return domain
+ .stream()
+ .skip(random.nextInt(domain.size()))
+ .findFirst()
+ .orElseThrow(() -> new RuntimeException("Should never happen"));
+ }
+
+ private synchronized long next0() {
+ long next = last + step;
+
+ if (jitter != null) {
+ int j = random.nextInt(jitter);
+ next += random.nextBoolean() ? j : -j;
+ }
+
+ if (next > max) {
+ return max;
+ }
+ if (next < min) {
+ return min;
+ }
+
+ return last = next;
+ }
+
+ @Override
+ public void reset() {
+ last = 0;
+ if (limitedDomain) {
+ domain.clear();
+ while (domain.size() < domainSize) {
+ domain.add(next0());
+ }
+ }
+ }
+
+ @Override
+ public String toString() {
+ return String.valueOf(next());
+ }
+
+ @Data
+ @Accessors(chain = true)
+ public static class Builder {
+ private long min = Long.MIN_VALUE;
+ private long max = Long.MAX_VALUE;
+ private long step = 1;
+ private Integer domainSize;
+ private Integer jitter;
Review comment:
*SameNameButDifferent:* The name `jitter;` refers to [java.lang.SuppressWarnings, lombok.Generated, java.lang.Integer] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/SequenceGenerator.java
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.ThreadLocalRandom;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.google.common.base.Preconditions;
+import lombok.Data;
+import lombok.experimental.Accessors;
+
+@JsonDeserialize(builder = SequenceGenerator.Builder.class)
+public final class SequenceGenerator implements Generator<Long> {
+ private final boolean limitedDomain;
+ private final long min;
+ private final long max;
+ private final long step;
+ private final Integer jitter;
+ private final Integer domainSize;
+ private final ThreadLocalRandom random = ThreadLocalRandom.current();
+ private final Set<Long> domain = new HashSet<>();
+ private volatile long last = 0;
+
+ public SequenceGenerator(Builder builder) {
+ min = builder.min;
+ max = builder.max;
+ step = builder.step;
+ jitter = builder.jitter;
+ domainSize = builder.domainSize;
+ limitedDomain = builder.domainSize != null && builder.domainSize > 0;
+
+ reset();
+ }
+
+ @Override
+ public Long next() {
+ if (!limitedDomain) {
+ return next0();
+ }
+ return domain
+ .stream()
+ .skip(random.nextInt(domain.size()))
+ .findFirst()
+ .orElseThrow(() -> new RuntimeException("Should never happen"));
+ }
+
+ private synchronized long next0() {
+ long next = last + step;
+
+ if (jitter != null) {
+ int j = random.nextInt(jitter);
+ next += random.nextBoolean() ? j : -j;
+ }
+
+ if (next > max) {
+ return max;
+ }
+ if (next < min) {
+ return min;
+ }
+
+ return last = next;
+ }
+
+ @Override
+ public void reset() {
+ last = 0;
+ if (limitedDomain) {
+ domain.clear();
+ while (domain.size() < domainSize) {
+ domain.add(next0());
+ }
+ }
+ }
+
+ @Override
+ public String toString() {
+ return String.valueOf(next());
+ }
+
+ @Data
+ @Accessors(chain = true)
+ public static class Builder {
+ private long min = Long.MIN_VALUE;
+ private long max = Long.MAX_VALUE;
Review comment:
*SameNameButDifferent:* The name `max = Long.MAX_VALUE;` refers to [java.lang.SuppressWarnings, lombok.Generated] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/SequenceGenerator.java
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.ThreadLocalRandom;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.google.common.base.Preconditions;
+import lombok.Data;
+import lombok.experimental.Accessors;
+
+@JsonDeserialize(builder = SequenceGenerator.Builder.class)
+public final class SequenceGenerator implements Generator<Long> {
+ private final boolean limitedDomain;
+ private final long min;
+ private final long max;
+ private final long step;
+ private final Integer jitter;
+ private final Integer domainSize;
+ private final ThreadLocalRandom random = ThreadLocalRandom.current();
+ private final Set<Long> domain = new HashSet<>();
+ private volatile long last = 0;
+
+ public SequenceGenerator(Builder builder) {
+ min = builder.min;
+ max = builder.max;
+ step = builder.step;
+ jitter = builder.jitter;
+ domainSize = builder.domainSize;
+ limitedDomain = builder.domainSize != null && builder.domainSize > 0;
+
+ reset();
+ }
+
+ @Override
+ public Long next() {
+ if (!limitedDomain) {
+ return next0();
+ }
+ return domain
+ .stream()
+ .skip(random.nextInt(domain.size()))
+ .findFirst()
+ .orElseThrow(() -> new RuntimeException("Should never happen"));
+ }
+
+ private synchronized long next0() {
+ long next = last + step;
+
+ if (jitter != null) {
+ int j = random.nextInt(jitter);
+ next += random.nextBoolean() ? j : -j;
+ }
+
+ if (next > max) {
+ return max;
+ }
+ if (next < min) {
+ return min;
+ }
+
+ return last = next;
+ }
+
+ @Override
+ public void reset() {
+ last = 0;
+ if (limitedDomain) {
+ domain.clear();
+ while (domain.size() < domainSize) {
+ domain.add(next0());
+ }
+ }
+ }
+
+ @Override
+ public String toString() {
+ return String.valueOf(next());
+ }
+
+ @Data
+ @Accessors(chain = true)
+ public static class Builder {
+ private long min = Long.MIN_VALUE;
Review comment:
*SameNameButDifferent:* The name `min = Long.MIN_VALUE;` refers to [java.lang.SuppressWarnings, lombok.Generated] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/SequenceGenerator.java
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.ThreadLocalRandom;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.google.common.base.Preconditions;
+import lombok.Data;
+import lombok.experimental.Accessors;
+
+@JsonDeserialize(builder = SequenceGenerator.Builder.class)
+public final class SequenceGenerator implements Generator<Long> {
+ private final boolean limitedDomain;
+ private final long min;
+ private final long max;
+ private final long step;
+ private final Integer jitter;
+ private final Integer domainSize;
+ private final ThreadLocalRandom random = ThreadLocalRandom.current();
+ private final Set<Long> domain = new HashSet<>();
+ private volatile long last = 0;
+
+ public SequenceGenerator(Builder builder) {
+ min = builder.min;
+ max = builder.max;
+ step = builder.step;
+ jitter = builder.jitter;
+ domainSize = builder.domainSize;
+ limitedDomain = builder.domainSize != null && builder.domainSize > 0;
+
+ reset();
+ }
+
+ @Override
+ public Long next() {
+ if (!limitedDomain) {
+ return next0();
+ }
+ return domain
+ .stream()
+ .skip(random.nextInt(domain.size()))
+ .findFirst()
+ .orElseThrow(() -> new RuntimeException("Should never happen"));
+ }
+
+ private synchronized long next0() {
+ long next = last + step;
+
+ if (jitter != null) {
+ int j = random.nextInt(jitter);
+ next += random.nextBoolean() ? j : -j;
+ }
+
+ if (next > max) {
+ return max;
+ }
+ if (next < min) {
+ return min;
+ }
+
+ return last = next;
+ }
+
+ @Override
+ public void reset() {
+ last = 0;
+ if (limitedDomain) {
+ domain.clear();
+ while (domain.size() < domainSize) {
+ domain.add(next0());
+ }
+ }
+ }
+
+ @Override
+ public String toString() {
+ return String.valueOf(next());
+ }
+
+ @Data
+ @Accessors(chain = true)
+ public static class Builder {
+ private long min = Long.MIN_VALUE;
+ private long max = Long.MAX_VALUE;
+ private long step = 1;
Review comment:
*SameNameButDifferent:* The name `step = 1;` refers to [java.lang.SuppressWarnings, lombok.Generated] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/SequenceGenerator.java
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.generator;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.ThreadLocalRandom;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.google.common.base.Preconditions;
+import lombok.Data;
+import lombok.experimental.Accessors;
+
+@JsonDeserialize(builder = SequenceGenerator.Builder.class)
+public final class SequenceGenerator implements Generator<Long> {
+ private final boolean limitedDomain;
+ private final long min;
+ private final long max;
+ private final long step;
+ private final Integer jitter;
+ private final Integer domainSize;
+ private final ThreadLocalRandom random = ThreadLocalRandom.current();
+ private final Set<Long> domain = new HashSet<>();
+ private volatile long last = 0;
+
+ public SequenceGenerator(Builder builder) {
+ min = builder.min;
+ max = builder.max;
+ step = builder.step;
+ jitter = builder.jitter;
+ domainSize = builder.domainSize;
+ limitedDomain = builder.domainSize != null && builder.domainSize > 0;
+
+ reset();
+ }
+
+ @Override
+ public Long next() {
+ if (!limitedDomain) {
+ return next0();
+ }
+ return domain
+ .stream()
+ .skip(random.nextInt(domain.size()))
+ .findFirst()
+ .orElseThrow(() -> new RuntimeException("Should never happen"));
+ }
+
+ private synchronized long next0() {
+ long next = last + step;
+
+ if (jitter != null) {
+ int j = random.nextInt(jitter);
+ next += random.nextBoolean() ? j : -j;
+ }
+
+ if (next > max) {
+ return max;
+ }
+ if (next < min) {
+ return min;
+ }
+
+ return last = next;
+ }
+
+ @Override
+ public void reset() {
+ last = 0;
+ if (limitedDomain) {
+ domain.clear();
+ while (domain.size() < domainSize) {
+ domain.add(next0());
+ }
+ }
+ }
+
+ @Override
+ public String toString() {
+ return String.valueOf(next());
+ }
+
+ @Data
Review comment:
*SameNameButDifferent:* The name `@Data` refers to [java.lang.SuppressWarnings, lombok.Generated, org.apache.skywalking.generator.SequenceGenerator.Builder, java.lang.Integer, java.lang.Override, java.lang.Object, java.lang.String] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
```suggestion
SequenceGenerator.@Data
```
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/LogRequest.java
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.skywalking.generator.Generator;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.query.type.ContentType;
+import org.apache.skywalking.oap.server.core.source.Log;
+import lombok.Data;
+import lombok.SneakyThrows;
+
+@Data
+@JsonTypeInfo(use = JsonTypeInfo.Id.NONE)
+public final class LogRequest implements Generator<Log> {
+ @JsonIgnore
+ private final ObjectMapper om = new ObjectMapper();
+
+ private Generator<Long> timestamp;
+ private Generator<String> serviceName;
+ private Generator<String> serviceInstanceName;
+ private Generator<String> endpointName;
+ private Generator<String> traceId;
+ private Generator<String> traceSegmentId;
+ private Generator<Long> spanId;
+ private Generator<Long> contentType;
+ private Generator<String> content;
+ private Generator<List<TagGenerator>> tags;
+ private Generator<Boolean> error;
+
+ @SneakyThrows
Review comment:
*SameNameButDifferent:* The name `@SneakyThrows` refers to [java.lang.Throwable, lombok.Lombok] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/LogRequest.java
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.skywalking.restapi;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.skywalking.generator.Generator;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.query.type.ContentType;
+import org.apache.skywalking.oap.server.core.source.Log;
+import lombok.Data;
+import lombok.SneakyThrows;
+
+@Data
Review comment:
*SameNameButDifferent:* The name `@Data` refers to [java.lang.SuppressWarnings, lombok.Generated, org.apache.skywalking.generator.Generator, java.lang.Long, java.lang.String, java.util.List, org.apache.skywalking.restapi.TagGenerator, java.lang.Boolean, java.lang.Override, java.lang.Object] within this file. It may be confusing to have the same name refer to multiple types. Consider qualifying them for clarity. [(details)](https://errorprone.info/bugpattern/SameNameButDifferent)
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
##########
File path: oap-server/server-tools/data-generator/pom.xml
##########
@@ -0,0 +1,43 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one
+ ~ or more contributor license agreements. See the NOTICE file
+ ~ distributed with this work for additional information
+ ~ regarding copyright ownership. The ASF licenses this file
+ ~ to you under the Apache License, Version 2.0 (the
+ ~ "License"); you may not use this file except in compliance
+ ~ with the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing,
+ ~ software distributed under the License is distributed on an
+ ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ ~ KIND, either express or implied. See the License for the
+ ~ specific language governing permissions and limitations
+ ~ under the License.
+-->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <artifactId>server-tools</artifactId>
+ <groupId>org.apache.skywalking</groupId>
+ <version>9.0.0-SNAPSHOT</version>
+ </parent>
+ <artifactId>data-generator</artifactId>
+ <name>data-generator</name>
+ <description>
+ Module to generate data for testing.
+ TODO: exclude this from the release tar
+ </description>
+
+ <dependencies>
+ <dependency>
Review comment:
*Critical OSS Vulnerability:*
### pkg:maven/org.apache.skywalking/server-starter@9.0.0-SNAPSHOT
8 Critical, 8 Severe, 2 Moderate, 0 Unknown vulnerabilities have been found across 16 dependencies
<details>
<summary><b>Components</b></summary><br/>
<ul>
<details>
<summary><b>pkg:maven/commons-dbcp/commons-dbcp@1.4</b></summary>
<ul>
<details>
<summary><b>CRITICAL Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2020-1349] CWE-522: Insufficiently Protected Credentials
> Apache Commons DBCP - Insufficiently Protected Credentials
This weakness occurs when the application transmits or stores authentication credentials and uses an insecure method that is susceptible to unauthorized interception and/or retrieval.
>
> **CVSS Score:** 7.5
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N
>
> **CWE:** CWE-522
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/io.netty/netty-codec-http@4.1.68.Final</b></summary>
<ul>
<details>
<summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [CVE-2021-43797] CWE-444: Inconsistent Interpretation of HTTP Requests ('HTTP Request Smuggling')
> Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients. Netty prior to version 4.1.71.Final skips control chars when they are present at the beginning / end of the header name. It should instead fail fast as these are not allowed by the spec and could lead to HTTP request smuggling. Failing to do the validation might cause netty to "sanitize" header names before it forward these to another remote system when used as proxy. This remote system can't see the invalid usage anymore, and therefore does not do the validation itself. Users should upgrade to version 4.1.71.Final.
>
> **CVSS Score:** 6.5
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:H/A:N
>
> **CWE:** CWE-444
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/org.bouncycastle/bcprov-jdk15on@1.69</b></summary>
<ul>
<details>
<summary><b>MODERATE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2019-0673] CWE-400: Uncontrolled Resource Consumption ('Resource Exhaustion')
> BouncyCastle - Denial of Service (DoS)
The software does not properly restrict the size or amount of resources that are requested or influenced by an actor, which can be used to consume more resources than intended.
>
> **CVSS Score:** 3.7
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:L
>
> **CWE:** CWE-400
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/commons-codec/commons-codec@1.11</b></summary>
<ul>
<details>
<summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2012-0050] CWE-20: Improper Input Validation
> commons-codec - Base32 would decode some invalid Base32 encoded string into arbitrary value
The product does not validate or incorrectly validates input that can affect the control flow or data flow of a program.
>
> **CVSS Score:** 5.3
>
> **CVSS Vector:** CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:L/A:N
>
> **CWE:** CWE-20
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/com.google.code.gson/gson@2.8.6</b></summary>
<ul>
<details>
<summary><b>CRITICAL Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2021-1694] CWE-502: Deserialization of Untrusted Data
> gson - Deserialization of Untrusted Data
The application deserializes untrusted data without sufficiently verifying that the resulting data will be valid.
>
> **CVSS Score:** 7.5
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
>
> **CWE:** CWE-502
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/io.grpc/grpc-core@1.43.2</b></summary>
<ul>
<details>
<summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2021-0818] CWE-404: Improper Resource Shutdown or Release
> grpc-core - Improper Resource Shutdown or Release
The program does not release or incorrectly releases a resource before it is made available for re-use.
>
> **CVSS Score:** 6.5
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H
>
> **CWE:** CWE-404
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/io.netty/netty-handler@4.1.68.Final</b></summary>
<ul>
<details>
<summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2020-0026] CWE-300: Channel Accessible by Non-Endpoint ('Man-in-the-Middle')
> netty-handler - Improper Certificate Validation
The product does not adequately verify the identity of actors at both ends of a communication channel, or does not adequately ensure the integrity of the channel, in a way that allows the channel to be accessed or influenced by an actor that is not an endpoint.
>
> **CVSS Score:** 6.5
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:L/A:N
>
> **CWE:** CWE-300
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/org.bouncycastle/bcprov-ext-jdk15on@1.69</b></summary>
<ul>
<details>
<summary><b>MODERATE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2019-0673] CWE-400: Uncontrolled Resource Consumption ('Resource Exhaustion')
> BouncyCastle - Denial of Service (DoS)
The software does not properly restrict the size or amount of resources that are requested or influenced by an actor, which can be used to consume more resources than intended.
>
> **CVSS Score:** 3.7
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:L
>
> **CWE:** CWE-400
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/com.h2database/h2@2.1.210</b></summary>
<ul>
<details>
<summary><b>CRITICAL Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2020-1324] CWE-94: Improper Control of Generation of Code ('Code Injection')
> h2 - Remote Code Execution (RCE)
The software constructs all or part of a code segment using externally-influenced input from an upstream component, but it does not neutralize or incorrectly neutralizes special elements that could modify the syntax or behavior of the intended code segment.
>
> **CVSS Score:** 8
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:H/PR:H/UI:N/S:C/C:H/I:H/A:H
>
> **CWE:** CWE-94
</ul>
</details>
<details>
<summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [CVE-2018-14335] CWE-59: Improper Link Resolution Before File Access ('Link Following')
> An issue was discovered in H2 1.4.197. Insecure handling of permissions in the backup function allows attackers to read sensitive files (outside of their permissions) via a symlink to a fake database file.
>
> **CVSS Score:** 6.5
>
> **CVSS Vector:** CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N
>
> **CWE:** CWE-59
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.12.2</b></summary>
<ul>
<details>
<summary><b>CRITICAL Vulnerabilities (2)</b></summary><br/>
<ul>
<details>
<summary>CVE-2020-36518</summary>
> #### [CVE-2020-36518] CWE-787: Out-of-bounds Write
> jackson-databind before 2.13.0 allows a Java StackOverflow exception and denial of service via a large depth of nested objects.
>
> **CVSS Score:** 7.5
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
>
> **CWE:** CWE-787
</details>
<details>
<summary>sonatype-2021-4682</summary>
> #### [sonatype-2021-4682] CWE-400: Uncontrolled Resource Consumption ('Resource Exhaustion')
> jackson-databind - Denial of Service (DoS)
The software does not properly restrict the size or amount of resources that are requested or influenced by an actor, which can be used to consume more resources than intended.
>
> **CVSS Score:** 7.5
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
>
> **CWE:** CWE-400
</details>
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/commons-io/commons-io@2.6</b></summary>
<ul>
<details>
<summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2018-0705] CWE-22: Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal')
> commons-io - Path Traversal [CVE-2021-29425]
The software uses external input to construct a pathname that is intended to identify a file or directory that is located underneath a restricted parent directory, but the software does not properly neutralize special elements within the pathname that can cause the pathname to resolve to a location that is outside of the restricted directory.
>
> **CVSS Score:** 5.3
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:N
>
> **CWE:** CWE-22
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/com.google.guava/guava@28.1-jre</b></summary>
<ul>
<details>
<summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2020-0926] CWE-379: Creation of Temporary File in Directory with Incorrect Permissions
> guava - Creation of Temporary File in Directory with Insecure Permissions [CVE-2020-8908]
The software creates a temporary file in a directory whose permissions allow unintended actors to determine the file's existence or otherwise access that file.
>
> **CVSS Score:** 6.2
>
> **CVSS Vector:** CVSS:3.1/AV:L/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N
>
> **CWE:** CWE-379
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/org.freemarker/freemarker@2.3.28</b></summary>
<ul>
<details>
<summary><b>CRITICAL Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [sonatype-2019-0870] CWE-94: Improper Control of Generation of Code ('Code Injection')
> Apache FreeMarker - Code Injection
The software constructs all or part of a code segment using externally-influenced input from an upstream component, but it does not neutralize or incorrectly neutralizes special elements that could modify the syntax or behavior of the intended code segment.
>
> **CVSS Score:** 7.7
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:H/PR:H/UI:N/S:C/C:H/I:H/A:N
>
> **CWE:** CWE-94
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/org.apache.kafka/kafka-clients@2.4.1</b></summary>
<ul>
<details>
<summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [CVE-2021-38153] CWE-203: Information Exposure Through Discrepancy
> Some components in Apache Kafka use `Arrays.equals` to validate a password or key, which is vulnerable to timing attacks that make brute force attacks for such credentials more likely to be successful. Users should upgrade to 2.8.1 or higher, or 3.0.0 or higher where this vulnerability has been fixed. The affected versions include Apache Kafka 2.0.0, 2.0.1, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.6.1, 2.6.2, 2.7.0, 2.7.1, and 2.8.0.
>
> **CVSS Score:** 5.9
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:N
>
> **CWE:** CWE-203
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/com.squareup.okhttp3/okhttp@3.14.9</b></summary>
<ul>
<details>
<summary><b>CRITICAL Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [CVE-2021-0341] CWE-295: Improper Certificate Validation
> In verifyHostName of OkHostnameVerifier.java, there is a possible way to accept a certificate for the wrong domain due to improperly used crypto. This could lead to remote information disclosure with no additional execution privileges needed. User interaction is not needed for exploitation.Product: AndroidVersions: Android-8.1 Android-9 Android-10 Android-11Android ID: A-171980069
>
> **CVSS Score:** 7.5
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N
>
> **CWE:** CWE-295
</ul>
</details>
</ul>
</details>
<details>
<summary><b>pkg:maven/org.postgresql/postgresql@42.3.2</b></summary>
<ul>
<details>
<summary><b>CRITICAL Vulnerabilities (1)</b></summary><br/>
<ul>
> #### [CVE-2022-26520] CWE-20: Improper Input Validation
> ** DISPUTED ** In pgjdbc before 42.3.3, an attacker (who controls the jdbc URL or properties) can call java.util.logging.FileHandler to write to arbitrary files through the loggerFile and loggerLevel connection properties. An example situation is that an attacker could create an executable JSP file under a Tomcat web root. NOTE: the vendor's position is that there is no pgjdbc vulnerability; instead, it is a vulnerability for any application to use the pgjdbc driver with untrusted connection properties.
>
> **CVSS Score:** 9.8
>
> **CVSS Vector:** CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H
>
> **CWE:** CWE-20
</ul>
</details>
</ul>
</details>
</ul>
</details>
(at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 edited a comment on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
kezhenxu94 edited a comment on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1075129299
Some examples
<details>
<summary>Submit a task to generate segments</summary>
```shell
### Submit a task to generate mock segment data, 100 segments per seconds
POST http://localhost:12800/mock-data/segments/tasks?qps=100
Content-Type: application/json
{
"traceId": {
"type": "randomString",
"length": "20",
"letters": true,
"numbers": true
},
"serviceInstanceId": {
"type": "randomString",
"length": "10",
"letters": true,
"numbers": true
},
"dataBinary": {
"type": "randomString",
"length": "10",
"prefix": "test",
"letters": true,
"numbers": true
},
"segmentId": {
"type": "randomString",
"length": "20",
"letters": true,
"numbers": true
},
"serviceId": {
"type": "randomString",
"length": "10",
"letters": true,
"numbers": true
},
"endpointId": {
"type": "randomString",
"length": "10",
"prefix": "test_",
"letters": true,
"numbers": true,
"domainSize": 10 // at most 10 different `endpointId`s will be generated in this task
},
"startTime": {
"type": "randomInt",
"min": "1647868381000",
"max": "1647878402000"
},
"latency": {
"type": "randomInt",
"min": 100,
"max": 1000
},
"error": {
"type": "randomInt",
"min": "0",
"max": "1"
},
"tags": {
"type": "randomList",
"size": 5,
"item": {
"key": {
"type": "randomString",
"length": "10",
"prefix": "test",
"letters": true,
"numbers": true,
"domainSize": 10 // At most 10 different tag key will generated in this task
},
"value": {
"type": "randomString",
"length": "10",
"prefix": "test",
"letters": true,
"numbers": true
}
}
}
}
```
</details>
<details>
<summary>Cancel the segment generating task</summary>
```shell
### Cancel the task, with the requestId return from the previous request
DELETE http://localhost:12800/mock-data/segments/tasks?requestId=8bbdaee6-2193-405d-bc0c-a650cd17597e
```
</details>
<details>
<summary>List all segment tasks</summary>
```shell
### List all tasks that are now running
GET http://localhost:12800/mock-data/segments/tasks
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1075847596
We don't need to make all UI functional working. But GrapgQL should at least be able to work.(considering a lot of logic about data consistent setups in UI and CLI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] lujiajing1126 commented on pull request #8725: Add data-generator module to run OAP in testing mode, generating mock data for testing
Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on pull request #8725:
URL: https://github.com/apache/skywalking/pull/8725#issuecomment-1076226913
Similar Question like Wu Sheng's 5th one:
Is it possible for us to specify how many spans will share the same `trace_id` since this is specially treated as a global index in BanyanDB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org