You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2020/09/08 16:26:37 UTC
[skywalking] branch master updated: Fix the bug #5443 and add
documents for the local E2E remote debugging. (#5457)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new c89e766 Fix the bug #5443 and add documents for the local E2E remote debugging. (#5457)
c89e766 is described below
commit c89e766f0d5b83281e0213becf9831d031081515
Author: CoderGang <32...@qq.com>
AuthorDate: Wed Sep 9 00:26:11 2020 +0800
Fix the bug #5443 and add documents for the local E2E remote debugging. (#5457)
* fix the E2E bug when running locally with multiple `docker-compose.yml` files #5443
* add documents for the for local remote debugging.
Co-authored-by: kezhenxu94 <ke...@apache.org>
---
docs/en/guides/E2E-local-remote-debug.md | 28 ++++++++++++++++++
docs/en/guides/README.md | 3 ++
.../skywalking/e2e/SkyWalkingAnnotations.java | 31 ++++++++++----------
.../skywalking/e2e/docker/DockerComposeFile.java | 6 ++--
.../e2e/docker/DockerComposeFileTest.java | 33 +++++++++++++++-------
.../{docker-compose.yml => docker-compose.one.yml} | 32 ++++++---------------
.../{docker-compose.yml => docker-compose.two.yml} | 0
7 files changed, 82 insertions(+), 51 deletions(-)
diff --git a/docs/en/guides/E2E-local-remote-debug.md b/docs/en/guides/E2E-local-remote-debug.md
new file mode 100644
index 0000000..21810bb
--- /dev/null
+++ b/docs/en/guides/E2E-local-remote-debug.md
@@ -0,0 +1,28 @@
+# Using E2E local remote debugging
+The E2E remote debugging port of service containers is `5005`. If the developer wants to use remote debugging, he needs to add remote debugging parameters to the start service command, and then expose the port `5005`.
+
+For example, this is the configuration of a container in the [skywalking/test/e2e/e2e-test/docker/base-compose.yml](https://github.com/apache/skywalking/blob/master/test/e2e/e2e-test/docker/base-compose.yml). [JAVA_OPTS](https://github.com/apache/skywalking/blob/190ca93b6bf48e9d966de5b05cd6490ba54b7266/docker/oap/docker-entrypoint.sh) is a preset variable for passing additional parameters in the AOP service startup command, so we only need to add the JAVA remote debugging parameters `age [...]
+```yml
+oap:
+ image: skywalking/oap:latest
+ expose:
+ ...
+ - 5005
+ ...
+ environment:
+ ...
+ JAVA_OPTS: >-
+ ...
+ -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005
+ ...
+```
+At last, if the E2E test failed and is retrying, the developer can get the ports mapping in the file `skywalking/test/e2e/e2e-test/remote_real_port` and selects the host port of the corresponding service for remote debugging. For example,
+```bash
+#remote_real_port
+
+#The remote debugging port on the host is 32783
+oap-localhost:32783
+
+#The remote debugging port on the host is 32782
+provider-localhost:32782
+```
\ No newline at end of file
diff --git a/docs/en/guides/README.md b/docs/en/guides/README.md
index 3800345..c2b89b0 100755
--- a/docs/en/guides/README.md
+++ b/docs/en/guides/README.md
@@ -97,6 +97,9 @@ We expose all the logs from all containers to the stdout in non-CI (local) mode,
**NOTE:** Please verify the newly-added E2E test case locally first, however, if you find it passed locally but failed in the PR check status, make sure all the updated/newly-added files (especially those in submodules)
are committed and included in that PR, or reset the git HEAD to the remote and verify locally again.
+#### E2E local remote debugging
+When the E2E test is executed locally, if any test case fails, the [E2E local remote debugging function](E2E-local-remote-debug.md) can be used to quickly troubleshoot the bug.
+
### Project Extensions
SkyWalking project supports many ways to extend existing features. If you are interesting in these ways,
read the following guides.
diff --git a/test/e2e/e2e-common/src/main/java/org/apache/skywalking/e2e/SkyWalkingAnnotations.java b/test/e2e/e2e-common/src/main/java/org/apache/skywalking/e2e/SkyWalkingAnnotations.java
index 5318536..96817d8 100644
--- a/test/e2e/e2e-common/src/main/java/org/apache/skywalking/e2e/SkyWalkingAnnotations.java
+++ b/test/e2e/e2e-common/src/main/java/org/apache/skywalking/e2e/SkyWalkingAnnotations.java
@@ -48,6 +48,7 @@ import org.testcontainers.containers.DockerComposeContainer;
import org.testcontainers.containers.output.Slf4jLogConsumer;
import org.testcontainers.containers.wait.strategy.Wait;
+import static java.util.stream.Collectors.joining;
import static org.apache.skywalking.e2e.utils.Yamls.load;
/**
@@ -203,22 +204,20 @@ public final class SkyWalkingAnnotations {
final DockerComposeContainer<?> compose = new DockerComposeContainer<>(IDENTIFIER, files);
if (!IS_CI) {
- files.forEach(file -> {
- try {
- DockerComposeFile dockerComposeFile = DockerComposeFile.getAllConfigInfo(
- file.getAbsolutePath());
- LOGGER.info(file.getAbsolutePath());
- dockerComposeFile.getServices().forEach(
- (service, ignored) -> {
- if (dockerComposeFile.isExposedPort(service, REMOTE_DEBUG_PORT)) {
- REMOTE_SERVICE_NAMES.add(service);
- compose.withExposedService(service, REMOTE_DEBUG_PORT, Wait.forListeningPort());
- }
- });
- } catch (IOException | InterruptedException e) {
- LOGGER.error(e.getMessage(), e);
- }
- });
+ List<String> filePathList = files.stream().map(File::getAbsolutePath).collect(Collectors.toList());
+ try {
+ LOGGER.info("Parse files:{}", filePathList.stream().collect(joining(" ", " ", "")));
+ DockerComposeFile dockerComposeFile = DockerComposeFile.getAllConfigInfo(filePathList);
+
+ dockerComposeFile.getServices().forEach((service, ignored) -> {
+ if (dockerComposeFile.isExposedPort(service, REMOTE_DEBUG_PORT)) {
+ REMOTE_SERVICE_NAMES.add(service);
+ compose.withExposedService(service, REMOTE_DEBUG_PORT, Wait.forListeningPort());
+ }
+ });
+ } catch (IOException | InterruptedException e) {
+ LOGGER.error(e.getMessage(), e);
+ }
}
for (final Field field : fields) {
diff --git a/test/e2e/e2e-common/src/main/java/org/apache/skywalking/e2e/docker/DockerComposeFile.java b/test/e2e/e2e-common/src/main/java/org/apache/skywalking/e2e/docker/DockerComposeFile.java
index dc5e1ea..f469fcb 100644
--- a/test/e2e/e2e-common/src/main/java/org/apache/skywalking/e2e/docker/DockerComposeFile.java
+++ b/test/e2e/e2e-common/src/main/java/org/apache/skywalking/e2e/docker/DockerComposeFile.java
@@ -26,6 +26,7 @@ import java.util.List;
import java.util.Map;
import lombok.Data;
+import static java.util.stream.Collectors.joining;
import static org.apache.skywalking.e2e.utils.Yamls.load;
@Data
@@ -34,8 +35,9 @@ public final class DockerComposeFile {
private Map<String, Map<String, Object>> services;
private Map<String, Map<String, Object>> networks;
- public static DockerComposeFile getAllConfigInfo(String composeFile) throws IOException, InterruptedException {
- String shStr = String.format("docker-compose -f %s config", composeFile);
+ public static DockerComposeFile getAllConfigInfo(List<String> composeFiles) throws IOException, InterruptedException {
+ String shStr = String.format("docker-compose %s config",
+ composeFiles.stream().collect(joining(" -f", " -f", "")));
Process process = Runtime.getRuntime().exec(shStr, null, null);
InputStreamReader ir = new InputStreamReader(process.getInputStream());
LineNumberReader input = new LineNumberReader(ir);
diff --git a/test/e2e/e2e-common/src/test/java/org/apache/skywalking/e2e/docker/DockerComposeFileTest.java b/test/e2e/e2e-common/src/test/java/org/apache/skywalking/e2e/docker/DockerComposeFileTest.java
index 120c27f..5562c8f 100644
--- a/test/e2e/e2e-common/src/test/java/org/apache/skywalking/e2e/docker/DockerComposeFileTest.java
+++ b/test/e2e/e2e-common/src/test/java/org/apache/skywalking/e2e/docker/DockerComposeFileTest.java
@@ -26,41 +26,54 @@ import org.junit.jupiter.api.Test;
import java.io.File;
import java.io.IOException;
import java.net.URISyntaxException;
+import java.util.ArrayList;
import java.util.List;
class DockerComposeFileTest {
- private static DockerComposeFile COMPOSE_FILE = null;
+ private static DockerComposeFile COMPOSE_FILE_ONE = null;
+ private static DockerComposeFile COMPOSE_FILE_TWO = null;
@BeforeAll
static void setUp() throws Exception {
- COMPOSE_FILE = Yamls.load("docker-compose.yml").as(DockerComposeFile.class);
+ COMPOSE_FILE_ONE = Yamls.load("docker-compose.one.yml").as(DockerComposeFile.class);
+ COMPOSE_FILE_TWO = Yamls.load("docker-compose.two.yml").as(DockerComposeFile.class);
}
@Test
void getAllConfigInfo() throws IOException, InterruptedException, URISyntaxException {
- File file = new File(DockerComposeFileTest.class
+ File composeOne = new File(DockerComposeFileTest.class
.getClassLoader()
- .getResource("docker-compose.yml")
+ .getResource("docker-compose.one.yml")
.toURI());
- DockerComposeFile testFile = DockerComposeFile.getAllConfigInfo(file.getAbsolutePath());
+
+ File composeTwo = new File(DockerComposeFileTest.class
+ .getClassLoader()
+ .getResource("docker-compose.two.yml")
+ .toURI());
+
+ List<String> filePathList = new ArrayList<>();
+ filePathList.add(composeOne.getAbsolutePath());
+ filePathList.add(composeTwo.getAbsolutePath());
+ DockerComposeFile testFile = DockerComposeFile.getAllConfigInfo(filePathList);
Assert.assertNotNull(testFile);
Assert.assertNotNull(testFile.getServices());
- Assert.assertEquals(COMPOSE_FILE.getServices().size(), testFile.getServices().size());
- Assert.assertEquals(COMPOSE_FILE.getVersion(), testFile.getVersion());
+ Assert.assertEquals(COMPOSE_FILE_ONE.getServices().size() + COMPOSE_FILE_TWO.getServices().size(),
+ testFile.getServices().size());
+ Assert.assertEquals(COMPOSE_FILE_ONE.getVersion(), testFile.getVersion());
}
@Test
void getServiceExposedPort() {
- List<String> ports = COMPOSE_FILE.getServiceExposedPorts("oap");
+ List<String> ports = COMPOSE_FILE_TWO.getServiceExposedPorts("oap");
Assert.assertNotNull(ports);
Assert.assertEquals(3, ports.size());
}
@Test
void isExposedPort() {
- boolean result = COMPOSE_FILE.isExposedPort("oap", 5005);
+ boolean result = COMPOSE_FILE_TWO.isExposedPort("oap", 5005);
Assert.assertTrue(result);
- result = COMPOSE_FILE.isExposedPort("oap", 5006);
+ result = COMPOSE_FILE_TWO.isExposedPort("oap", 5006);
Assert.assertFalse(result);
}
}
\ No newline at end of file
diff --git a/test/e2e/e2e-common/src/test/resources/docker-compose.yml b/test/e2e/e2e-common/src/test/resources/docker-compose.one.yml
similarity index 52%
copy from test/e2e/e2e-common/src/test/resources/docker-compose.yml
copy to test/e2e/e2e-common/src/test/resources/docker-compose.one.yml
index e23afb6..8097189 100644
--- a/test/e2e/e2e-common/src/test/resources/docker-compose.yml
+++ b/test/e2e/e2e-common/src/test/resources/docker-compose.one.yml
@@ -17,28 +17,14 @@
version: '2.1'
services:
- oap:
- image: skywalking/oap:latest
+ ui:
+ depends_on:
+ oap:
+ condition: service_healthy
+ environment:
+ SW_OAP_ADDRESS: oap:12800
expose:
- - 11800
- - 12800
- - 5005
+ - '8080'
+ image: skywalking/ui:latest
networks:
- - e2e
- restart: on-failure
- environment:
- SW_CLUSTER_ZK_HOST_PORT: zk:2181
- SW_STORAGE_ES_CLUSTER_NODES: es:9200
- SW_JDBC_URL: jdbc:mysql://mysql:3306/swtest
- SW_STORAGE_INFLUXDB_URL: http://influxdb:8086
- JAVA_OPTS: >-
- -javaagent:/jacoco/jacocoagent.jar=classdumpdir=/jacoco/classes/oap,destfile=/jacoco/oap.exec,includes=org.apache.skywalking.*,excludes=org.apache.skywalking.oap.query.*:org.apache.skywalking.oap.server.core.query.*
- -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005
- healthcheck:
- test: ["CMD", "sh", "-c", "nc -zn 127.0.0.1 11800"]
- interval: 5s
- timeout: 60s
- retries: 120
-
-networks:
- e2e: {}
\ No newline at end of file
+ - e2e
\ No newline at end of file
diff --git a/test/e2e/e2e-common/src/test/resources/docker-compose.yml b/test/e2e/e2e-common/src/test/resources/docker-compose.two.yml
similarity index 100%
rename from test/e2e/e2e-common/src/test/resources/docker-compose.yml
rename to test/e2e/e2e-common/src/test/resources/docker-compose.two.yml