You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by zh...@apache.org on 2022/12/02 12:53:21 UTC
[shardingsphere] branch master updated: Fixes documentation for building GraalVM Native Image (#21341)
This is an automated email from the ASF dual-hosted git repository.
zhangliang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/shardingsphere.git
The following commit(s) were added to refs/heads/master by this push:
new b097f4db96b Fixes documentation for building GraalVM Native Image (#21341)
b097f4db96b is described below
commit b097f4db96b0fb65d80df086807a742a2f17b9c5
Author: Ling Hengqian <li...@outlook.com>
AuthorDate: Fri Dec 2 20:53:14 2022 +0800
Fixes documentation for building GraalVM Native Image (#21341)
---
.../startup/graalvm-native-image.cn.md | 87 +++++++++++++++-----
.../startup/graalvm-native-image.en.md | 92 ++++++++++++++++++----
2 files changed, 144 insertions(+), 35 deletions(-)
diff --git a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md
index f5446d1e9ac..8e523158383 100644
--- a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md
+++ b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md
@@ -11,7 +11,20 @@ weight = 2
## 注意事项
- ShardingSphere Proxy 尚未准备好与 GraalVM Native Image 集成。
- 其在 https://github.com/apache/shardingsphere/actions/ 存在每日构建的任务用于测试构建。
+ 其在 https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native 存在每夜构建。
+ 假设存在包含`server.yaml` 的 `conf` 文件夹为 `./custom/conf`，你可通过如下的 `docker-compose.yml` 文件进行测试。
+
+```yaml
+version: "3.8"
+
+services:
+ apache-shardingsphere-proxy-native:
+ image: ghcr.io/apache/shardingsphere-proxy-native:latest
+ volumes:
+ - ./custom/conf:/conf
+ ports:
+ - "3307:3307"
+```
- 若你发现构建过程存在缺失的 GraalVM Reachability Metadata,
应当在 https://github.com/oracle/graalvm-reachability-metadata 打开新的 issue ，
@@ -21,8 +34,8 @@ weight = 2
需要等待 Junit 5 Platform 的集成，你总是需要在构建 GraalVM Native Image 的过程中，
加上特定于 `GraalVM Native Build Tools` 的 `-DskipNativeTests` 或 `-DskipTests` 参数跳过 Native Image 中的单元测试。
-- 本节假定处于 Linux（amd64，aarch64）， MacOS（ amd64 ）或 Windows（amd64）环境。
- 如果你位于 MacOS(aarch64/M1) 环境, 你需要关注尚未关闭的 https://github.com/oracle/graal/issues/2666 。
+- 本节假定处于 Linux（amd64，aarch64）， MacOS（amd64）或 Windows（amd64）环境。
+ 如果你位于 MacOS（aarch64/M1） 环境，你需要关注尚未关闭的 https://github.com/oracle/graal/issues/2666。
## 前提条件
@@ -49,21 +62,21 @@ weight = 2
- 在 Git Source 同级目录下执行如下命令, 直接完成 Native Image 的构建。
```bash
-./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
+./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
```
- 情形二：需要使用存在 SPI 实现的 JAR 或 GPL V2 等 LICENSE 的第三方依赖的 JAR。
-- 在 `shardingsphere-distribution/shardingsphere-proxy-native-distribution/pom.xml` 的 `dependencies` 加入存在 SPI 实现的 JAR
+- 在 `distribution/proxy-native/pom.xml` 的 `dependencies` 加入存在 SPI 实现的 JAR
或第三方依赖的 JAR。示例如下
```xml
<dependencies>
<dependency>
- <groupId>mysql</groupId>
- <artifactId>mysql-connector-java</artifactId>
- <version>8.0.30</version>
+ <groupId>com.mysql</groupId>
+ <artifactId>mysql-connector-j</artifactId>
+ <version>8.0.31</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
@@ -76,7 +89,7 @@ weight = 2
- 通过命令行构建 GraalVM Native Image。
```bash
-./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
+./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
```
3. 通过命令行启动 Native Image, 需要带上两个参数，
@@ -84,13 +97,13 @@ weight = 2
假设已存在文件夹`./custom/conf`，示例为
```bash
-./apache-shardingsphere-proxy 3307 ./custom/conf
+./apache-shardingsphere-proxy-native 3307 ./custom/conf
```
4. 如果需要构建 Docker Image, 在添加后存在 SPI 实现的依赖或第三方依赖后, 在命令行执行如下命令。
```shell
-./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative,docker.native -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
+./mvnw -am -pl distribution/proxy-native -B -Pnative,docker.native -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
```
- 假设存在包含`server.yaml` 的 `conf` 文件夹为 `./custom/conf`，可通过如下的 `docker-compose.yml` 文件启动 GraalVM Native
@@ -108,11 +121,49 @@ services:
- "3307:3307"
```
-- 如果您使用默认构建配置， 你当然可以为 `shardingsphere-distribution/shardingsphere-proxy-native-distribution/Dockerfile`
- 使用 `scratch` 作为 base docker image。
- 但如果您主动为`pom.xml`的`native profile`添加`jvmArgs`为`-H:+StaticExecutableWithDynamicLibC`，
- 以静态链接除 `glic` 之外的所有内容，您应该切换 base image 到 `busybox:glic`。
- 参考 https://www.graalvm.org/22.2/reference-manual/native-image/guides/build-static-executables/ 。
- 另请注意，某些第三方依赖将需要更多系统库，例如 `libdl`。
- 因此请确保根据您的使用情况调整 base docker image 和`shardingsphere-distribution/shardingsphere-proxy-native-distribution`
+- 如果你不对 Git Source 做任何更改， 上文提及的命令将使用 `oraclelinux:9-slim` 作为 Base Docker Image。
+ 但如果你希望使用 `busybox:glic`，`gcr.io/distroless/base` 或 `scratch` 等更小体积的 Docker Image 作为 Base Docker
+ Image，你需要根据 https://www.graalvm.org/22.2/reference-manual/native-image/guides/build-static-executables/ 的要求，
+ 做为 `pom.xml`的 `native profile` 添加 `-H:+StaticExecutableWithDynamicLibC` 的 `jvmArgs` 等操作。
+ 另请注意，某些第三方依赖将需要在 `Dockerfile` 安装更多系统库，例如 `libdl`。
+ 因此请确保根据你的使用情况调整 `distribution/proxy-native`
下的 `pom.xml` 和 `Dockerfile` 的内容。
+
+# 可观察性
+
+- 针对 GraalVM Native Image 形态的 ShardingSphere
+ Proxy，其提供的可观察性的能力与 https://shardingsphere.apache.org/document/current/cn/user-manual/shardingsphere-proxy/observability/
+ 并不一致。
+
+- 你可以使用 https://www.graalvm.org/22.2/tools/ 提供的一系列命令行工具或可视化工具观察 GraalVM Native Image
+ 的内部行为，并根据其要求使用 VSCode 完成调试工作。
+ 如果你正在使用 IntelliJ IDEA 并且希望调试生成的 GraalVM Native Image，
+ 你可以关注 https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java
+ 及其后继。
+
+- 对于使用 `ShardingSphere Agent` 等 APM Java Agent 的情形，
+ GraalVM 的 `native-image` 组件尚未完全支持在构建 Native Image 时使用
+ javaagent，你需要关注尚未关闭的 https://github.com/oracle/graal/issues/1065。
+
+- 以下部分采用 `Apache SkyWalking Java Agent` 作为示例，可用于跟踪 GraalVM 社区的对应 issue。
+
+1. 下载 https://dlcdn.apache.org/skywalking/java-agent/8.12.0/apache-skywalking-java-agent-8.12.0.tgz ，
+ 并解压到 ShardingSphere Git Source 的 `distribution/proxy-native`。
+
+2. 修改 `distribution/proxy-native/pom.xml` 的 `native profile`，
+ 为 `org.graalvm.buildtools:native-maven-plugin` 的 `configuration` 添加如下的 `jvmArgs`。
+
+```xml
+
+<jvmArgs>
+ <arg>-Dskywalking.agent.service_name="your service name"</arg>
+ <arg>-Dskywalking.collector.backend_service="your skywalking oap ip and port"</arg>
+ <arg>-javaagent:./skywalking-agent/skywalking-agent.jar</arg>
+</jvmArgs>
+```
+
+3. 通过命令行构建 GraalVM Native Image。
+
+```bash
+./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
+```
\ No newline at end of file
diff --git a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md
index b253007729b..bf482eeb6f4 100644
--- a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md
+++ b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md
@@ -11,7 +11,22 @@ corresponding `Docker Image` through the `native-image` component of `GraalVM`.
## Notice
- ShardingSphere Proxy is not yet ready to integrate with GraalVM Native Image.
- It has daily build tasks at https://github.com/apache/shardingsphere/actions/ for testing builds.
+ Fixes documentation for building GraalVM Native Image It exists nightly builds
+ at https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native.
+ Assuming there is a `conf` folder containing `server.yaml` as `./custom/conf`, you can test it with the
+ following `docker-compose.yml` file.
+
+````yaml
+version: "3.8"
+
+services:
+ apache-shardingsphere-proxy-native:
+ image: ghcr.io/apache/shardingsphere-proxy-native:latest
+ volumes:
+ - ./custom/conf:/conf
+ ports:
+ - "3307:3307"
+````
- If you find that the build process has missing GraalVM Reachability Metadata,
A new issue should be opened at https://github.com/oracle/graalvm-reachability-metadata,
@@ -53,22 +68,22 @@ corresponding `Docker Image` through the `native-image` component of `GraalVM`.
Image.
```bash
-./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
+./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
```
- Scenario 2: It is necessary to use a JAR that has an SPI implementation or a third-party dependent JAR of a LICENSE
such as GPL V2.
- Add SPI implementation JARs or third-party dependent JARs to `dependencies`
- in `shardingsphere-distribution/shardingsphere-proxy-native-distribution/pom.xml`. Examples are as follows
+ in `distribution/proxy-native/pom.xml`. Examples are as follows
```xml
<dependencies>
<dependency>
- <groupId>mysql</groupId>
- <artifactId>mysql-connector-java</artifactId>
- <version>8.0.30</version>
+ <groupId>com.mysql</groupId>
+ <artifactId>mysql-connector-j</artifactId>
+ <version>8.0.31</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
@@ -81,7 +96,7 @@ corresponding `Docker Image` through the `native-image` component of `GraalVM`.
- Build GraalVM Native Image via command line.
```bash
-./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
+./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package
```
3. Start Native Image through the command line, you need to bring two parameters,
@@ -90,14 +105,14 @@ corresponding `Docker Image` through the `native-image` component of `GraalVM`.
Assuming the folder `./custom/conf` already exists, the example is
```bash
-./apache-shardingsphere-proxy 3307 ./custom/conf
+./apache-shardingsphere-proxy-native 3307 ./custom/conf
````
4. If you need to build a Docker Image, after adding the dependencies of the SPI implementation or third-party
dependencies, execute the following commands on the command line.
```shell
-./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative,docker.native -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat .skip=true clean package
+./mvnw -am -pl distribution/proxy-native -B -Pnative,docker.native -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat .skip=true clean package
```
- Assuming that there is a `conf` folder containing `server.yaml` as `./custom/conf`, you can start the Docker Image
@@ -115,11 +130,54 @@ services:
- "3307:3307"
```
-- If you use the default build configuration, you can of course use `scratch` as the base docker image
- for `shardingsphere-distribution/shardingsphere-proxy-native-distribution/Dockerfile`.
- But if you actively add `jvmArgs` to `-H:+StaticExecutableWithDynamicLibC` for the `native profile` of `pom.xml`,
- To statically link everything except `glic`, you should switch the base image to `busybox:glic`. Refer
- to https://www.graalvm.org/22.2/reference-manual/native-image/guides/build-static-executables/.
- Also note that some third-party dependencies will require more system libraries, such as `libdl`.
- So make sure to adjust the base docker image and the content of `pom.xml` and `Dockerfile`
- under `shardingsphere-distribution/shardingsphere-proxy-native-distribution` according to your usage.
+- If you don't make any changes to the Git Source, the commands mentioned above will use `oraclelinux:9-slim` as the
+ Base Docker Image.
+ But if you want to use a smaller Docker Image like `busybox:glic`, `gcr.io/distroless/base` or `scratch` as the Base
+ Docker Image, you need according
+ to https://www.graalvm.org/22.2/reference-manual/native-image/guides/build-static-executables/,
+ Add operations such as `-H:+StaticExecutableWithDynamicLibC` to `jvmArgs` as the `native profile` of `pom.xml`.
+ Also note that some 3rd party dependencies will require more system libraries such as `libdl` to be installed in
+ the `Dockerfile`.
+ So make sure to tune `distribution/proxy-native` according to your usage
+ `pom.xml` and `Dockerfile` below.
+
+# Observability
+
+- ShardingSphere for GraalVM Native Image form Proxy, which provides observability capabilities
+ with https://shardingsphere.apache.org/document/current/cn/user-manual/shardingsphere-proxy/observability/
+ Not consistent.
+
+- You can observe GraalVM Native Image using a series of command line tools or visualization tools available
+ at https://www.graalvm.org/22.2/tools/, and use VSCode to debug it according to its requirements.
+ If you are using IntelliJ IDEA and want to debug the generated GraalVM Native Image,
+ You can
+ follow https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java
+ and its successors.
+
+- In the case of using APM Java Agent such as `ShardingSphere Agent`,
+ GraalVM's `native-image` component is not yet fully supported when building Native Images
+ javaagent, you need to follow https://github.com/oracle/graal/issues/1065 which has not been closed.
+
+- The following sections use the `Apache SkyWalking Java Agent` as an example, which can be used to track corresponding
+ issues from the GraalVM community.
+
+1. Download https://dlcdn.apache.org/skywalking/java-agent/8.12.0/apache-skywalking-java-agent-8.12.0.tgz and `untar` it
+ to `distribution/proxy-native` in ShardingSphere Git Source.
+
+2. Modify the `native profile` of `distribution/proxy-native/pom.xml`,
+ Add the following `jvmArgs` to the `configuration` of `org.graalvm.buildtools:native-maven-plugin`.
+
+```xml
+
+<jvmArgs>
+ <arg>-Dskywalking.agent.service_name="your service name"</arg>
+ <arg>-Dskywalking.collector.backend_service="your skywalking oap ip and port"</arg>
+ <arg>-javaagent:./skywalking-agent/skywalking-agent.jar</arg>
+</jvmArgs>
+```
+
+3. Build the GraalVM Native Image from the command line.
+
+```bash
+./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip =true clean package
+```
\ No newline at end of file