You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by al...@apache.org on 2022/08/10 08:16:46 UTC
[dubbo-samples] branch master updated: Proxy mesh demo (#492)
This is an automated email from the ASF dual-hosted git repository.
albumenj pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dubbo-samples.git
The following commit(s) were added to refs/heads/master by this push:
new 808a2d69 Proxy mesh demo (#492)
808a2d69 is described below
commit 808a2d69b44438561d07fc9a9a6cd37e73259a4f
Author: ken.lj <ke...@gmail.com>
AuthorDate: Wed Aug 10 16:16:42 2022 +0800
Proxy mesh demo (#492)
* reorganize mesh README
* improve mesh demo
* proxy mesh demo
* use right provider image version
* finish readme
---
dubbo-samples-mesh-k8s/README.md | 377 ++++++---------------
dubbo-samples-mesh-k8s/assets/thinsdk.png | Bin 0 -> 7418 bytes
.../deploy/consumer/Deployment.yml | 4 +-
.../provider/{Deployment.yml => Deployment-v2.yml} | 12 +-
.../deploy/provider/Deployment.yml | 8 +-
dubbo-samples-mesh-k8s/deploy/provider/Service.yml | 34 --
.../Service.yml => traffic/virtual-service.yml} | 35 +-
.../dubbo-samples-mesh-consumer/pom.xml | 28 ++
.../apache/dubbo/samples/ConsumerBootstrap.java | 6 +-
.../dubbo-samples-mesh-provider/pom.xml | 28 ++
.../org/apache/dubbo/samples/impl/GreeterImpl.java | 6 +-
.../resources/spring/dubbo-provider.properties | 2 +-
12 files changed, 195 insertions(+), 345 deletions(-)
diff --git a/dubbo-samples-mesh-k8s/README.md b/dubbo-samples-mesh-k8s/README.md
index 253070f4..9ee10d60 100644
--- a/dubbo-samples-mesh-k8s/README.md
+++ b/dubbo-samples-mesh-k8s/README.md
@@ -1,38 +1,38 @@
# Dubbo mesh using Istio
-可以按照下文步骤，将 Demo 部署到本地集群，也可在 [KataCoda 在线快速体验]()。
+可以按照下文步骤，将 Demo 部署到本地集群。
* [1 总体目标](#target)
* [2 基本流程](#basic)
* [3 详细步骤](#detail)
+ [3.1 环境要求](#env)
+ [3.2 前置条件](#prepare)
- + [3.3 项目与镜像打包（可跳过）](#image)
- + [3.4 部署到 Kubernetes](#deploy)
+ + [3.3 部署到 Kubernetes](#deploy)
- [3.4.1 部署 Provider](#deploy_provider)
- [3.4.2 部署 Consumer](#deploy_consumer)
+ [3.5 检查 Provider 和 Consumer 正常通信](#check)
-* [4 健康检查](#health_check)
- + [4.1 Istio 服务的健康检查](#health_check_istio)
- + [4.2 Envoy 主动健康检查](#health_check_envoy)
-* [5 下一步计划](#next)
-* [6 常用命令](#common)
+ + [3.6 Istio 流量治理](#3.6-Istio-流量治理)
+* [4 修改示例与镜像打包](#4-修改示例)
+* [5 附录：常用命令](#common)
<h2 id="target">1 总体目标</h2>
-* 部署 Dubbo 应用到 Kubernetes + Istio
-* 基于 Kubernetes 内置 Service 实现服务发现, Pilot 监听 api-server ，通过 XDS 下发给 Envoy 实例，去掉注册中心
-* 将 Dubbo 应用对接到 Kubernetes 生命周期
-* 利用 Envoy 代理实现服务调用、负载均衡
-* 基于 EnvoyFilter 对 consumer 的上游集群做主动健康检查
+* 部署 Dubbo 应用到 Kubernetes
+* Istio 自动注入 Envoy 并实现流量拦截
+* 基于 Istio 规则进行流量治理
-<h2 id="basic">2 基本流程</h2>
+<h2 id="basic">2 基本流程与工作原理</h2>
+这个示例演示了如何将 Dubbo 开发的应用部署在 Istio 体系下，以实现 Envoy 对 Dubbo 服务的自动代理，示例总体架构如下图所示。
+
+[thinsdk](./assets/thinsdk.png)
+
+完成示例将需要的步骤如下：
1. 创建一个 Dubbo 应用( [dubbo-samples-mesh-k8s](https://github.com/apache/dubbo-samples/tree/master/dubbo-samples-mesh-k8s) )
-2. 构建容器镜像并推送到镜像仓库( [dubbomesh 示例镜像](https://hub.docker.com/u/15841721425) )
-3. 分别部署 Dubbo Provider 与 Dubbo Consumer 到 Kubernetes
-4. 验证服务发现与调用正常
-5. 观测负载均衡与主动健康检查
+2. 构建容器镜像并推送到镜像仓库（ [本示例官方镜像](https://hub.docker.com/u/dubboteam) ）
+3. 分别部署 Dubbo Provider 与 Dubbo Consumer 到 Kubernetes 并验证 Envoy 代理注入成功
+4. 验证 Envoy 发现服务地址、正常拦截 RPC 流量并实现负载均衡
+5. 优化并配置健康检查流程
<h2 id="detail">3 详细步骤</h2>
@@ -43,8 +43,8 @@
* [Docker](https://www.docker.com/get-started/)
* [Minikube](https://minikube.sigs.k8s.io/docs/start/)
* [Kubectl](https://kubernetes.io/docs/tasks/tools/)
+* [Istio](https://istio.io/latest/docs/setup/getting-started/)
* [Kubens(optional)](https://github.com/ahmetb/kubectx)
-* [Istio](https://istio.io/latest/zh/)
通过以下命令启动本地 Kubernetes 集群
@@ -60,10 +60,7 @@ kubectl cluster-info
<h3 id="prepare">3.2 前置条件</h3>
-由于示例 Dubbo 项目均部署在 Pod 中且与 API-SERVER 有交互，因此有相应的权限要求，我们这里创建独立 ServiceAccount 并绑定必须的 Roles，后面所有的 Dubbo Kubernetes
-资源都将使用这里新建的 ServiceAccount。
-
-通过以下命令我们创建了独立的 Namespace `dubbo-demo` 与 ServiceAccount `dubbo-sa`。
+通过以下命令为示例项目创建独立的 Namespace `dubbo-demo`，同时开启 sidecar 自动注入。
```shell
# 初始化命名空间
@@ -72,69 +69,9 @@ kubectl apply -f https://raw.githubusercontent.com/apache/dubbo-samples/master/d
# 切换命名空间
kubens dubbo-demo
-# 注意项目路径一定是英文，否则protobuf编译失败
-```
-
-由于 dubbo 3.1.0-SNAPSHOT 未上线，需要对 dubbo 3.1.0-SNAPSHOT 源码构建后，install 本地仓库，才可使用。
-
-参考[Dubbo 源码构建](https://dubbo.apache.org/zh/docsv2.7/dev/build/)
-
-<h3 id="image">3.3 项目与镜像打包（可跳过）</h3>
-
-示例项目及相关镜像均已就绪，以下仅为指引说明，可直接跳过此步骤直接查看 [3.4](#deploy) 小节。
-
-<text id="properties">设置 Dubbo 项目配置：</text>
-
-```properties
-# provider
-dubbo.application.name=dubbo-samples-mesh-provider
-dubbo.application.metadataServicePort=20885
-dubbo.registry.address=N/A
-dubbo.protocol.name=tri
-dubbo.protocol.port=50052
-dubbo.application.qosEnable=true
-# 为了使 Kubernetes 集群能够正常访问到探针，需要开启 QOS 允许远程访问，此操作有可能带来安全风险，请仔细评估后再打开
-dubbo.application.qosAcceptForeignIp=true
-
-```
-
-```properties
-# consumer
-dubbo.application.name=dubbo-samples-mesh-consumer
-dubbo.application.metadataServicePort=20885
-dubbo.registry.address=N/A
-dubbo.protocol.name=tri
-dubbo.protocol.port=50052
-dubbo.consumer.timeout=3000
-dubbo.application.qosEnable=true
-# 为了使 Kubernetes 集群能够正常访问到探针，需要开启 QOS 允许远程访问，此操作有可能带来安全风险，请仔细评估后再打开
-dubbo.application.qosAcceptForeignIp=true
-# enable mesh
-dubbo.consumer.meshEnable=true
-
-```
-
-如果要在本地打包镜像，可通过提供的Dockerfile打包镜像（也可以直接使用示例提供好的镜像包）
-
-```shell
-# 打包镜像
-cd dubbo-samples-mesh-provider
-docker build -f ./Dockerfile -t dubbo-samples-mesh-provider .
-
-cd dubbo-samples-mesh-consumer
-docker build -f ./Dockerfile -t dubbo-samples-mesh-consumer .
-
-# 下面的15841721425是私人地址，大家用的时候推到自己的仓库即可
-
-# 重命名镜像
-docker tag dubbo-samples-mesh-provider:latest 15841721425/dubbo-samples-mesh-provider-${version}
+# dubbo-demo 开启自动注入
+kubectl label namespace dubbo-demo istio-injection=enabled
-docker tag dubbo-samples-mesh-consumer:latest 15841721425/dubbo-samples-mesh-consumer-${version}
-
-# 推到镜像仓库
-docker push 15841721425/dubbo-samples-mesh-provider-${version}
-
-docker push 15841721425/dubbo-samples-mesh-consumer-${version}
```
<h3 id="deploy">3.4 部署到 Kubernetes</h3>
@@ -151,10 +88,6 @@ kubectl apply -f https://raw.githubusercontent.com/apache/dubbo-samples/master/d
以上命令创建了一个名为 `dubbo-samples-mesh-provider` 的 Service，注意这里的 service name 与项目中的 dubbo 应用名是一样的。
-同时 Service 里也包含 VirtualService 和 DestinationRule，是对 provider
-的流量进行治理，详细配置可参考[VirtualService 配置](https://istio.io/latest/zh/docs/reference/config/networking/virtual-service/)
-、[DestinationRule 配置](https://istio.io/latest/zh/docs/reference/config/networking/destination-rule/)。
-
接着 Deployment 部署了一个 2 副本的 pod 实例，至此 Provider 启动完成。
可以通过如下命令检查启动日志。
@@ -167,6 +100,8 @@ kubectl get pods -l app=dubbo-samples-mesh-provider
kubectl logs your-pod-id
```
+这时 pod 中应该有一个 dubbo provider 容器实例，同时还有一个 Envoy Sidecar 容器实例。
+
<h4 id="deploy_consumer">3.4.2 部署 Consumer</h3>
```shell
@@ -177,30 +112,13 @@ kubectl apply -f https://raw.githubusercontent.com/apache/dubbo-samples/master/d
kubectl apply -f https://raw.githubusercontent.com/apache/dubbo-samples/master/dubbo-samples-mesh-k8s/deploy/consumer/Deployment.yml
```
-部署 consumer 与 provider 是一样的，这里也保持了 K8S Service 与 Dubbo consumer 名字一致： dubbo-samples-mesh-consumer。
-
-<h3 id="check">3.5 检查 Provider 和 Consumer 正常通信</h3>
-
-**注：**
-
-grpc 通过 HTTP/2 运行，与通过 HTTP/1.1 运行相比具有若干优点，例如高效的二进制编码显著降低序列化成本，通过单个连接复用请求和响应减少 TCP
-管理开销，以及自动类型检查。但是kubernetes的默认负载平衡通常不能对grpc起到作用，需要添加其他grpc负载均衡服务。
-
-- 为什么 grpc 需要特殊的负载均衡
-
-因为 grpc 构建在HTTP/2上，而 HTTP/2 被设计为具有单个长期 TCP
-连接，所有请求都被多路复用，意味着多个请求可以在任何时间点在同一连接上处于活动状态。这减少了连接管理开销，但也意味着连接级的均衡没有作用。一旦建立连接，就不再需要进行平衡。所有的请求都将固定到单个目标实例上，如下所示：
-
-<div style="text-align: center"><img src="./assets/grpc_load.png" width="600" alt="grpc负载不均"></div>
-
-Kubernetes 的 kube-proxy 本质上是一个 L4 负载平衡器，因此我们不能依靠它来平衡微服务之间的 gRPC 调用。
+部署 consumer 与 provider 是一样的，这里也保持了 K8S Service 与 Dubbo consumer application name(在 [dubbo.properties](https://github.com/apache/dubbo-samples/blob/master/dubbo-samples-mesh-k8s/dubbo-samples-mesh-consumer/src/main/resources/spring/dubbo-consumer.properties) 中定义) 一致： `dubbo.application.name=dubbo-samples-mesh-consumer`。
-使用 Envoy 就可以优雅的解决上诉问题。 详细配置在 Service.yml 中，其中：
+> Dubbo Consumer 服务声明中还指定了消费的 Provider 服务（应用）名 `@DubboReference(version = "1.0.0", providedBy = "dubbo-samples-mesh-provider", lazy = true)`
-- grpc 对应的服务端口，name 要加 **grpc-** 前缀；
-- Service 不能配置成 Headless 服务。
+<h3 id="check">3.5 检查 Provider 和 Consumer 正常通信</h3>
-继执行 [3.4](#deploy) 步骤后， 检查启动日志，查看 consumer 完成对 provider 服务的消费。
+继执行 3.4 步骤后， 检查启动日志，查看 consumer 完成对 provider 服务的消费。
```shell
# 查看 pod 列表
@@ -216,18 +134,12 @@ kubectl logs your-pod-id -c istio-proxy
可以看到 consumer pod 日志输出如下( Triple 协议被 Envoy 代理负载均衡):
```bash
-[15/07/22 11:37:50:050 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message:
-"hello,service mesh, response from provider: 172.17.0.11:50052, client: 172.17.0.11, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-
-==================== dubbo invoke 1513 end ====================
-[15/07/22 11:38:00:000 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message:
-"hello,service mesh, response from provider: 172.17.0.8:50052, client: 172.17.0.8, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
+==================== dubbo invoke 0 end ====================
+[10/08/22 07:07:36:036 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.22:50052, client: 172.18.96.22, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-==================== dubbo invoke 1514 end ====================
-[15/07/22 11:38:10:010 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message:
-"hello,service mesh, response from provider: 172.17.0.6:50052, client: 172.17.0.6, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
+==================== dubbo invoke 1 end ====================
+[10/08/22 07:07:42:042 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.18:50052, client: 172.18.96.18, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-==================== dubbo invoke 1515 end ====================
```
consumer istio-proxy 日志输出如下:
@@ -242,8 +154,9 @@ outbound|50052||dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local 172.17.
可以看到 provider pod 日志输出如下:
```shell
-[22/06/22 08:22:36:036 UTC] Dubbo-protocol-50052-thread-8 INFO impl.GreeterImpl:
-Server test dubbo tri k8s received greet request name: "Kubernetes Api Server"
+[10/08/22 07:08:47:047 UTC] tri-protocol-50052-thread-8 INFO impl.GreeterImpl: Server test dubbo tri mesh received greet request name: "service mesh"
+
+[10/08/22 07:08:57:057 UTC] tri-protocol-50052-thread-9 INFO impl.GreeterImpl: Server test dubbo tri mesh received greet request name: "service mesh"
```
provider istio-proxy 日志输出如下:
@@ -256,184 +169,104 @@ provider istio-proxy 日志输出如下:
outbound_.50052_._.dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local default
```
-<h2 id="health_check">4 健康检查</h2>
-
-<h3 id="health_check_istio">4.1 Istio 服务的健康检查</h3>
-
-Pod 的生命周期 与服务调度息息相关，通过对 Kubernetes 官方探针的实现，能够使 Dubbo 乃至整个应用的生命周期与 Pod 的生命周期对齐。
-
-**存活检测**
+#### 3.6 Istio 流量治理
-对于 livenessProbe 存活检测，由于 Dubbo 框架本身无法获取到应用的存活状态，因此本接口无默认实现，且默认返回成功。开发者可以根据 SPI 定义对此 SPI 接口进行拓展，从应用层次对是否存活进行判断。
-
-**就绪检测**
+部署 v2 版本的 demo provider
+```shell
+kubectl apply -f https://raw.githubusercontent.com/apache/dubbo-samples/master/dubbo-samples-mesh-k8s/deploy/provider/Deployment-v2.yml
+```
-对于 readinessProbe 就绪检测，目前 Dubbo 默认提供了两个检测维度，一是对 Dubbo 服务自身是否启停做判断，另外是对所有服务是否存在已注册接口，如果所有服务均已从注册中心下线（可以通过 QOS
-运维进行操作）将返回未就绪的状态。（readinessProbe 目前 dubbo 实现方式不适用于 mesh，mesh 模式不配置注册中心，dubbo 的 readinessProbe 会返回 false）
+设置 VirtualService 与 DestinationRule，观察流量按照 4:1 的比例分别被引导到 provider v1 与 provider v2 版本。
+```shell
+kubectl apply -f https://raw.githubusercontent.com/apache/dubbo-samples/master/dubbo-samples-mesh-k8s/deploy/traffic/virtual-service.yml
+```
-**启动检测**
+从消费端日志输出中，观察流量分布效果如下图：
-对于 startupProbe 启动检测，目前Dubbo 默认提供了一个检测维度，即是在所有启动流程（接口暴露、注册中心写入等）均结束后返回已就绪状态。
+```java
+==================== dubbo invoke 100 end ====================
+[10/08/22 07:15:58:058 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.18:50052, client: 172.18.96.18, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-**使用方法：**
+==================== dubbo invoke 101 end ====================
+[10/08/22 07:16:03:003 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.22:50052, client: 172.18.96.22, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-参考配置(具体可以参考 [dubbo-samples-mesh-provider 的配置文件](#properties))
+==================== dubbo invoke 102 end ====================
+[10/08/22 07:16:08:008 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.18:50052, client: 172.18.96.18, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-```yaml
-livenessProbe:
- httpGet:
- path: /live
- port: 22222
- initialDelaySeconds: 5
- periodSeconds: 5
-readinessProbe:
- httpGet:
- path: /ready
- port: 22222
- initialDelaySeconds: 5
- periodSeconds: 5
-startupProbe:
- httpGet:
- path: /startup
- port: 22222
- failureThreshold: 30
- periodSeconds: 10
-```
+==================== dubbo invoke 103 end ====================
+[10/08/22 07:16:13:013 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v2: 172.18.96.6:50052, client: 172.18.96.6, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-**注意：** 开启 sidecar 后，需在 Deployment 中加入如下注解配置，才能保证探针不被 istio 重写指向 envoy：
+==================== dubbo invoke 104 end ====================
+[10/08/22 07:16:18:018 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.22:50052, client: 172.18.96.22, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-```yaml
- template:
- metadata:
- labels:
- app: dubbo-samples-mesh-provider
- annotations:
- # Prevent istio rewrite http probe
- sidecar.istio.io/rewriteAppHTTPProbers: "false"
-```
+==================== dubbo invoke 105 end ====================
+[10/08/22 07:16:24:024 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.18:50052, client: 172.18.96.18, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-**结果：**
+==================== dubbo invoke 106 end ====================
+[10/08/22 07:16:29:029 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.22:50052, client: 172.18.96.22, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-查看istio-proxy日志如下：
+==================== dubbo invoke 107 end ====================
+[10/08/22 07:16:34:034 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.18:50052, client: 172.18.96.18, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-```shell
-[2022-07-22T10:51:43.535Z] "GET /live HTTP/1.1" 200 - via_upstream - "-" 0 4 1 1 "-" "kube-probe/1.23"
-"b5c9971f-c19c-9152-9889-fa2221d7d164" "172.17.0.8:22222" "172.17.0.8:22222"
-inbound|22222|| 127.0.0.6:53249 172.17.0.8:22222 172.17.0.1:34936 - default
-```
+==================== dubbo invoke 108 end ====================
+[10/08/22 07:16:39:039 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.22:50052, client: 172.18.96.22, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-```shell
-[2022-07-22T10:25:58.535Z] "GET /startup HTTP/1.1" 200 - via_upstream - "-" 0 4 55 54 "-" "kube-probe/1.23"
-"7e43c75f-4e36-9102-986a-0c0dac1bb5fd" "172.17.0.8:22222" "172.17.0.8:22222"
-inbound|22222|| 127.0.0.6:56057 172.17.0.8:22222 172.17.0.1:40044 - default
-```
+==================== dubbo invoke 109 end ====================
+[10/08/22 07:16:44:044 UTC] main INFO action.GreetingServiceConsumer: consumer Unary reply <-message: "hello,service mesh, response from provider-v1: 172.18.96.18:50052, client: 172.18.96.18, local: dubbo-samples-mesh-provider, remote: null, isProviderSide: true"
-<h3 id="health_check_envoy">4.2 Envoy 主动健康检查</h3>
-
-本次从 consumer 侧的 Envoy ，对其上游集群 provider 做主动健康检查(GRPC)。
-
-配置EnvoyFilter如下：
-
-```yaml
-apiVersion: networking.istio.io/v1alpha3
-kind: EnvoyFilter
-metadata:
- name: cluster
- namespace: dubbo-demo
-spec:
- workloadSelector:
- labels:
- app: dubbo-samples-mesh-consumer
- configPatches:
- - applyTo: CLUSTER
- match:
- cluster:
- name: outbound|50052||dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local
- patch:
- operation: MERGE
- value:
- health_checks:
- - timeout: 5s
- interval: 5s
- initial_jitter: 1s
- interval_jitter: 1s
- interval_jitter_percent: 50
- unhealthy_threshold: 1
- healthy_threshold: 1
- reuse_connection: true
- no_traffic_interval: 2s
- no_traffic_healthy_interval: 4s
- unhealthy_interval: 5s
- unhealthy_edge_interval: 10s
- healthy_edge_interval: 10s
- tls_options:
- alpn_protocols:
- - http1.1
- - h2
- transport_socket_match_criteria:
- useMTLS: true
- grpc_health_check:
- authority: dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local
-```
-Envoy健康检查的配置说明(
-详见[Envoy 健康检查文档](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/health_check.proto#envoy-v3-api-msg-config-core-v3-healthcheck)):
-
-```yaml
-{
- "timeout": "{...}",超时时间
- "interval": "{...}",检测间隔
- "initial_jitter": "{...}",初始抖动
- "interval_jitter": "{...}",间隔抖动
- "interval_jitter_percent": "...",间隔抖动比例
- "unhealthy_threshold": "{...}",不健康阈值
- "healthy_threshold": "{...}",健康阈值
- "reuse_connection": "{...}",重用连接
- "http_health_check": "{...}",http类型健康检测
- "tcp_health_check": "{...}",tcp类型健康检测
- "grpc_health_check": "{...}",grpc类型健康检测
- "custom_health_check": "{...}",自定义健康检测
- "no_traffic_interval": "{...}",没有流量时的间隔
- "no_traffic_healthy_interval": "{...}",没有流量健康后的间隔
- "unhealthy_interval": "{...}",不健康的间隔
- "unhealthy_edge_interval": "{...}",不健康边缘间隔
- "healthy_edge_interval": "{...}",健康边缘间隔
- "event_log_path": "...",日志路径
- "always_log_health_check_failures": "...",失败总是记录日志
- "tls_options": "{...}",选项
- "transport_socket_match_criteria": "{...}"trasport_socket匹配条件
-}
```
-**实验结果**
+#### 查看 dashboard
+Istio 官网查看 [如何启动 dashboard](https://istio.io/latest/docs/setup/getting-started/#dashboard)。
-- 首先按照步骤 3.4 启动好 provider 和 consumer。
+<h3 id="image">4 修改示例</h3>
-运行 `kubectl apply -f https://raw.githubusercontent.com/apache/dubbo-samples/master/dubbo-samples-mesh-k8s/deploy/EnvoyFilter.yml`
+> 1. 注意项目存储路径一定是英文，否则 protobuf 编译失败。
+> 2. 以为应用开发与打包的指引说明。
-- 随后查看 provider 侧的 istio-proxy 日志，可观察到 provider 接收到 GRPC 健康检查：
+修改 Dubbo Provider 配置 `dubbo-provider.properties`
-```bash
-[2022-07-08T09:03:34.152Z] "POST /grpc.health.v1.Health/Check HTTP/2" 200 - via_upstream
- - "-" 5 7 5 5 "-" "Envoy/HC" "b33a9988-2c15-99b0-be0c-e085b64d77fa" "dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local" "172.17.0.7:50052"
- inbound|50052|| 127.0.0.6:44823 172.17.0.7:50052 172.17.0.8:53858 - default
+```properties
+# provider
+dubbo.application.name=dubbo-samples-mesh-provider
+dubbo.application.metadataServicePort=20885
+dubbo.registry.address=N/A
+dubbo.protocol.name=tri
+dubbo.protocol.port=50052
+dubbo.application.qosEnable=true
+# 为了使 Kubernetes 集群能够正常访问到探针，需要开启 QOS 允许远程访问，此操作有可能带来安全风险，请仔细评估后再打开
+dubbo.application.qosAcceptForeignIp=true
```
-**参考：**
+修改 Dubbo Consumer 配置 `dubbo-consumer.properties`
-- [Dubbo 探针文档](https://dubbo.apache.org/zh/docs/v3.0/references/lifecycle/brief/)
-- [Istio 服务的健康检查官方文档](https://istio.io/latest/zh/docs/ops/configuration/mesh/app-health-check/)
-- [Envoy 健康检查文档](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/health_check.proto#envoy-v3-api-msg-config-core-v3-healthcheck)
+```properties
+# consumer
+dubbo.application.name=dubbo-samples-mesh-consumer
+dubbo.application.metadataServicePort=20885
+dubbo.registry.address=N/A
+dubbo.protocol.name=tri
+dubbo.protocol.port=20880
+dubbo.consumer.timeout=30000
+dubbo.application.qosEnable=true
+# 为了使 Kubernetes 集群能够正常访问到探针，需要开启 QOS 允许远程访问，此操作有可能带来安全风险，请仔细评估后再打开
+dubbo.application.qosAcceptForeignIp=true
+# 标记开启 mesh sidecar 代理模式
+dubbo.consumer.meshEnable=true
+```
-<h2 id="next">5 下一步计划</h2>
+完成代码修改后，通过项目提供的 Dockerfile 打包镜像
-TODO
+```shell
+# 打包并推送镜像
+mvn compile jib:build
+```
-* 探索envoy和istio支持的服务治理的内容，比如开发者需要实现重试，API处要传什么值。
-* 解决目前readinessProbe不适用mesh模式的问题
-* 精简SDK。
+> Jib 插件会自动打包并发布镜像。注意，本地开发需将 jib 插件配置中的 docker registry 组织 dubboteam 改为自己有权限的组织（包括其他 kubernetes manifests 中的 dubboteam 也要修改，以确保 kubernetes 部署的是自己定制后的镜像），如遇到 jib 插件认证问题，请参考[相应链接](https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#what-should-i-do-when-the-registry-responds-with-unauthorized)配置 docker registry 认证信息。
+> 可以通过直接在命令行指定 `mvn compile jib:build -Djib.to.auth.username=x -Djib.to.auth.password=x -Djib.from.auth.username=x -Djib.from.auth.username=x`，或者使用 docker-credential-helper.
-<h2 id="common">6 常用命令</h2>
+<h2 id="common">5 常用命令</h2>
```shell
# dump current Envoy configs
diff --git a/dubbo-samples-mesh-k8s/assets/thinsdk.png b/dubbo-samples-mesh-k8s/assets/thinsdk.png
new file mode 100644
index 00000000..287409cc
Binary files /dev/null and b/dubbo-samples-mesh-k8s/assets/thinsdk.png differ
diff --git a/dubbo-samples-mesh-k8s/deploy/consumer/Deployment.yml b/dubbo-samples-mesh-k8s/deploy/consumer/Deployment.yml
index 48e86b45..bd5320b3 100644
--- a/dubbo-samples-mesh-k8s/deploy/consumer/Deployment.yml
+++ b/dubbo-samples-mesh-k8s/deploy/consumer/Deployment.yml
@@ -20,8 +20,8 @@ spec:
spec:
containers:
- name: server
- image: 15841721425/dubbo-samples-mesh-consumer-v1.0
- imagePullPolicy: IfNotPresent
+ image: dubboteam/dubbo-samples-mesh-consumer
+ imagePullPolicy: Always
ports:
- name: grpc-tri
containerPort: 50052
diff --git a/dubbo-samples-mesh-k8s/deploy/provider/Deployment.yml b/dubbo-samples-mesh-k8s/deploy/provider/Deployment-v2.yml
similarity index 85%
copy from dubbo-samples-mesh-k8s/deploy/provider/Deployment.yml
copy to dubbo-samples-mesh-k8s/deploy/provider/Deployment-v2.yml
index 11953fe2..a04b89b0 100644
--- a/dubbo-samples-mesh-k8s/deploy/provider/Deployment.yml
+++ b/dubbo-samples-mesh-k8s/deploy/provider/Deployment-v2.yml
@@ -1,27 +1,27 @@
apiVersion: apps/v1
kind: Deployment
metadata:
- name: dubbo-samples-mesh-provider
+ name: dubbo-samples-mesh-provider-v2
namespace: dubbo-demo
spec:
- replicas: 3
+ replicas: 2
selector:
matchLabels:
app: dubbo-samples-mesh-provider
- version: v1
+ version: v2
template:
metadata:
labels:
app: dubbo-samples-mesh-provider
- version: v1
+ version: v2
annotations:
# Prevent istio rewrite http probe
sidecar.istio.io/rewriteAppHTTPProbers: "false"
spec:
containers:
- name: server
- image: 15841721425/dubbo-samples-mesh-provider-v1.0
- imagePullPolicy: IfNotPresent
+ image: dubboteam/dubbo-samples-mesh-provider-v2:0.1
+ imagePullPolicy: Always
ports:
- name: grpc-tri
containerPort: 50052
diff --git a/dubbo-samples-mesh-k8s/deploy/provider/Deployment.yml b/dubbo-samples-mesh-k8s/deploy/provider/Deployment.yml
index 11953fe2..1f449f37 100644
--- a/dubbo-samples-mesh-k8s/deploy/provider/Deployment.yml
+++ b/dubbo-samples-mesh-k8s/deploy/provider/Deployment.yml
@@ -1,10 +1,10 @@
apiVersion: apps/v1
kind: Deployment
metadata:
- name: dubbo-samples-mesh-provider
+ name: dubbo-samples-mesh-provider-v1
namespace: dubbo-demo
spec:
- replicas: 3
+ replicas: 2
selector:
matchLabels:
app: dubbo-samples-mesh-provider
@@ -20,8 +20,8 @@ spec:
spec:
containers:
- name: server
- image: 15841721425/dubbo-samples-mesh-provider-v1.0
- imagePullPolicy: IfNotPresent
+ image: dubboteam/dubbo-samples-mesh-provider-v1:0.1
+ imagePullPolicy: Always
ports:
- name: grpc-tri
containerPort: 50052
diff --git a/dubbo-samples-mesh-k8s/deploy/provider/Service.yml b/dubbo-samples-mesh-k8s/deploy/provider/Service.yml
index dd6fa139..761c9250 100644
--- a/dubbo-samples-mesh-k8s/deploy/provider/Service.yml
+++ b/dubbo-samples-mesh-k8s/deploy/provider/Service.yml
@@ -12,37 +12,3 @@ spec:
- name: grpc-tri
port: 50052
targetPort: 50052
-
----
-apiVersion: networking.istio.io/v1alpha3
-kind: VirtualService
-metadata:
- name: dubbo-samples-mesh-provider
- namespace: dubbo-demo
-spec:
- hosts:
- - dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local
- http:
- - route:
- - destination:
- host: dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local
- subset: v1
- port:
- # Specifies the port on the host being addressed. If the service exposes only one port, you don't need to choose the port explicitly
- number: 50052
----
-apiVersion: networking.istio.io/v1alpha3
-kind: DestinationRule
-metadata:
- name: dubbo-samples-mesh-provider
- namespace: dubbo-demo
-spec:
- host: dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local
- subsets:
- - name: v1
- labels:
- version: v1
- trafficPolicy:
- loadBalancer:
- # Envoy load balancing strategy
- simple: ROUND_ROBIN
\ No newline at end of file
diff --git a/dubbo-samples-mesh-k8s/deploy/provider/Service.yml b/dubbo-samples-mesh-k8s/deploy/traffic/virtual-service.yml
similarity index 61%
copy from dubbo-samples-mesh-k8s/deploy/provider/Service.yml
copy to dubbo-samples-mesh-k8s/deploy/traffic/virtual-service.yml
index dd6fa139..d075cc1f 100644
--- a/dubbo-samples-mesh-k8s/deploy/provider/Service.yml
+++ b/dubbo-samples-mesh-k8s/deploy/traffic/virtual-service.yml
@@ -1,18 +1,3 @@
-apiVersion: v1
-kind: Service
-metadata:
- name: dubbo-samples-mesh-provider
- namespace: dubbo-demo
-spec:
- type: ClusterIP
- sessionAffinity: None
- selector:
- app: dubbo-samples-mesh-provider
- ports:
- - name: grpc-tri
- port: 50052
- targetPort: 50052
-
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
@@ -30,6 +15,15 @@ spec:
port:
# Specifies the port on the host being addressed. If the service exposes only one port, you don't need to choose the port explicitly
number: 50052
+ weight: 80
+ - destination:
+ host: dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local
+ subset: v2
+ port:
+ # Specifies the port on the host being addressed. If the service exposes only one port, you don't need to choose the port explicitly
+ number: 50052
+ weight: 20
+
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
@@ -38,11 +32,14 @@ metadata:
namespace: dubbo-demo
spec:
host: dubbo-samples-mesh-provider.dubbo-demo.svc.cluster.local
+ trafficPolicy:
+ loadBalancer:
+ # Envoy load balancing strategy
+ simple: ROUND_ROBIN
subsets:
- name: v1
labels:
version: v1
- trafficPolicy:
- loadBalancer:
- # Envoy load balancing strategy
- simple: ROUND_ROBIN
\ No newline at end of file
+ - name: v2
+ labels:
+ version: v2
\ No newline at end of file
diff --git a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-consumer/pom.xml b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-consumer/pom.xml
index 252b529f..3c580ecd 100644
--- a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-consumer/pom.xml
+++ b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-consumer/pom.xml
@@ -194,6 +194,34 @@
</execution>
</executions>
</plugin>
+ <plugin>
+ <groupId>com.google.cloud.tools</groupId>
+ <artifactId>jib-maven-plugin</artifactId>
+ <version>3.2.1</version>
+ <configuration>
+ <to>
+ <image>dubboteam/dubbo-samples-mesh-consumer</image>
+ </to>
+ <container>
+ <jvmFlags>
+ <jvmFlag>-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=31000</jvmFlag>
+ </jvmFlags>
+ <ports>
+ <port>20880</port>
+ <port>31000</port>
+ <port>22222</port>
+ </ports>
+ </container>
+ </configuration>
+ <!-- <executions>-->
+ <!-- <execution>-->
+ <!-- <phase>package</phase>-->
+ <!-- <goals>-->
+ <!-- <goal>build</goal>-->
+ <!-- </goals>-->
+ <!-- </execution>-->
+ <!-- </executions>-->
+ </plugin>
</plugins>
</build>
diff --git a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-consumer/src/main/java/org/apache/dubbo/samples/ConsumerBootstrap.java b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-consumer/src/main/java/org/apache/dubbo/samples/ConsumerBootstrap.java
index 33b584d5..a249d933 100644
--- a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-consumer/src/main/java/org/apache/dubbo/samples/ConsumerBootstrap.java
+++ b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-consumer/src/main/java/org/apache/dubbo/samples/ConsumerBootstrap.java
@@ -37,14 +37,12 @@ public class ConsumerBootstrap {
GreetingServiceConsumer greetingServiceConsumer = context.getBean(GreetingServiceConsumer.class);
AtomicInteger count = new AtomicInteger(0);
- for (int i = 0; i < 30; i++) {
+ while (true) {
greetingServiceConsumer.doSayHello("service mesh");
System.out.println("==================== dubbo invoke " + count.get() + " end ====================");
count.getAndIncrement();
- Thread.sleep(10000);
+ Thread.sleep(5000);
}
-
- new CountDownLatch(1).await();
}
@Configuration
diff --git a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/pom.xml b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/pom.xml
index 00737bbc..91c16400 100644
--- a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/pom.xml
+++ b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/pom.xml
@@ -176,6 +176,34 @@
</execution>
</executions>
</plugin>
+ <plugin>
+ <groupId>com.google.cloud.tools</groupId>
+ <artifactId>jib-maven-plugin</artifactId>
+ <version>3.2.1</version>
+ <configuration>
+ <to>
+ <image>dubboteam/dubbo-samples-mesh-provider-v1:0.1</image>
+ </to>
+ <container>
+ <jvmFlags>
+ <jvmFlag>-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=31000</jvmFlag>
+ </jvmFlags>
+ <ports>
+ <port>50052</port>
+ <port>31000</port>
+ <port>22222</port>
+ </ports>
+ </container>
+ </configuration>
+ <!-- <executions>-->
+ <!-- <execution>-->
+ <!-- <phase>package</phase>-->
+ <!-- <goals>-->
+ <!-- <goal>build</goal>-->
+ <!-- </goals>-->
+ <!-- </execution>-->
+ <!-- </executions>-->
+ </plugin>
</plugins>
</build>
diff --git a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/src/main/java/org/apache/dubbo/samples/impl/GreeterImpl.java b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/src/main/java/org/apache/dubbo/samples/impl/GreeterImpl.java
index 7a1115f4..13cb6984 100644
--- a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/src/main/java/org/apache/dubbo/samples/impl/GreeterImpl.java
+++ b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/src/main/java/org/apache/dubbo/samples/impl/GreeterImpl.java
@@ -58,9 +58,9 @@ public class GreeterImpl extends DubboGreeterTriple.GreeterImplBase {
String application = RpcContext.getContext().getUrl().getParameter("application");
String protocol = RpcContext.getContext().getProtocol();
return GreeterReply.newBuilder()
- .setMessage("hello," + request.getName() + ", response from provider: " + RpcContext.getContext().getLocalAddress() +
- ", client: " + clientIP + ", local: " + application + ", remote: " + remoteApplication +
- ", isProviderSide: " + isProviderSide)
+ .setMessage("hello," + request.getName() + ", response from provider-v1: " + RpcContext.getContext().getLocalAddress() +
+ ", client: " + clientIP + ", local: " + application + ", remote: " + remoteApplication +
+ ", isProviderSide: " + isProviderSide)
.build();
}
diff --git a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/src/main/resources/spring/dubbo-provider.properties b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/src/main/resources/spring/dubbo-provider.properties
index bc54a7e8..4716ea28 100644
--- a/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/src/main/resources/spring/dubbo-provider.properties
+++ b/dubbo-samples-mesh-k8s/dubbo-samples-mesh-provider/src/main/resources/spring/dubbo-provider.properties
@@ -21,6 +21,6 @@ dubbo.application.metadataServicePort=20885
dubbo.registry.address=N/A
dubbo.protocol.name=tri
dubbo.protocol.port=50052
-# qos??
+# qos
dubbo.application.qosEnable=true
dubbo.application.qosAcceptForeignIp=true
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org