You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/02/16 01:17:26 UTC
[GitHub] [apisix-ingress-controller] tao12345666333 opened a new issue #870: bug: resolveGranularity: service fails, when restarted multiple times
tao12345666333 opened a new issue #870:
URL: https://github.com/apache/apisix-ingress-controller/issues/870
I found that `resolveGranularity: service ` no longer takes effect after multiple restarts of the APISIX Ingress controller
xref: #853
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] MirtoBusico commented on issue #870: bug: resolveGranularity: service fails, when restarted multiple times
Posted by GitBox <gi...@apache.org>.
MirtoBusico commented on issue #870:
URL: https://github.com/apache/apisix-ingress-controller/issues/870#issuecomment-1041491421
Hi @tao12345666333 the problem appeared after 5 cluster restarts
How I've done the tests:
1) start the cluster and wait everything goes green
2) inspect the upstream address with apisix dashboard
3) shutdown the cluster.
I've done this 5 times. In the first four times in the upstream there is the service address; but in the fifth attempt appears the POD address
In apisix dashboard
<img width="1714" alt="apisix_pod_address" src="https://user-images.githubusercontent.com/11090934/154269893-886ca705-9125-4b09-871d-15772f8308f8.png">
The kubernetes addresses
<img width="1126" alt="apisix_svc-pod-addresses" src="https://user-images.githubusercontent.com/11090934/154269950-4c996a13-58c7-46aa-8dc6-e2ecb70a03a6.png">
Nothing in the apisix log (UTC time -1 hour respect to dashboard):
```
2022-02-16T12:46:37.271Z | 127.0.0.6 - - [16/Feb/2022:12:46:34 +0000] apisix-admin.apisix.svc.cluster.local:9180 "GET /apisix/admin/upstreams/f3118c88 HTTP/1.1" 404 39 0.002 "-" "Go-http-client/1.1" - - - "http://apisix-admin.apisix.svc.cluster.local:9180"
2022-02-16T12:46:37.271Z | 127.0.0.6 - - [16/Feb/2022:12:46:34 +0000] apisix-admin.apisix.svc.cluster.local:9180 "GET /apisix/admin/upstreams/23ef4d0 HTTP/1.1" 404 39 0.002 "-" "Go-http-client/1.1" - - - "http://apisix-admin.apisix.svc.cluster.local:9180"
2022-02-16T12:46:37.271Z | 127.0.0.6 - - [16/Feb/2022:12:46:34 +0000] apisix-admin.apisix.svc.cluster.local:9180 "GET /apisix/admin/upstreams/4a5806af HTTP/1.1" 404 39 0.002 "-" "Go-http-client/1.1" - - - "http://apisix-admin.apisix.svc.cluster.local:9180"
2022-02-16T12:46:37.271Z | 127.0.0.6 - - [16/Feb/2022:12:46:34 +0000] apisix-admin.apisix.svc.cluster.local:9180 "GET /apisix/admin/upstreams/6b312a6d HTTP/1.1" 404 39 0.003 "-" "Go-http-client/1.1" - - - "http://apisix-admin.apisix.svc.cluster.local:9180"
2022-02-16T12:46:37.271Z | 127.0.0.6 - - [16/Feb/2022:12:46:34 +0000] apisix-admin.apisix.svc.cluster.local:9180 "GET /apisix/admin/upstreams/7dc55665 HTTP/1.1" 404 39 0.002 "-" "Go-http-client/1.1" - - - "http://apisix-admin.apisix.svc.cluster.local:9180"
```
Nothing in the apisix ingress controller log:
```2022-02-16T13:05:29.633Z | [GIN] 2022/02/16 - 21:05:29 | 200 | 26.58µs | 127.0.0.6 | GET "/healthz"
2022-02-16T13:05:35.738Z | [GIN] 2022/02/16 - 21:05:35 | 200 | 38.15µs | 127.0.0.6 | GET "/healthz"
2022-02-16T13:05:39.633Z | [GIN] 2022/02/16 - 21:05:39 | 200 | 27.66µs | 127.0.0.6 | GET "/healthz"
2022-02-16T13:05:45.738Z | [GIN] 2022/02/16 - 21:05:45 | 200 | 38.349µs | 127.0.0.6 | GET "/healthz"
2022-02-16T13:05:49.633Z | [GIN] 2022/02/16 - 21:05:49 | 200 | 39.56µs | 127.0.0.6 | GET "/healthz"
```
Recap of framework and installation:
1) three virtual machines cluster with K3S
```
sysop@m01serv:~$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
m01kw1 Ready <none> 28d v1.22.5+k3s1 192.168.102.122 <none> Ubuntu 20.04.3 LTS 5.4.0-99-generic containerd://1.5.8-k3s1
m01kw2 Ready <none> 28d v1.22.5+k3s1 192.168.102.123 <none> Ubuntu 20.04.3 LTS 5.4.0-99-generic containerd://1.5.8-k3s1
m01km Ready control-plane,master 28d v1.22.5+k3s1 192.168.102.121 <none> Ubuntu 20.04.3 LTS 5.4.0-99-generic containerd://1.5.8-k3s1
sysop@m01serv:~$
```
2) Istio version 1.12.2
```
sysop@m01serv:~$ istioctl version
client version: 1.12.2
control plane version: 1.12.2
data plane version: 1.12.2 (16 proxies)
sysop@m01serv:~$
```
3) Helm chart version
```
sysop@m01serv:~$ helm show chart apisix/apisix
apiVersion: v2
appVersion: 2.12.0
dependencies:
- condition: etcd.enabled
name: etcd
repository: https://charts.bitnami.com/bitnami
version: 6.2.6
- alias: dashboard
condition: dashboard.enabled
name: apisix-dashboard
repository: https://charts.apiseven.com
version: 0.4.0
- alias: ingress-controller
condition: ingress-controller.enabled
name: apisix-ingress-controller
repository: https://charts.apiseven.com
version: 0.9.0
description: A Helm chart for Apache APISIX
icon: https://apache.org/logos/res/apisix/apisix.png
maintainers:
- name: tao12345666333
name: apisix
type: application
version: 0.8.2
sysop@m01serv:~$
```
4) changes in values.yaml
```
...
gateway:
type: LoadBalancer
...
tls:
enabled: true
servicePort: 443
containerPort: 9443
existingCASecret: "m01cacert"
certCAFilename: "cert"
http2:
enabled: true
...
discovery:
enabled: true
registry:
dns:
servers:
- "10.43.0.10:53"
...
dashboard:
enabled: true
ingress-controller:
enabled: true
```
5) Apisix installation:
```
kubectl create ns apisix
kubectl label namespace apisix istio-injection=enabled
kubectl -n apisix create secret generic m01cacert --from-file=cert=./m01ca.pem
helm install apisix apisix/apisix -f apisix-values.yaml \
--set ingress-controller.config.apisix.serviceNamespace=apisix \
--set ingress-controller.config.apisix.serviceName=apisix-admin \
--set ingress-controller.config.kubernetes.apisixRouteVersion=apisix.apache.org/v2beta3 \
--namespace apisix
```
6) route definition:
```
sysop@m01serv:~/software/apisisx$ cat productpage-ar.yaml
apiVersion: apisix.apache.org/v2beta3
kind: ApisixRoute
metadata:
name: productpage
spec:
http:
- name: rule1
match:
hosts:
- www2.m01.net
paths:
- /*
backends:
- serviceName: productpage
servicePort: 9080
resolveGranularity: service
sysop@m01serv:~/software/apisisx$
```
7) route installation:
```
kubectl -n bookinfo apply -f productpage-ar.yaml
```
8) Get upstream from apisix pod shell
```
bash-5.1# curl http://127.0.0.1:9180/apisix/admin/upstreams/49b914ed -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X GET
{"action":"get","count":1,"node":{"value":{"name":"bookinfo_productpage_9080","update_time":1645015594,"labels":{"managed-by":"apisix-ingress-controller"},"nodes":[{"port":9080,"weight":100,"priority":0,"host":"10.42.1.111"}],"id":"49b914ed","desc":"Created by apisix-ingress-controller, DO NOT modify it manually","hash_on":"vars","scheme":"http","type":"roundrobin","pass_host":"pass","create_time":1644952229},"key":"\/apisix\/upstreams\/49b914ed"}}
bash-5.1#
```
Hope this helps.
Can I inspect anithing else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #870: bug: resolveGranularity: service fails, when restarted multiple times
Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #870:
URL: https://github.com/apache/apisix-ingress-controller/issues/870#issuecomment-1041845810
@MirtoBusico Thanks!
I can reproduce the problem by restarting the APISIX Ingress controller multiple times. I also plan to fix this bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org