You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/10/27 10:33:57 UTC
[GitHub] [apisix-ingress-controller] dickens7 opened a new issue, #1412: bug: failed to reflect pluginConfig delete to cache: still in use
dickens7 opened a new issue, #1412:
URL: https://github.com/apache/apisix-ingress-controller/issues/1412
### Issue description
An exception occurs when ApisixPluginConfig is used by multiple ApisixRoute or Ingress
https://github.com/apache/apisix-ingress-controller/blob/v1.5.0/pkg/ingress/apisix_route.go#L329-L336
https://github.com/apache/apisix-ingress-controller/blob/v1.5.0/pkg/kube/translation/apisix_route.go#L924-L927
### Environment
- your apisix-ingress-controller version (output of apisix-ingress-controller version --long):
Version: 1.5.0
Git SHA: no-git-module
Go Version: go1.19.2
Building OS/Arch: linux/amd64
Running OS/Arch: linux/amd64
- your Kubernetes cluster version (output of kubectl version):
v1.18.4+k3s1
- if you run apisix-ingress-controller in Bare-metal environment, also show your OS version (uname -a):
### Minimal test code / Steps to reproduce
1. create ApisixPluginConfig
```yaml
apiVersion: apisix.apache.org/v2
kind: ApisixPluginConfig
metadata:
name: cors-plugins
spec:
plugins:
- name: cors
enable: true
```
2. create ApisixRoute `httpserver` and `httpserver2`
```yaml
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
name: httpserver
spec:
http:
- name: httpserver
match:
hosts:
- local.httpbin.org
paths:
- /
backends:
- serviceName: httpbin
servicePort: 80
plugin_config_name: cors-plugins
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
name: httpserver2
spec:
http:
- name: httpserver
match:
hosts:
- local.httpbin.org
paths:
- /
backends:
- serviceName: httpbin
servicePort: 80
plugin_config_name: cors-plugins
```
### Actual result
ApisixPluginConfig will be deleted when ApisixRoute or Ingress is updated and deleted
### Error log
```shell
2022-10-27T18:20:23+08:00 info utils/manifest.go:361 plugin_config was referenced by other routes {"plugin_config_id": "9c092931", "plugin_config_name": ""}
2022-10-27T18:20:23+08:00 debug ingress/apisix_route.go:507 ApisixRoute update event arrived {"key": "xxx-env-dev/oa2-orgfront", "new object": {"metadata":{"name":"oa2-orgfront","namespace":"xxx-env-dev","selfLink":"/apis/apisix.apache.org/v2/namespaces/xxx-env-dev/apisixroutes/oa2-orgfront","uid":"2f2036ed-f973-4961-9534-3ce24627cec5","resourceVersion":"181495831","generation":4,"creationTimestamp":"2022-06-24T03:06:34Z","labels":{"app.kubernetes.io/managed-by":"Helm"},"annotations":{"meta.helm.sh/release-name":"oa2-orgfront","meta.helm.sh/release-namespace":"xxx-env-dev"},"managedFields":[{"manager":"apisix-ingress-controller","operation":"Update","apiVersion":"apisix.apache.org/v2beta3","time":"2022-10-15T10:05:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{}}},{"manager":"aslan","operation":"Update","apiVersion":"apisix.apache.org/v2beta3","time":"2022-10-15T10:05:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:meta.he
lm.sh/release-name":{},"f:meta.helm.sh/release-namespace":{}},"f:labels":{".":{},"f:app.kubernetes.io/managed-by":{}}},"f:spec":{}}},{"manager":"apisix-ingress-controller","operation":"Update","apiVersion":"apisix.apache.org/v2","time":"2022-10-27T10:20:22Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:http":{}},"f:status":{"f:conditions":{}}}}]},"spec":{"http":[{"name":"oa2-orgfront","match":{"paths":["/","/*"],"hosts":["org.k3s-jcyfb-dev.xxx.cn"]},"backends":[{"serviceName":"oa2-orgfront","servicePort":80,"weight":null}],"websocket":false,"plugin_config_name":"oa2-front-plugins","authentication":{"enable":false,"type":"","keyAuth":{},"jwtAuth":{}}}]},"status":{"conditions":[{"type":"ResourcesAvailable","status":"True","observedGeneration":4,"lastTransitionTime":null,"reason":"ResourcesSynced","message":"Sync Successfully"}]}}, "old object": {"metadata":{"name":"oa2-orgfront","namespace":"xxx-env-dev","selfLink":"/apis/apisix.apache.org/v2/namespaces/xxx-env-dev/apisixroutes/oa
2-orgfront","uid":"2f2036ed-f973-4961-9534-3ce24627cec5","resourceVersion":"181495839","generation":4,"creationTimestamp":"2022-06-24T03:06:34Z","labels":{"app.kubernetes.io/managed-by":"Helm"},"annotations":{"meta.helm.sh/release-name":"oa2-orgfront","meta.helm.sh/release-namespace":"xxx-env-dev"},"managedFields":[{"manager":"apisix-ingress-controller","operation":"Update","apiVersion":"apisix.apache.org/v2beta3","time":"2022-10-15T10:05:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{}}},{"manager":"aslan","operation":"Update","apiVersion":"apisix.apache.org/v2beta3","time":"2022-10-15T10:05:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:meta.helm.sh/release-name":{},"f:meta.helm.sh/release-namespace":{}},"f:labels":{".":{},"f:app.kubernetes.io/managed-by":{}}},"f:spec":{}}},{"manager":"apisix-ingress-controller","operation":"Update","apiVersion":"apisix.apache.org/v2","time":"2022-10-27T10:20:23Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec
":{"f:http":{}},"f:status":{"f:conditions":{}}}}]},"spec":{"http":[{"name":"oa2-orgfront","match":{"paths":["/","/*"],"hosts":["org.k3s-jcyfb-dev.xxx.cn"]},"backends":[{"serviceName":"oa2-orgfront","servicePort":80,"weight":null}],"websocket":false,"plugin_config_name":"oa2-front-plugins","authentication":{"enable":false,"type":"","keyAuth":{},"jwtAuth":{}}}]},"status":{"conditions":[{"type":"ResourcesAvailable","status":"True","observedGeneration":4,"lastTransitionTime":null,"reason":"ResourcesSynced","message":"Sync Successfully"}]}}}
2022-10-27T18:20:23+08:00 debug apisix/pluginconfig.go:47 try to look up pluginConfig {"name": "xxx-env-dev_oa2-plugins", "url": "http://apisix-internal.ingress-apisix.svc.cluster.local:9180/apisix/admin/plugin_configs", "cluster": "default"}
2022-10-27T18:20:23+08:00 debug ingress/apisix_route.go:304 translated ApisixRoute {"routes": [{"id":"1eedc198","name":"xxx-env-dev_oa2-org_oa2-org","desc":"Created by apisix-ingress-controller, DO NOT modify it manually","labels":{"managed-by":"apisix-ingress-controller"},"hosts":["oa2.k3s-jcyfb-dev.xxx.cn"],"uris":["/org/api/","/org/api/*","/workflow/api/","/workflow/api/*"],"upstream_id":"f9f67824","plugin_config_id":"a7a986af"}], "upstreams": [{"id":"f9f67824","name":"xxx-env-dev_oa2-org_8888","desc":"Created by apisix-ingress-controller, DO NOT modify it manually","labels":{"managed-by":"apisix-ingress-controller"},"type":"roundrobin","nodes":[{"host":"10.42.1.156","port":8888,"weight":100}],"scheme":"http"}], "apisix_route": {}, "pluginConfigs": null}
2022-10-27T18:20:23+08:00 debug apisix/route.go:47 try to look up route {"name": "xxx-env-dev_oa2-org_oa2-org", "url": "http://apisix-internal.ingress-apisix.svc.cluster.local:9180/apisix/admin/routes", "cluster": "default"}
2022-10-27T18:20:23+08:00 debug apisix/upstream.go:203 try to update upstream {"id": "f9f67824", "name": "xxx-env-dev_oa2-org_8888", "cluster": "default", "url": "http://apisix-internal.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams"}
2022-10-27T18:20:23+08:00 debug apisix/upstreamservicerelation.go:95 try to create upstreamService in cache {"cluster": "default"}
2022-10-27T18:20:23+08:00 debug apisix/upstream.go:223 updating upstream {"body": "{\"id\":\"f9f67824\",\"name\":\"xxx-env-dev_oa2-org_8888\",\"desc\":\"Created by apisix-ingress-controller, DO NOT modify it manually\",\"labels\":{\"managed-by\":\"apisix-ingress-controller\"},\"type\":\"roundrobin\",\"nodes\":[{\"host\":\"10.42.1.156\",\"port\":8888,\"weight\":100}],\"scheme\":\"http\"}", "url": "http://apisix-internal.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/f9f67824"}
2022-10-27T18:20:23+08:00 debug apisix/resource.go:123 got upstream: {"pass_host":"pass","nodes":[{"port":8888,"host":"10.42.1.156","priority":0,"weight":100}],"hash_on":"vars","desc":"Created by apisix-ingress-controller, DO NOT modify it manually","name":"xxx-env-dev_oa2-org_8888","update_time":1666866023,"labels":{"managed-by":"apisix-ingress-controller"},"id":"f9f67824","create_time":1656040035,"scheme":"http","type":"roundrobin"}
2022-10-27T18:20:23+08:00 debug apisix/route.go:205 try to update route {"id": "1eedc198", "name": "xxx-env-dev_oa2-org_oa2-org", "cluster": "default", "url": "http://apisix-internal.ingress-apisix.svc.cluster.local:9180/apisix/admin/routes"}
2022-10-27T18:20:23+08:00 debug apisix/route.go:219 updating route {"body": "{\"id\":\"1eedc198\",\"name\":\"xxx-env-dev_oa2-org_oa2-org\",\"desc\":\"Created by apisix-ingress-controller, DO NOT modify it manually\",\"labels\":{\"managed-by\":\"apisix-ingress-controller\"},\"hosts\":[\"oa2.k3s-jcyfb-dev.xxx.cn\"],\"uris\":[\"/org/api/\",\"/org/api/*\",\"/workflow/api/\",\"/workflow/api/*\"],\"upstream_id\":\"f9f67824\",\"plugin_config_id\":\"a7a986af\"}", "url": "http://apisix-internal.ingress-apisix.svc.cluster.local:9180/apisix/admin/routes/1eedc198"}
2022-10-27T18:20:23+08:00 debug apisix/resource.go:93 got route: {"hosts":["oa2.k3s-jcyfb-dev.xxx.cn"],"desc":"Created by apisix-ingress-controller, DO NOT modify it manually","upstream_id":"f9f67824","plugin_config_id":"a7a986af","update_time":1666866023,"create_time":1658209424,"priority":0,"labels":{"managed-by":"apisix-ingress-controller"},"id":"1eedc198","status":1,"uris":["\/org\/api\/","\/org\/api\/*","\/workflow\/api\/","\/workflow\/api\/*"],"name":"xxx-env-dev_oa2-org_oa2-org"}
2022-10-27T18:20:23+08:00 debug apisix/pluginconfig.go:180 try to delete pluginConfig {"id": "a7a986af", "name": "", "cluster": "default", "url": "http://apisix-internal.ingress-apisix.svc.cluster.local:9180/apisix/admin/plugin_configs"}
2022-10-27T18:20:23+08:00 error apisix/pluginconfig.go:191 failed to reflect pluginConfig delete to cache: still in use```
### Expected result
ApisixPluginConfig is not deleted when ApisixRoute or Ingress are updated and deleted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #1412: bug: failed to reflect pluginConfig delete to cache: still in use
Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #1412:
URL: https://github.com/apache/apisix-ingress-controller/issues/1412#issuecomment-1302632634
I saw your PR #1413 want to fix v1.5 branch.
But we should follow "upstream first", we should fix it in master branch.
https://github.com/apache/apisix-ingress-controller/blob/53f26c1b5c078b39b448f8adb7db27e662f5bd51/pkg/providers/apisix/translation/apisix_route.go#L984-L988
Could you open another PR to fix it (master branch)? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] dickens7 commented on issue #1412: bug: failed to reflect pluginConfig delete to cache: still in use
Posted by GitBox <gi...@apache.org>.
dickens7 commented on issue #1412:
URL: https://github.com/apache/apisix-ingress-controller/issues/1412#issuecomment-1293581271
https://github.com/apache/apisix-ingress-controller/blob/21f39e966dedb0765a9848302f8cb713aa461cfe/pkg/ingress/apisix_route.go#L329-L339
- oldCtx
https://github.com/apache/apisix-ingress-controller/blob/21f39e966dedb0765a9848302f8cb713aa461cfe/pkg/kube/translation/apisix_route.go#L924-L928
After the above code `m.Diff(om)` is executed, deleted.PluginConfigs is not empty;
### Expected result
ApisixPluginConfig is not deleted when ApisixRoute or Ingress are updated and deleted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] tao12345666333 closed issue #1412: bug: failed to reflect pluginConfig delete to cache: still in use
Posted by GitBox <gi...@apache.org>.
tao12345666333 closed issue #1412: bug: failed to reflect pluginConfig delete to cache: still in use
URL: https://github.com/apache/apisix-ingress-controller/issues/1412
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #1412: bug: failed to reflect pluginConfig delete to cache: still in use
Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #1412:
URL: https://github.com/apache/apisix-ingress-controller/issues/1412#issuecomment-1293479453
If pluginconfig is still used, we should not delete it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #1412: bug: failed to reflect pluginConfig delete to cache: still in use
Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #1412:
URL: https://github.com/apache/apisix-ingress-controller/issues/1412#issuecomment-1293480664
I see you describe the actual result. Can you describe the expected result?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org