You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/11/25 10:23:27 UTC

[GitHub] [apisix-ingress-controller] xiaokunliu opened a new issue #773: apisix ingress deployed on eks happend upstream not found error

xiaokunliu opened a new issue #773:
URL: https://github.com/apache/apisix-ingress-controller/issues/773


   ### Issue description
   
   ### Environment
   
    **./apisix-ingress-controller version --long**
   Version: 1.3.0
   Git SHA: no-git-module
   Go Version: go1.13.8
   Building OS/Arch: linux/amd64
   Running OS/Arch: linux/amd64
   
    **uname -a**
   Linux apisix-ingress-controller-6d979c9b4-sp9hw 5.4.129-63.229.amzn2.x86_64 #1 SMP Tue Jul 20 21:22:08 UTC 2021 x86_64 Linux
   
   **kubectl version**
   Client Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.4-eks-6b7464", GitCommit:"6b746440c04cb81db4426842b4ae65c3f7035e53", GitTreeState:"clean", BuildDate:"2021-03-19T19:35:50Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
   Server Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.7-eks-d88609", GitCommit:"d886092805d5cc3a47ed5cf0c43de38ce442dfcb", GitTreeState:"clean", BuildDate:"2021-07-31T00:29:12Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"linux/amd64"}
   
   
   i have changed the route version to **apisix.apache.org/v2beta2** , but also have some warn log which its upstrem not found as follows, how to fix it?
   
   2021-11-25T18:10:02+08:00	info	ingress/ingress.go:112	apisix ingress controller started
   2021-11-25T18:10:02+08:00	info	ingress/ingress.go:114	version:
   Version: 1.3.0
   Git SHA: no-git-module
   Go Version: go1.13.8
   Building OS/Arch: linux/amd64
   Running OS/Arch: linux/amd64
   2021-11-25T18:10:02+08:00	info	ingress/ingress.go:120	use configuration
   {
   	"cert_file": "/etc/webhook/certs/cert.pem",
   	"key_file": "/etc/webhook/certs/key.pem",
   	"log_level": "info",
   	"log_output": "stderr",
   	"http_listen": ":8080",
   	"https_listen": ":8443",
   	"enable_profiling": true,
   	"kubernetes": {
   		"kubeconfig": "",
   		"resync_interval": "6h0m0s",
   		"app_namespaces": [
   			"etcd-apisix",
   			"ingress-apisix",
   			"bigdata"
   		],
   		"election_id": "ingress-apisix-leader",
   		"ingress_class": "apisix",
   		"ingress_version": "networking/v1",
   		"watch_endpoint_slices": false,
   		"apisix_route_version": "apisix.apache.org/v2beta2"
   	},
   	"apisix": {
   		"default_cluster_name": "default",
   		"default_cluster_base_url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin",
   		"default_cluster_admin_key": "edd1c9f034335f136f87ad84b625c8f1",
   		"base_url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin",
   		"admin_key": "edd1c9f034335f136f87ad84b625c8f1"
   	}
   }
   2021-11-25T18:10:02+08:00	warn	api/server.go:71	failed to load x509 key pair, will not start admission server	{"Error": "open /etc/webhook/certs/cert.pem: no such file or directory", "CertFilePath": "/etc/webhook/certs/cert.pem", "KeyFilePath": "/etc/webhook/certs/key.pem"}
   I1125 18:10:02.323810       1 leaderelection.go:243] attempting to acquire leader lease ingress-apisix/ingress-apisix-leader...
   2021-11-25T18:10:02+08:00	warn	ingress/controller.go:328	found a new leader apisix-ingress-controller-7cb7dccd9f-rqjv9
   2021-11-25T18:10:02+08:00	info	ingress/controller.go:330	controller now is running as a candidate	{"namespace": "ingress-apisix", "pod": "apisix-ingress-controller-6d979c9b4-sp9hw"}
   [GIN] 2021/11/25 - 18:10:03 | 200 |     101.431µs |   172.31.19.242 | GET      "/healthz"
   [GIN] 2021/11/25 - 18:10:08 | 200 |      38.991µs |   172.31.19.242 | GET      "/healthz"
   [GIN] 2021/11/25 - 18:10:13 | 200 |       55.88µs |   172.31.19.242 | GET      "/healthz"
   [GIN] 2021/11/25 - 18:10:18 | 200 |      51.331µs |   172.31.19.242 | GET      "/healthz"
   [GIN] 2021/11/25 - 18:10:23 | 200 |      47.901µs |   172.31.19.242 | GET      "/healthz"
   2021-11-25T18:10:25+08:00	info	ingress/controller.go:290	LeaderElection	{"message": "apisix-ingress-controller-6d979c9b4-sp9hw became leader", "event_type": "Normal"}
   I1125 18:10:25.566916       1 leaderelection.go:253] successfully acquired lease ingress-apisix/ingress-apisix-leader
   2021-11-25T18:10:25+08:00	info	ingress/controller.go:370	controller tries to leading ...	{"namespace": "ingress-apisix", "pod": "apisix-ingress-controller-6d979c9b4-sp9hw"}
   2021-11-25T18:10:25+08:00	warn	apisix/cluster.go:304	waiting cluster default to ready, it may takes a while
   2021-11-25T18:10:25+08:00	info	apisix/cluster.go:344	syncing schema	{"cluster": "default"}
   2021-11-25T18:10:25+08:00	warn	ingress/controller.go:328	found a new leader apisix-ingress-controller-6d979c9b4-sp9hw
   2021-11-25T18:10:25+08:00	info	apisix/cluster.go:155	syncing cache	{"cluster": "default"}
   2021-11-25T18:10:25+08:00	info	apisix/cluster.go:159	cache synced	{"cost_time": "24.3895ms", "cluster": "default"}
   2021-11-25T18:10:25+08:00	warn	apisix/cluster.go:315	cluster default now is ready, cost time 28.966573ms
   W1125 18:10:25.608202       1 warnings.go:70] apisix.apache.org/v2beta1 ApisixRoute is deprecated; use apisix.apache.org/v2beta2 ApisixRoute
   W1125 18:10:25.609915       1 warnings.go:70] apisix.apache.org/v2beta1 ApisixRoute is deprecated; use apisix.apache.org/v2beta2 ApisixRoute
   W1125 18:10:25.614050       1 warnings.go:70] apisix.apache.org/v2beta1 ApisixRoute is deprecated; use apisix.apache.org/v2beta2 ApisixRoute
   2021-11-25T18:10:25+08:00	info	ingress/controller.go:482	controller now is running as leader	{"namespace": "ingress-apisix", "pod": "apisix-ingress-controller-6d979c9b4-sp9hw"}
   2021-11-25T18:10:25+08:00	info	ingress/apisix_consumer.go:56	ApisixConsumer controller started
   2021-11-25T18:10:25+08:00	info	ingress/endpoint.go:56	endpoints controller started
   2021-11-25T18:10:25+08:00	info	ingress/apisix_upstream.go:58	ApisixUpstream controller started
   2021-11-25T18:10:25+08:00	info	ingress/ingress.go:58	ingress controller started
   2021-11-25T18:10:25+08:00	info	ingress/apisix_route.go:57	ApisixRoute controller started
   2021-11-25T18:10:25+08:00	info	ingress/apisix_cluster_config.go:57	ApisixClusterConfig controller started
   2021-11-25T18:10:25+08:00	info	ingress/apisix_tls.go:58	ApisixTls controller started
   2021-11-25T18:10:25+08:00	info	ingress/secret.go:62	secret controller started
   2021-11-25T18:10:25+08:00	info	ingress/pod.go:47	pod controller started
   2021-11-25T18:10:26+08:00	warn	apisix/upstream.go:70	upstream not found	{"name": "etcd-apisix_etcd-apisix_2379", "url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/6b8de6f5", "cluster": "default"}
   2021-11-25T18:10:26+08:00	warn	ingress/controller.go:586	upstream is not referenced	{"cluster": "name=default; base_url=http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin", "upstream": "etcd-apisix_etcd-apisix_2379"}
   2021-11-25T18:10:26+08:00	warn	apisix/upstream.go:70	upstream not found	{"name": "etcd-apisix_etcd-apisix_2380", "url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/95c9429e", "cluster": "default"}
   2021-11-25T18:10:26+08:00	warn	ingress/controller.go:586	upstream is not referenced	{"cluster": "name=default; base_url=http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin", "upstream": "etcd-apisix_etcd-apisix_2380"}
   2021-11-25T18:10:26+08:00	warn	apisix/upstream.go:70	upstream not found	{"name": "ingress-apisix_apisix-admin_9180", "url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/ff9d87bf", "cluster": "default"}
   2021-11-25T18:10:26+08:00	warn	ingress/controller.go:586	upstream is not referenced	{"cluster": "name=default; base_url=http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin", "upstream": "ingress-apisix_apisix-admin_9180"}
   2021-11-25T18:10:26+08:00	warn	apisix/upstream.go:70	upstream not found	{"name": "ingress-apisix_apisix-ingress-controller_80", "url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/f94e10e6", "cluster": "default"}
   2021-11-25T18:10:26+08:00	warn	ingress/controller.go:586	upstream is not referenced	{"cluster": "name=default; base_url=http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin", "upstream": "ingress-apisix_apisix-ingress-controller_80"}
   2021-11-25T18:10:26+08:00	warn	apisix/upstream.go:70	upstream not found	{"name": "ingress-apisix_apisix-gateway_80", "url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/ffd51960", "cluster": "default"}
   2021-11-25T18:10:26+08:00	warn	ingress/controller.go:586	upstream is not referenced	{"cluster": "name=default; base_url=http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin", "upstream": "ingress-apisix_apisix-gateway_80"}
   2021-11-25T18:10:26+08:00	warn	apisix/upstream.go:70	upstream not found	{"name": "ingress-apisix_apisix-dashboard_80", "url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/65e49f92", "cluster": "default"}
   2021-11-25T18:10:26+08:00	warn	ingress/controller.go:586	upstream is not referenced	{"cluster": "name=default; base_url=http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin", "upstream": "ingress-apisix_apisix-dashboard_80"}
   2021-11-25T18:10:26+08:00	warn	apisix/upstream.go:70	upstream not found	{"name": "etcd-apisix_etcd-apisix-client_2379", "url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/d55e933", "cluster": "default"}
   2021-11-25T18:10:26+08:00	warn	ingress/controller.go:586	upstream is not referenced	{"cluster": "name=default; base_url=http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin", "upstream": "etcd-apisix_etcd-apisix-client_2379"}
   [GIN] 2021/11/25 - 18:10:28 | 200 |      55.981µs |   172.31.19.242 | GET      "/healthz"
   [GIN] 2021/11/25 - 18:10:33 | 200 |      37.991µs |   172.31.19.242 | GET      "/healthz"
   [GIN] 2021/11/25 - 18:10:38 | 200 |      62.731µs |   172.31.19.242 | GET      "/healthz"
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #773: apisix ingress deployed on eks happend upstream not found error

Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #773:
URL: https://github.com/apache/apisix-ingress-controller/issues/773#issuecomment-1059308488


   #893 will fix it.
   And #899 will unify the logs.
   I will close this one. Using #893 and #899 for track.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix-ingress-controller] tao12345666333 closed issue #773: apisix ingress deployed on eks happend upstream not found error

Posted by GitBox <gi...@apache.org>.
tao12345666333 closed issue #773:
URL: https://github.com/apache/apisix-ingress-controller/issues/773


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix-ingress-controller] tokers commented on issue #773: apisix ingress deployed on eks happend upstream not found error

Posted by GitBox <gi...@apache.org>.
tokers commented on issue #773:
URL: https://github.com/apache/apisix-ingress-controller/issues/773#issuecomment-979595212


   The errors on warning level are not harmful basically. Maybe these log messages are confusing.
   
   @tao12345666333 Did you talk about it on the last community meeting? If not, I think we should add a plan to optimize it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org