You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/02/24 08:24:14 UTC

[GitHub] [apisix-ingress-controller] GhangZh opened a new issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

GhangZh opened a new issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886


   ### Issue description
   
   ingress-controller always shows this log, but the service is actually there
   ![image](https://user-images.githubusercontent.com/92301646/155486114-a44a6117-8610-4c8a-942d-c855788c1b52.png)
   ![image](https://user-images.githubusercontent.com/92301646/155486141-bd817922-c7b3-41a1-a4ff-fd22e2a1efb9.png)
   ![image](https://user-images.githubusercontent.com/92301646/155486153-ffb17a45-89e2-42aa-b62f-561b9093499c.png)
   
   
   ### Environment
   
   - your apisix-ingress-controller version (output of apisix-ingress-controller version --long):1.4.0
   - your Kubernetes cluster version (output of kubectl version):1.18
   - if you run apisix-ingress-controller in Bare-metal environment, also show your OS version (uname -a):
   
   
   ### Minimal test code / Steps to reproduce
   
   kubectl  -n ingress-apisix logs -f  apisix-ingress-controller-xxxxxx |grep retry
   kubectl  -n ingress-apisix describe ar apisixroute-dashboard
   
   ### Actual result
   
   no error log 
   
   ### Error log
   
   2022-02-24T15:39:39+08:00	warn	ingress/apisix_route.go:307	sync ApisixRoute failed, will retry	{"object": {"Type":1,"Object":{"Key":"ingress-apisix/apisixroute-dashboard","OldObject":null,"GroupVersion":"apisix.apache.org/v2beta1"},"Tombstone":null}, "error": "service \"apisix-dashboard\" not found"}
   
   ### Expected result
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1054977754


   hi @drrrMikado  I see that your ApisixRoute resource is in `flow-control-server-go` namespace, but Service `flow-control-server-go-dev` is in `flow-control-server-go-dev` namespace, so it will not be found. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] drrrMikado commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
drrrMikado commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1053858479


   > Are you using the latest CRD? https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix-ingress-controller/crds/customresourcedefinitions.yaml
   
   Same. It's using ltest CRD.
   ![image](https://user-images.githubusercontent.com/11224565/155919483-6b69a660-fdea-4c35-8b21-6c2967d98329.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1055482569


   Could you please post the complete installation and reproduction steps? 
   
   Your newly posted log indicates that you have enabled the Ingress LB status function, which requires you to check your Kubernetes environment.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1053935349


   @drrrMikado can you show me your full logs?
   And definitions for resources such as ApisixRoute and Service 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1049644813


   can you run  `kubectl api-resources --api-group=apisix.apache.org` ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] drrrMikado commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
drrrMikado commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1057590137


   > Could you please post the complete installation and reproduction steps?
   
   It took a while for the problem to occur, so I can't give you the reproduction steps.
   
   > Your newly posted log indicates that you have enabled the Ingress LB status function, which requires you to check your Kubernetes environment.
   
   Thanks, I will check the Kubernetes environment.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] drrrMikado commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
drrrMikado commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1054899795


   log:
   ```
    2022-03-01T09:49:03+08:00	error	translation/apisix_route.go:338	failed to get service port in backend	{"backend": {"serviceName":"flow-control-server-go-dev","servicePort":80,"weight":null}, "apisix_route": {"kind":"ApisixRoute","apiVersion":"apisix.apache.org/v2beta2","metadata":{"name":"flow-control-server-go-dev","namespace":"flow-control-server-go","selfLink":"/apis/apisix.apache.org/v2beta2/namespaces/flow-control-server-go/apisixroutes/flow-control-server-go-dev","uid":"178a1217-3674-465e-b07b-be7d60acc57c","resourceVersion":"9564966417","generation":1,"creationTimestamp":"2022-02-28T03:51:33Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apisix.apache.org/v2beta3\",\"kind\":\"ApisixRoute\",\"metadata\":{\"annotations\":{},\"name\":\"flow-control-server-go-dev\",\"namespace\":\"flow-control-server-go\"},\"spec\":{\"http\":[{\"backends\":[{\"serviceName\":\"flow-control-server-go-dev\",\"servicePort\":80}],\"match\":{\"hosts\":[\"flow-
 control-server-go-dev.xxx.com\"],\"paths\":[\"/ws\"]},\"name\":\"websocket\",\"websocket\":true},{\"backends\":[{\"serviceName\":\"flow-control-server-go-dev\",\"servicePort\":80}],\"match\":{\"hosts\":[\"flow-control-server-go-dev.xxx.com\"],\"paths\":[\"/*\"]},\"name\":\"default\"}]}}\n"},"managedFields":[{"manager":"apisix-ingress-controller","operation":"Update","apiVersion":"apisix.apache.org/v2beta2","time":"2022-02-28T03:51:33Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:http":{}},"f:status":{".":{},"f:conditions":{}}}},{"manager":"kubectl-client-side-apply","operation":"Update","apiVersion":"apisix.apache.org/v2beta3","time":"2022-02-28T03:51:33Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{}}}]},"spec":{"http":[{"name":"websocket","match":{"paths":["/ws"],"hosts":["flow-control-server-go-dev.xxx.com"]},"backends":[{"serviceName":"flow-control-server-go-dev","servicePort":
 80,"weight":null}],"websocket":true,"authentication":{"enable":false,"type":"","keyauth":{}}},{"name":"default","match":{"paths":["/*"],"hosts":["flow-control-server-go-dev.xxx.com"]},"backends":[{"serviceName":"flow-control-server-go-dev","servicePort":80,"weight":null}],"websocket":false,"authentication":{"enable":false,"type":"","keyauth":{}}}]},"status":{"conditions":[{"type":"ResourcesAvailable","status":"True","observedGeneration":1,"lastTransitionTime":null,"reason":"ResourcesSynced","message":"Sync Successfully"}]}}, "error": "service \"flow-control-server-go-dev\" not found"}
   
    2022-03-01T09:49:03+08:00	error	ingress/apisix_route.go:159	failed to translate ApisixRoute v2beta2	{"error": "service \"flow-control-server-go-dev\" not found", "object": {}}
   
    2022-03-01T09:49:03+08:00	warn	ingress/apisix_route.go:307	sync ApisixRoute failed, will retry	{"object": {"Type":1,"Object":{"Key":"flow-control-server-go/flow-control-server-go-dev","OldObject":null,"GroupVersion":"apisix.apache.org/v2beta2"},"Tombstone":null}, "error": "service \"flow-control-server-go-dev\" not found"}
   
   // 'platform/account-system' also not found, but log level is info, not error.
    2022-03-01T09:49:03+08:00	info	ingress/controller.go:609	service platform/account-system not found
   ```
   
   ApisixRoute:
   ```yaml
   apiVersion: apisix.apache.org/v2beta3
   kind: ApisixRoute
   metadata:
     name: flow-control-server-go-dev
     namespace: flow-control-server-go
   spec:
     http:
     - name: websocket
       match:
         hosts:
         - flow-control-server-go-dev.xxx.com
         paths:
         - /ws
       backends:
         - serviceName: flow-control-server-go-dev
           servicePort: 80
       websocket: true
     - name: default
       match:
         hosts:
         - flow-control-server-go-dev.xxx.com
         paths:
         - /*
       backends:
         - serviceName: flow-control-server-go-dev
           servicePort: 80
   ```
   
   Service:
   ```yaml
   kind: Service
   apiVersion: v1
   metadata:
     name: flow-control-server-go-dev
     namespace: flow-control-server-go-dev
     labels:
       app: flow-control-server-go-dev
   spec:
     ports:
       - name: web
         protocol: TCP
         port: 80
         targetPort: 8000
     selector:
       app: flow-control-server-go-dev
       version: v1
     type: NodePort
     sessionAffinity: None
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] GhangZh commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
GhangZh commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1049647043


   > kubectl api-resources --api-group=apisix.apache.org
   
   ![image](https://user-images.githubusercontent.com/92301646/155495078-1c1c7291-64ca-4c85-ab48-8c32df1aef64.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1049654267


   Are you using the latest CRD?
   https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix-ingress-controller/crds/customresourcedefinitions.yaml


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] drrrMikado edited a comment on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
drrrMikado edited a comment on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1054899795


   log:
   ```
    2022-03-01T09:49:03+08:00	error	translation/apisix_route.go:338	failed to get service port in backend	{"backend": {"serviceName":"flow-control-server-go-dev","servicePort":80,"weight":null}, "apisix_route": {"kind":"ApisixRoute","apiVersion":"apisix.apache.org/v2beta2","metadata":{"name":"flow-control-server-go-dev","namespace":"flow-control-server-go","selfLink":"/apis/apisix.apache.org/v2beta2/namespaces/flow-control-server-go/apisixroutes/flow-control-server-go-dev","uid":"178a1217-3674-465e-b07b-be7d60acc57c","resourceVersion":"9564966417","generation":1,"creationTimestamp":"2022-02-28T03:51:33Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apisix.apache.org/v2beta3\",\"kind\":\"ApisixRoute\",\"metadata\":{\"annotations\":{},\"name\":\"flow-control-server-go-dev\",\"namespace\":\"flow-control-server-go\"},\"spec\":{\"http\":[{\"backends\":[{\"serviceName\":\"flow-control-server-go-dev\",\"servicePort\":80}],\"match\":{\"hosts\":[\"flow-
 control-server-go-dev.xxx.com\"],\"paths\":[\"/ws\"]},\"name\":\"websocket\",\"websocket\":true},{\"backends\":[{\"serviceName\":\"flow-control-server-go-dev\",\"servicePort\":80}],\"match\":{\"hosts\":[\"flow-control-server-go-dev.xxx.com\"],\"paths\":[\"/*\"]},\"name\":\"default\"}]}}\n"},"managedFields":[{"manager":"apisix-ingress-controller","operation":"Update","apiVersion":"apisix.apache.org/v2beta2","time":"2022-02-28T03:51:33Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:http":{}},"f:status":{".":{},"f:conditions":{}}}},{"manager":"kubectl-client-side-apply","operation":"Update","apiVersion":"apisix.apache.org/v2beta3","time":"2022-02-28T03:51:33Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{}}}]},"spec":{"http":[{"name":"websocket","match":{"paths":["/ws"],"hosts":["flow-control-server-go-dev.xxx.com"]},"backends":[{"serviceName":"flow-control-server-go-dev","servicePort":
 80,"weight":null}],"websocket":true,"authentication":{"enable":false,"type":"","keyauth":{}}},{"name":"default","match":{"paths":["/*"],"hosts":["flow-control-server-go-dev.xxx.com"]},"backends":[{"serviceName":"flow-control-server-go-dev","servicePort":80,"weight":null}],"websocket":false,"authentication":{"enable":false,"type":"","keyauth":{}}}]},"status":{"conditions":[{"type":"ResourcesAvailable","status":"True","observedGeneration":1,"lastTransitionTime":null,"reason":"ResourcesSynced","message":"Sync Successfully"}]}}, "error": "service \"flow-control-server-go-dev\" not found"}
   
    2022-03-01T09:49:03+08:00	error	ingress/apisix_route.go:159	failed to translate ApisixRoute v2beta2	{"error": "service \"flow-control-server-go-dev\" not found", "object": {}}
   
    2022-03-01T09:49:03+08:00	warn	ingress/apisix_route.go:307	sync ApisixRoute failed, will retry	{"object": {"Type":1,"Object":{"Key":"flow-control-server-go/flow-control-server-go-dev","OldObject":null,"GroupVersion":"apisix.apache.org/v2beta2"},"Tombstone":null}, "error": "service \"flow-control-server-go-dev\" not found"}
   
   // 'platform/account-system' also not found, but log level is info, not error.
    2022-03-01T09:49:03+08:00	info	ingress/controller.go:609	service platform/account-system not found
   ```
   
   ApisixRoute:
   ```yaml
   apiVersion: apisix.apache.org/v2beta3
   kind: ApisixRoute
   metadata:
     name: flow-control-server-go-dev
     namespace: flow-control-server-go
   spec:
     http:
     - name: websocket
       match:
         hosts:
         - flow-control-server-go-dev.xxx.com
         paths:
         - /ws
       backends:
         - serviceName: flow-control-server-go-dev
           servicePort: 80
       websocket: true
     - name: default
       match:
         hosts:
         - flow-control-server-go-dev.xxx.com
         paths:
         - /*
       backends:
         - serviceName: flow-control-server-go-dev
           servicePort: 80
   ```
   
   Service:
   ```yaml
   kind: Service
   apiVersion: v1
   metadata:
     name: flow-control-server-go-dev
     namespace: flow-control-server-go
     labels:
       app: flow-control-server-go-dev
   spec:
     ports:
       - name: web
         protocol: TCP
         port: 80
         targetPort: 8000
     selector:
       app: flow-control-server-go-dev
       version: v1
     type: NodePort
     sessionAffinity: None
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1049770653


   We have made some adjustments to CRD in the new version


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] GhangZh commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
GhangZh commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1049751610


   > Are you using the latest CRD? https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix-ingress-controller/crds/customresourcedefinitions.yaml
   
   Okay, I'll try it out.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] drrrMikado commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
drrrMikado commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1055186323


   > hi @drrrMikado I see that your ApisixRoute resource is in `flow-control-server-go` namespace, but Service `flow-control-server-go-dev` is in `flow-control-server-go-dev` namespace, so it will not be found.
   
   I'm sorry. It's my fault. I copied it wrong. I updated the above configuration and added screenshots.
   ![image](https://user-images.githubusercontent.com/11224565/156138830-37c14610-33d0-438e-ad3d-6b6061d639e4.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix-ingress-controller] drrrMikado commented on issue #886: bug: ingress-controller always shows this log not found service, but the service is actually there

Posted by GitBox <gi...@apache.org>.
drrrMikado commented on issue #886:
URL: https://github.com/apache/apisix-ingress-controller/issues/886#issuecomment-1055245468


   Also, I get a different error.
   ```
    2022-03-01T17:55:49+08:00	error	ingress/status.go:197	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   
    2022-03-01T17:55:49+08:00	error	ingress/status.go:197	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org