You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "sum-elier (via GitHub)" <gi...@apache.org> on 2023/05/02 22:24:32 UTC
[GitHub] [apisix-ingress-controller] sum-elier opened a new issue, #1823: request help: Guidance on using gRPC + Kind
sum-elier opened a new issue, #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823
### Issue description
Hello. I am trying to use K8s, gRPC, and APISIX ingress controller locally through Kind. I have a small toy project to play around with these and it currently works with Contour without issues. I am trying to migrate it to APISIX but I am facing issues since what used to work with Contour as ingress no longer does with APISIX. The error I face is related to connectivity:
```
"error":"rpc error: code = Unavailable desc = connection closed before server preface received"
```
I am starting out with K8s and so I am not sure where to begin to look for issues.
I have tried to use APISIX as an Ingress or as an Gateway, and both give the same error. Following are the two setups for both approaches.
## Kind cluster configuration
```yaml
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
featureGates:
"GRPCContainerProbe": true
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
extraPortMappings:
- containerPort: 30980
hostPort: 80
- containerPort: 30180
hostPort: 9180
- role: worker
- role: worker
```
## As Ingress
### Command to install ingress controller:
```bash
helm upgrade apisix apisix/apisix \
--create-namespace \
--namespace default \
--set admin.nodePort=30180 \
--set admin.type=NodePort \
--set apisix.timezone="UTC" \
--set dashboard.enabled=true \
--set gateway.http.nodePort=30980 \
--set gateway.type=NodePort \
--set ingress-controller.config.apisix.adminAPIVersion=v3 \
--set ingress-controller.config.apisix.serviceNamespace=default \
--set ingress-controller.enabled=true \
--atomic --install --debug
```
### Service
```yaml
apiVersion: v1
kind: Service
metadata:
name: todos-service
labels:
app.kubernetes.io/name: todos
app.kubernetes.io/component: server
app.kubernetes.io/instance: todos-service-default
app.kubernetes.io/version: "0.1.0"
spec:
selector:
app.kubernetes.io/name: todos
app.kubernetes.io/component: server
ports:
- name: todos-app
port: 6666
targetPort: grpc
```
### ApisixUpstream
```yaml
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
name: todos-service
labels:
app.kubernetes.io/name: todos
app.kubernetes.io/component: server
app.kubernetes.io/instance: todos-service-apisix-upstream-default
app.kubernetes.io/version: "0.1.0"
spec:
scheme: grpc
```
## As Gateway API
### Command to install Gateway:
```bash
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v0.7.0-rc1/experimental-install.yaml
```
### Command to install ingress controller:
```bash
helm upgrade apisix apisix/apisix \
--create-namespace \
--namespace default \
--set admin.nodePort=30180 \
--set admin.type=NodePort \
--set apisix.timezone="UTC" \
--set dashboard.enabled=true \
--set gateway.http.nodePort=30980 \
--set gateway.type=NodePort \
--set ingress-controller.config.apisix.adminAPIVersion=v3 \
--set ingress-controller.config.apisix.serviceNamespace=default \
--set ingress-controller.config.kubernetes.enableGatewayAPI=true \
--set ingress-controller.enabled=true \
--atomic --install --debug
```
### Service (same as ingress)
```yaml
apiVersion: v1
kind: Service
metadata:
name: todos-service
labels:
app.kubernetes.io/name: todos
app.kubernetes.io/component: server
app.kubernetes.io/instance: todos-service-default
app.kubernetes.io/version: "0.1.0"
spec:
selector:
app.kubernetes.io/name: todos
app.kubernetes.io/component: server
ports:
- name: todos-app
port: 6666
targetPort: grpc
```
### Gateway
```yaml
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: GatewayClass
metadata:
name: todos-apisix-gatewayclass
labels:
app.kubernetes.io/name: todos
app.kubernetes.io/component: gateway
app.kubernetes.io/instance: todos-gateway-class-default
app.kubernetes.io/version: "0.1.0"
spec:
controllerName: apisix.apache.org/gateway-controller
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: Gateway
metadata:
name: todos-gateway
labels:
app.kubernetes.io/name: todos
app.kubernetes.io/component: gateway
app.kubernetes.io/instance: todos-gateway-default
app.kubernetes.io/version: "0.1.0"
spec:
gatewayClassName: todos-apisix-gatewayclass
listeners:
- name: grpc
protocol: HTTP
port: 30980
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: GRPCRoute
metadata:
name: todos-grpc-route
labels:
app.kubernetes.io/name: todos
app.kubernetes.io/component: gateway
app.kubernetes.io/instance: todos-grpc-route-default
app.kubernetes.io/version: "0.1.0"
spec:
parentRefs:
- name: todos-gateway
rules:
- backendRefs:
- name: todos-service
port: 6666
```
What is wrong with my configuration or where should I start looking for the problem(s)?
### Environment
- your apisix-ingress-controller version (output of apisix-ingress-controller version --long): **1.6.1**.
This was taken from `helm search repo apisix`. I am using Kind and I don't know much about K8s, so I don't know where to run `apisix-ingress-controller version --long`.
- your Kubernetes cluster version (output of kubectl version):
```
Client Version: v1.25.4
Kustomize Version: v4.5.7
Server Version: v1.26.3
```
- if you run apisix-ingress-controller in Bare-metal environment, also show your OS version (uname -a): N/A
- Helm output for `helm search repo apisix`:
```
NAME CHART VERSION APP VERSION
apisix/apisix 1.3.1 3.2.0
apisix/apisix-dashboard 0.8.0 3.0.0
apisix/apisix-ingress-controller 0.11.6 1.6.1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] sum-elier commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "sum-elier (via GitHub)" <gi...@apache.org>.
sum-elier commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1548608548
@tao12345666333 That would be awesome if possible. Thank you for your responses. I've been struggling with this for various weeks but now I can rest easy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] github-actions[bot] commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1709327936
This issue has been marked as stale due to 90 days of inactivity. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] request help: Guidance on using gRPC + Kind [apisix-ingress-controller]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #1823: request help: Guidance on using gRPC + Kind
URL: https://github.com/apache/apisix-ingress-controller/issues/1823
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] sum-elier commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "sum-elier (via GitHub)" <gi...@apache.org>.
sum-elier commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1572447643
@tao12345666333 Great, let me try that. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "tao12345666333 (via GitHub)" <gi...@apache.org>.
tao12345666333 commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1552758219
@sum-elier We have already implemented an annotation that can accomplish this function.
```
k8s.apisix.apache.org/upstream-scheme: grpc
```
https://github.com/apache/apisix-ingress-controller/blob/e91dbf5303c2ec8a775eabc8a7d9955e053b20f1/test/e2e/suite-annotations/upstreamprotocol.go#L31-L49
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] sum-elier commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "sum-elier (via GitHub)" <gi...@apache.org>.
sum-elier commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1572625112
After applying your suggestion as follows (notice `ingressClassName` instead of the annotation)
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
k8s.apisix.apache.org/upstream-scheme: grpcs
name: todos-ingress
labels:
app.kubernetes.io/name: todos
app.kubernetes.io/component: server
app.kubernetes.io/instance: todos-ingress-default
app.kubernetes.io/version: "0.1.0"
spec:
ingressClassName: apisix
rules:
- http:
paths:
- pathType: ImplementationSpecific
path: /
backend:
service:
name: todos-service
port:
name: todos-app
```
when I check the configuration of the ingress I see there's no address and no events, is this normal?
```shell
> kubectl describe ingress todos-ingress
Name: todos-ingress
Labels: app.kubernetes.io/component=server
app.kubernetes.io/instance=todos-ingress-default
app.kubernetes.io/name=todos
app.kubernetes.io/version=0.1.0
Namespace: default
Address:
Ingress Class: apisix
Default backend: <default>
Rules:
Host Path Backends
---- ---- --------
*
/ todos-service:todos-app (10.244.2.7:47586)
Annotations: k8s.apisix.apache.org/upstream-scheme: grpcs
Events: <none>
```
I suppose no because now the grpc error reads "error reading server preface: EOF"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] sum-elier commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "sum-elier (via GitHub)" <gi...@apache.org>.
sum-elier commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1548599801
> The `GRPCRoute` has not been implemented in APISIX Ingress currently.
>
> You can refer to this file
>
> https://github.com/apache/apisix-ingress-controller/blob/c6a13b3b8619b4ac6ba0cd9c921da4e13550a945/test/e2e/suite-features/scheme.go#L61-L85
@tao12345666333 That makes sense. But what about then when I try to use it as Ingress API, shouldn't it work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "tao12345666333 (via GitHub)" <gi...@apache.org>.
tao12345666333 commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1582172695
You can refer to the test case I provided above and use the same backend for verification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] sum-elier commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "sum-elier (via GitHub)" <gi...@apache.org>.
sum-elier commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1546999871
When I check the Admin API there are no registered routes even though I have defined `GRPCRoute`:
```shell
> curl http://127.0.0.1:9180/apisix/admin/routes -H 'X-API-Key: edd1c9f034335f136f87ad84b625c8f1'
{"total":0,"list":[]}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] request help: Guidance on using gRPC + Kind [apisix-ingress-controller]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1751544647
This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "tao12345666333 (via GitHub)" <gi...@apache.org>.
tao12345666333 commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1547342657
The `GRPCRoute` has not been implemented in APISIX Ingress currently.
You can refer to this file https://github.com/apache/apisix-ingress-controller/blob/c6a13b3b8619b4ac6ba0cd9c921da4e13550a945/test/e2e/suite-features/scheme.go#L61-L85
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-ingress-controller] tao12345666333 commented on issue #1823: request help: Guidance on using gRPC + Kind
Posted by "tao12345666333 (via GitHub)" <gi...@apache.org>.
tao12345666333 commented on issue #1823:
URL: https://github.com/apache/apisix-ingress-controller/issues/1823#issuecomment-1548605213
grpc is not yet supported in Ingress, we can add this feature
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org