You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "mikkel1156 (via GitHub)" <gi...@apache.org> on 2026/06/05 18:54:05 UTC
[I] bug: getting "404 route not found" in version 2.1.0 [apisix-ingress-controller]
mikkel1156 opened a new issue, #2782:
URL: https://github.com/apache/apisix-ingress-controller/issues/2782
### Current Behavior
I am getting "404 Route Not Found" after updating the APISIX helm chart to latest version, and after testing it is only happening on the latest 2.1.0 release.
It is creating the route, but is unable to match the request to it. I did a output of both configurations between the versions, and the only difference is the server_port in vars as introduced in #2703
Working route:
```json
{
"uris": [
"/",
"/*"
],
"priority": 32338085888,
"service_id": "a2594164",
"labels": {
"k8s/controller-name": "apisix.apache.org/apisix-ingress-controller",
"k8s/kind": "HTTPRoute",
"k8s/namespace": "forgejo",
"manager-by": "apisix-ingress-controller",
"k8s/name": "forgejo"
},
"id": "47600540",
"modifiedIndex": 1780683114135,
"status": 1,
"name": "forgejo_forgejo_0-0"
}
```
Route in new version that is not being matched:
```json
{
"priority": 32338085888,
"vars": [
[
"server_port",
"in",
[
"80",
"443"
]
]
],
"modifiedIndex": 1780682959811,
"status": 1,
"name": "forgejo_forgejo_0-0",
"service_id": "a2594164",
"id": "47600540",
"labels": {
"k8s/namespace": "forgejo",
"k8s/kind": "HTTPRoute",
"k8s/name": "forgejo",
"manager-by": "apisix-ingress-controller",
"k8s/controller-name": "apisix.apache.org/apisix-ingress-controller"
},
"uris": [
"/",
"/*"
]
},
```
### Expected Behavior
Route would be matched and traffic would be proxied.
### Error Logs
None found.
### Steps to Reproduce
I'll put some details of the environment here (note: I am changing my actual domain to example.com below):
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
annotations:
cert-manager.io/cluster-issuer: letsencrypt
name: apisix
namespace: apisix
spec:
gatewayClassName: apisix
infrastructure:
parametersRef:
group: apisix.apache.org
kind: GatewayProxy
name: apisix-config
listeners:
- allowedRoutes:
namespaces:
from: All
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: All
hostname: '*.example.com'
name: https
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: gateway-example-com
mode: Terminate
```
HTTPRoute:
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: forgejo
namespace: forgejo
spec:
hostnames:
- git.example.com
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: apisix
namespace: apisix
rules:
- backendRefs:
- group: ""
kind: Service
name: forgejo
port: 80
weight: 1
filters: []
matches:
- path:
type: PathPrefix
value: /
```
Helm values:
```yaml
service:
type: "LoadBalancer"
externalTrafficPolicy: Local
etcd:
enabled: false
ingress-controller:
enabled: true
config:
enableHTTP2: true
provider:
type: apisix-standalone
apisix:
adminService:
namespace: apisix
gatewayProxy:
createDefault: false
apisix:
pluginAttrs:
redirect:
https_port: 443
admin:
allow:
ipList:
- 127.0.0.1/24
- 10.0.0.0/8
ssl:
enabled: true
deployment:
role: traditional
role_traditional:
config_provider: yaml
```
```
### Environment
- APISIX Ingress controller version: 2.1.0
- Kubernetes cluster version (run `kubectl version`): 1.34.1
- OS version if running APISIX Ingress controller in a bare-metal environment (run `uname -a`): NixOS 25.11
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: getting "404 route not found" in version 2.1.0 [apisix-ingress-controller]
Posted by "mikkel1156 (via GitHub)" <gi...@apache.org>.
mikkel1156 commented on issue #2782:
URL: https://github.com/apache/apisix-ingress-controller/issues/2782#issuecomment-4671393331
Setting the option to `off` works.
I set it in helm like so:
```yaml
ingress-controller:
enabled: true
config:
listenerPortMatchMode: "off"
```
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: getting "404 route not found" in version 2.1.0 [apisix-ingress-controller]
Posted by "AlinsRan (via GitHub)" <gi...@apache.org>.
AlinsRan commented on issue #2782:
URL: https://github.com/apache/apisix-ingress-controller/issues/2782#issuecomment-4665882153
During troubleshooting I found that the `auto` mode of `listener_port_match_mode` (#2703) is **not backward compatible** — it injects a `server_port` var into multi-listener routes by default, and the injected Gateway listener ports (`80`/`443`) never match `$server_port`, which is APISIX's actual listening port (`9080`/`9443` in the standard Helm deploy). This breaks existing deployments on upgrade with no config change.
Two workarounds:
**1. Disable the injection via config (recommended):**
```yaml
listener_port_match_mode: off
```
**2. Align the Gateway listener `port` with the data-plane port** (e.g. `9080`/`9443`):
This makes the injected value equal `$server_port`, so matching succeeds even when the LoadBalancer exposes 80/443 externally (the LB remaps `80 → 9080`, and `$server_port` reflects the internal port, not the external one). However, the Gateway listener port no longer represents the logical port clients connect to, which misrepresents the listener and can affect TLS / redirect logic that keys off 443. So this is a hack, not a clean fix — option 1 is preferred.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: getting "404 route not found" in version 2.1.0 [apisix-ingress-controller]
Posted by "AlinsRan (via GitHub)" <gi...@apache.org>.
AlinsRan commented on issue #2782:
URL: https://github.com/apache/apisix-ingress-controller/issues/2782#issuecomment-4665840885
### #2703 is not backward compatible
Before #2703 (i.e. 2.0.x), routes carried no `server_port` var, so requests on any listening port matched. #2703 first shipped in 2.1.0 and, with the default `listener_port_match_mode: auto`, injects a `server_port` var into every route that matches multiple listener ports — with **no config change** on the user's side. The "current behavior maintained" claim only refers to compatibility with the unreleased PR, not with the last release 2.0.x. For upgrading users this is a silent breaking change.
**Why it gets injected here (even without `sectionName`/`port`):** the default mode `auto` injects when a route explicitly targets a listener **or** when it matches multiple listener ports:
```go
case config.ListenerPortMatchModeAuto:
return explicit || len(ports) > 1
```
The HTTPRoute in this report has no `sectionName`/`port` and hostname `git.example.com`, which matches **both** listeners — `http` (80, no hostname → matches any) and `https` (443, `*.example.com` → matches `git.example.com`). So `listenerPorts = {80, 443}`, `len(ports) > 1` is true, and `server_port in ["80","443"]` is injected.
**Why it then 404s:** the injected values `80`/`443` are the Gateway's **logical** listener ports, but the nginx/APISIX `$server_port` variable reflects the port **APISIX actually listens on**. In the standard Helm deploy APISIX listens internally on `9080`/`9443` (the Service/LoadBalancer maps external 80/443 → 9080/9443), so `$server_port` is `9080`/`9443` at request time and never matches `in ["80","443"]` → 404.
(The #2703 e2e tests don't catch this because they set APISIX `node_listen` to the same ports as the Gateway listeners — `9080`/`9081` — so listener port == actual listening port there, a condition that does not hold in typical deployments.)
### Temporary fix for affected users
Set the controller config to disable the injection:
```yaml
listener_port_match_mode: off
```
This restores the 2.0.x matching behavior without any other change.
### How this should be handled going forward
1. **The default should be `off`.** `server_port` injection must be opt-in:
- **Semantics:** a route attaching without `sectionName`/`port` means "attach to all compatible listeners" and should match on all of them. The `auto` heuristic (`len(ports) > 1`) turns the *normal* multi-listener attachment into port filtering even though the user never expressed any port-routing intent. The case that legitimately wants listener isolation is explicit `sectionName`/`port` targeting, which is opt-in by nature.
- **Compatibility:** `auto` as default silently changes matching for every multi-listener deployment on upgrade.
- This should ship as a 2.1.x patch so existing users recover without touching their manifests.
2. **The mechanism itself needs a redesign before any inject-by-default mode is safe.** Matching `$server_port` against the Gateway's logical listener port is incorrect whenever port remapping is in play (Service/LoadBalancer, custom `node_listen`, etc.) — which is the common case. Toggling *when* to inject does not fix *what* is injected. A correct implementation has to map the Gateway listener port to APISIX's actual listening port (or drop `server_port` for listener disambiguation). This deserves a separate design discussion and should not block the default change in (1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: getting "404 route not found" in version 2.1.0 [apisix-ingress-controller]
Posted by "mikkel1156 (via GitHub)" <gi...@apache.org>.
mikkel1156 closed issue #2782: bug: getting "404 route not found" in version 2.1.0
URL: https://github.com/apache/apisix-ingress-controller/issues/2782
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org