[DISCUSS] Proposal: support external service for Apache Ingress Controller

[Ling Samuel <li...@gmail.com>]

 Background

The users' environment may be not a simple single K8s cluster, but may be
multiple K8s clusters, a mix of K8s clusters and bare metal machines, or
rely on third-party services. The current ApisixRoute does not support such
complex environments.

It would be a useful feature if ApisixRoute supports using ExternalName
Service or domain name as a backend directly.
Solution

APISIX does support setting domain names or IP addresses as upstream nodes.
We can extend our CRD to support this scenario.

For external service, both the ExternalName service inside the K8s cluster
and the third-party service outside the cluster will only have one node
configured in the translated upstream nodes field, unlike the K8s service
which may have several nodes.
This allows multiple external services to be merged into a single upstream
and gain healthCheck capabilities.
Design

We add a field named upstream to ApisixRoute, which has the same level as
"backends". The user must fill in at least one of upstream or backends.
The field upstream will refer to an ApisixUpstream CRD, where the external
services are defined in the newly added field externalNode, indicating that
the upstream node is pre-defined and does not need to resolve.
An externalNode must have a name field pointing to the final node location,
which can be a service name pointing to a k8s external name service, an IP
address or a domain name. If it points to a k8s external name service, the
type field should be service, or it should be the default value of direct.
The node can also be configured with an optional load balancing weight,
which defaults to 1.

apiVersion: apisix.apache.org/v2kind: ApisixRoutemetadata:
  name: external-examplespec:
  http:
  - name: example
    match:
      hosts:
      - external.example.com
      paths:
      - "/*"
    upstreams:
    - name: external-example # Refers to ApisixUpstream
      weight: 50
---apiVersion: apisix.apache.org/v2kind: ApisixUpstreammetadata:
  name: external-examplespec:
  externalNodes: # Indicates that this upstream doesn't need to resolve nodes
  - name: "httpbin.org"
    # could be "domain", "serivce"
    # if the value is `service`, the name must refers to a K8s
ExternalName service
    type: domain
    weight: 1 # default value is 1
  healthCheck: # same as normal ApisixUpstream
    # ....

Using upstream and backends together

Users can use both upstream and backend. A route-level traffic-split plugin
will be generated normally.
Please note that since APISIX does not support health checks between
multiple upstreams, we will only support health checks within the
upstream.nodes.
This means that the healthCheck section defined in ApisixUpstream is at the
upstream level and its effect is limited to that upstream only, not at the
Route level.

Re: [DISCUSS] Proposal: support external service for Apache Ingress Controller

[Jintao Zhang <zh...@apache.org>]

SGTM

This method can ensure good compatibility and rich functions

Ling Samuel <li...@gmail.com> 于2022年8月5日周五 15:07写道：

>  Background
>
> The users' environment may be not a simple single K8s cluster, but may be
> multiple K8s clusters, a mix of K8s clusters and bare metal machines, or
> rely on third-party services. The current ApisixRoute does not support such
> complex environments.
>
> It would be a useful feature if ApisixRoute supports using ExternalName
> Service or domain name as a backend directly.
> Solution
>
> APISIX does support setting domain names or IP addresses as upstream nodes.
> We can extend our CRD to support this scenario.
>
> For external service, both the ExternalName service inside the K8s cluster
> and the third-party service outside the cluster will only have one node
> configured in the translated upstream nodes field, unlike the K8s service
> which may have several nodes.
> This allows multiple external services to be merged into a single upstream
> and gain healthCheck capabilities.
> Design
>
> We add a field named upstream to ApisixRoute, which has the same level as
> "backends". The user must fill in at least one of upstream or backends.
> The field upstream will refer to an ApisixUpstream CRD, where the external
> services are defined in the newly added field externalNode, indicating that
> the upstream node is pre-defined and does not need to resolve.
> An externalNode must have a name field pointing to the final node location,
> which can be a service name pointing to a k8s external name service, an IP
> address or a domain name. If it points to a k8s external name service, the
> type field should be service, or it should be the default value of direct.
> The node can also be configured with an optional load balancing weight,
> which defaults to 1.
>
> apiVersion: apisix.apache.org/v2kind: ApisixRoutemetadata:
>   name: external-examplespec:
>   http:
>   - name: example
>     match:
>       hosts:
>       - external.example.com
>       paths:
>       - "/*"
>     upstreams:
>     - name: external-example # Refers to ApisixUpstream
>       weight: 50
> ---apiVersion: apisix.apache.org/v2kind: ApisixUpstreammetadata:
>   name: external-examplespec:
>   externalNodes: # Indicates that this upstream doesn't need to resolve
> nodes
>   - name: "httpbin.org"
>     # could be "domain", "serivce"
>     # if the value is `service`, the name must refers to a K8s
> ExternalName service
>     type: domain
>     weight: 1 # default value is 1
>   healthCheck: # same as normal ApisixUpstream
>     # ....
>
> Using upstream and backends together
>
> Users can use both upstream and backend. A route-level traffic-split plugin
> will be generated normally.
> Please note that since APISIX does not support health checks between
> multiple upstreams, we will only support health checks within the
> upstream.nodes.
> This means that the healthCheck section defined in ApisixUpstream is at the
> upstream level and its effect is limited to that upstream only, not at the
> Route level.
>