You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2020/10/27 14:10:04 UTC
[GitHub] [apisix-dashboard] nic-chen opened a new issue #607: proposal: e2e test
nic-chen opened a new issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607
Hi, Community,
here is a proposal about e2e test :
# Background
Manager api no longer interacts with admin api, but writes data directly to etcd.
At present, we do not have the e2e test cases that cover from dashboard, manager api to Apache APISIX.
# Plan
1. e2e test is divided into two parts, front-end e2e and back-end e2e
2. Because the front-end only uses part of the interface of `manager api`, the front-end e2e only needs to ensure normal interaction with `manager api`
3. Back-end e2e, APISIX is started through docker, Go calls manager api to configure routing and other resources, then access APISIX for verification.
# Details
Development language: Go
Test library: https://github.com/h2non/gock
Test environment: Start with docker-compose, a 3-node ETCD cluster, and 2-node APISIX cluster (2 nodes need to be verified at the same time).
Since APISIX already has complete test cases, we plan to migrate the test cases under the APISIX t directory.
Because of the huge workload, I will first migrate the test cases under t/node/ and t/router/, and leave other use cases to others who have time, or migrate later when I have time to complete the coverage.
# Test cases
1. route
(1) host:
Configure the correct host and hosts through the `manager api`, including wildcard domain name, and verify the correctness of the route by accessing APISIX.
Confirm that the `manager api` cannot configure host and hosts at the same time
Make sure that the `manager api` cannot be configured with wrong hosts and hosts, including illegal strings, too high length, wrong data types, etc.
(2) invalid conf
Confirm that the `manager api` verifies the correctness of each field. include:
String: length check, regular expression check
IP: v4 v6 format verification
int, float, bool and other types check
(3) config route with not exist service_id or upstream_id
(4) remote_addr, remote_addrs: v4 v6 format verification
(5) uri, uris
invalid value check
empty value check
(6) vars: args, cookie, header, different operators
check combinations of different types of vars and operators
(7) methods
Wrong data type (not string array)
Not in enumerated values
(8) priority
Customize different priority checks
Custom and default priority verification
(9) plugins
Extract at least one plug-in from auth, limit, log, trace, metrics and other types respectively for verification
(10) plugin orchestration check (normal and invalid check)
(11) other fields: name, desc, create_time, update_time, labels, filter_func(can't verify in Go, TODO)
(12) not exist field
Confirm that the `manager api` can refuse to configure undefined fields
(13) Verify routing correctness under multi routes
(14) router engine: check radixtree_uri first, radixtree_host_uri TODO
2. upstream
(1) key: remote_addr, query_string, arg_xxx
(2) hash_on: consumer, header, cookie, vars
(3) node: ipv6, domain, array nodes (struct)
(4) ewma: latency, frequency
(5) roundrobin:, weight
(6) timeout
(7) retries
(8) websocket
(9) health check: active, passive, healthy, unhealthy
(10) invalid conf
3. ssl
(1) cert + key + sni
(2) cert + key + snis
(3) certs + keys + snis
(4) wildcard domain name
4.consumer
(1) missing auth plugin
(2) two or more auth plugins
(3) with plugins
5. service
(1) invalid route
(2) route with empty service
6. global rules
(1) add
(2) update
(3) delete
(4) cover by route plugins
7. plugins merge
(1) route + service
(2) route + consumer
(3) route + service + consumer
___
The test cases only describe the route part in detail, and the other parts will be detailed in the issue later.
What do you think? Any suggestion is welcome.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] nic-chen commented on issue #607: proposal: e2e test
Posted by GitBox <gi...@apache.org>.
nic-chen commented on issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607#issuecomment-718779733
- Here is an example PR:
https://github.com/apache/apisix-dashboard/pull/612
- For the overall process of backend e2e testing, please refer to:
https://github.com/apache/apisix-dashboard/pull/612/files#diff-22340e8e8ab51438af10cdd230be186d1ef57c49ca1a89ac51fe4f6034bb870f
- Initialization and login of manager api have been implemented in base.go:
https://github.com/apache/apisix-dashboard/pull/612/files#diff-7fc06fc4ee2fd9aa89677944844b938da1f18530ebbd9a86b0869d93f46b3c29
- For specific test cases, please refer to:
https://github.com/apache/apisix-dashboard/pull/612/files#diff-a51586806ed3d846236a041f0b2fadb57413d13b73b2d5199526d88c00f796a6
- Local debugging
```sh
# run APISIX and ETCD
$ cd api/test/docker && docker-compose up -d
# run test
$ export APIX_ETCD_ENDPOINTS=127.0.0.1:2379,127.0.0.1:3379,127.0.0.1:4379
$ cd ../e2e && go test
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] moonming commented on issue #607: proposal: e2e test
Posted by GitBox <gi...@apache.org>.
moonming commented on issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607#issuecomment-719116525
please add tag for those small issues.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] membphis commented on issue #607: proposal: e2e test
Posted by GitBox <gi...@apache.org>.
membphis commented on issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607#issuecomment-717662544
Is there a small PR as a sample? For example, test the `host` part of `route`.
And we can run it in CI, confirm the manager works fine.
BTW, running e2e use cases, can we get the latest code coverage results?
Code coverage is a very important indicator for testing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] nic-chen commented on issue #607: proposal: e2e test
Posted by GitBox <gi...@apache.org>.
nic-chen commented on issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607#issuecomment-737140355
put it to next milestone to continue this job.
will create small issues to add test cases step by step.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] nic-chen commented on issue #607: proposal: e2e test
Posted by GitBox <gi...@apache.org>.
nic-chen commented on issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607#issuecomment-717659297
> it will be best if add checkbox for each test case
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] moonming edited a comment on issue #607: proposal: e2e test
Posted by GitBox <gi...@apache.org>.
moonming edited a comment on issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607#issuecomment-717289451
it will be best if add checkbox for each test case
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] moonming commented on issue #607: proposal: e2e test
Posted by GitBox <gi...@apache.org>.
moonming commented on issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607#issuecomment-717289451
it will。be best if add checkbox for each test case
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] juzhiyuan closed issue #607: proposal: e2e test
Posted by GitBox <gi...@apache.org>.
juzhiyuan closed issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] juzhiyuan commented on issue #607: proposal: e2e test
Posted by GitBox <gi...@apache.org>.
juzhiyuan commented on issue #607:
URL: https://github.com/apache/apisix-dashboard/issues/607#issuecomment-717662243
Just added this issue to 2.1, feel free to adjust it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org