[GitHub] [apisix-ingress-controller] lingsamuel commented on issue #1226: e2e: introduce phase function to split cases

[GitBox <gi...@apache.org>]

lingsamuel commented on issue #1226:
URL: https://github.com/apache/apisix-ingress-controller/issues/1226#issuecomment-1235310362

   The general idea is that our e2e cases are a combination of many small cases. We can achieve complex cases by combining small cases.
   Here are some possible code snippets that describe how I think this proposal would work
   
   Let's assume that we have a type called Tester that will contain the automation features we need.
   
   1. Code snippet of basic functionality
   ```go
   // Assuming we are testing the ApisixRoute match rule
   tester := CreateRouteTester(&RouteTesterOption{header: "X-Foo", headerMatch: "bar", url: "/ip"})
   // data preparation
   tester.ApplyRoute() // ApisixRoute will be generated automatically base on the route rule and other options will remain default.
   // validation
   tester.ExpectAccess() // make a request to the route base on our route rule and expect it to succeed.
   tester.ExpectAccessFailedWithWrongRule() // make a request to the route but with a wrong options and expect it to failed
   ```
   Here we can notice that both `ExpectAccess` and `ExpectAccesFailedWithWrongRule` should always be called together in our validation.
   So we provide another function `ExecuteAllValidators()` to do this.
   
   2. Code snippet of update functionality
   ```go
   // data preparation
   tester.ApplyWrongRoute() // Create a wrong ApisixRoute that doesn't match our route rule
   // validation
   tester.ExpectAccessFailed() // make a request to the route with correct route rule, and expect it fail
   // data update
   tester.ApplyRoute() // Apply the correct ApisixRoute, it's a update behavior here
   // validation
   tester.ExecuteAllValidators() // It should be OK now
   ```
   
   Some other cases may also need to use ApisixRoute. They can reuse the tester now.
   For example, we need to verify that namespace selector is working correctly.
   ```go
   ns := "test-namespace"
   namespaceTester := CreateNamespaceTester(&NamespaceTesterOption{name: ns})
   routeTester := CreateRouteTester(&RouteTesterOption{namespace: ns})
   // data preparation
   namespaceTester.CreateWithWatchLabel()
   routeTester.ApplyRoute()
   // validation
   routeTester.ExecuteAllValidators()
   ```
   ```go
   // data preparation
   namespaceTester.CreateWithoutWatchLabel()
   routeTester.ApplyRoute()
   // validation
   routeTester.ExpectAccessFailed()
   // update to watch
   namespaceTester.AddWatchLabel()
   // validate it works
   routeTester.ExecuteAllValidators()
   // update to unwatch
   namespaceTester.RemoveWatchLabel()
   // validate it doesn't work
   routeTester.ExpectAccessFailed()
   ```
   
   We have only tested that the request is working as expected so far. What if we want to make sure the route resource is actually removed in APISIX? Suppose we have a `ExpectNoRouteInAPISIX` function to do this.
   
   In this case, we notice that the `ExpectedAccessFailed` function is always called together with `ExpectNoRouteInAPISIX`.
   Here is my solution.
   ```go
   tester := CreateTester() // a generic test runner
   tester.RegisterPhase("data-preparation", function() {
     namespaceTester.CreateWithoutWatchLabel()
     routeTester.ApplyRoute()
   })
   tester.RegisterPhase("watch", namespaceTester.AddWatchLabel)
   tester.RegisterPhase("unwatch", namespaceTester.RemoveWatchLabel)
   
   tester.RegisterValidatorForPhases([]string{"data-preparation", "unwatch"}, function() {
     routeTester.ExpectAccessFailed()
     routeTester.ExpectNoRouteInAPISIX()
   })
   tester.RegisterValidatorForPhases([]string{"watch"}, function() {
     routeTester.ExecuteAllValidators()
   })
   
   tester.RunPhases("data-preparation", "watch", "unwatch") //  all done!
   ```
   
   The API design is very early and unproven, any suggestions are welcome!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org