[GitHub] [incubator-seatunnel] EricJoy2048 commented on issue #2691: [e2e] How about Generate a task like fake2Mysql to generate all type for source, and generate 100W row data to test performance

[GitBox <gi...@apache.org>]

EricJoy2048 commented on issue #2691:
URL: https://github.com/apache/incubator-seatunnel/issues/2691#issuecomment-1245385578

   With the addition of `SeaTunnel Engine` every connector need add e2e-test for three engine. This is a difficult and boring job for developers. So I think it's time to add an automated test framework for SeaTunnel. The test framework need support some key features:
   
   1. Automatically generate job configuration files based on the information provided by the connector. If the connector is a source connector, we need to determine whether a sink connector already exists. If already have a sink connector, we need generate two jobs. This first job is to generate test data, it looks like `FakeSource -> XxxSink` and another job is `XxxSource -> AssertSink`. On the other handle, If the connector is a sink connector, we need to determine whether a source connector already exists. If already have a source connector, we need generate two jobs. One of the job is `FakeSource->XxxSink` and another job is `XxxSource->AssertSink`.
   2. Test all engines automatically. The test framework need run the jobs in all of the engine SeaTunnel supported now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org