You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@streampipes.apache.org by Philipp Zehnder <ze...@apache.org> on 2021/06/23 11:04:19 UTC

Automated E2E Testing

Hi together,

since we have many different components, it is always difficult to keep them up to date and to test when something changes.
Therefore I started to implement an E2E test framework with the help of Cypress to test a StreamPipes instance when it was started.

The code is currently in a separate branch [1]. I created a new folder in the ui, called cypress [2], containing all the test code.
I also added a short README.

The main idea is to test the workflows of the application as well as the adapters and processing elements.

I created some example adapter tests [3], both for specific and generic adapters.
For them, it must be ensured that we have access to the corresponding data source and that this contains some sample data.
This will not work for all the adapters, but should be feasible for most of them. (e.g. for Kafka we can create a pipeline containing a Kafka sink and then read the data with the adapter).
It would be great if we had some simulators for the PLCs, because then we can include tests for them as well. Does anyone have an idea how to do that?

Additionally, I added two examples to test processing elements [4].
For a test definition, two csv files must be added to [5]. One contains the input data set and one contains the expected result.
For the test a data set adapter is started, then a pipeline is build with the processing element and the data lake sink is used to store the data.
Once the data is processed the result is exported from the data lake and compared to the file ‘expected.csv’.
Currently the configuration for the processor test must be set in a typescript file, but for the future I plan to add a configuration file. Then we only require the files with the test data and the configuration.

This is the first time I am implementing such a framework. So if any of you has more experience and ideas how we can improve it, please let me know.

[1] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380 <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380>
[2] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress>
[3] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/adapter <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/adapter>
[4] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/pipelineElement <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/pipelineElement>
[5] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/fixtures/pipelineElement <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/fixtures/pipelineElement>

Re: Automated E2E Testing

Posted by Philipp Zehnder <ze...@apache.org>.

Thanks Dominik.

I extended the framework a bit.
Now it is possible to describe a processor test via a json file.

So the only things that are required for a new test are:
* The input.csv file
* A csv containing the expected results
* A json file with the configuration

See [1] for examples. For the future, we can think of moving these configuration directories directly into the extensions project and load them dynamically when the tests start.
For now, I would suggest leaving it in the Cypress directory, as I expect things will change in the near future.

Philipp


[1] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/fixtures/pipelineElement

> On 24. Jun 2021, at 08:13, Dominik Riemer <ri...@apache.org> wrote:
> 
> Hi Philipp,
> 
> very cool! That's a very important addition and will ease our future release processes.
> Maybe Chris has some idea concerning simulation of the PLC adapter?
> 
> Dominik
> 
> 
> On 2021/06/23 11:04:19, Philipp Zehnder <ze...@apache.org> wrote: 
>> Hi together,
>> 
>> since we have many different components, it is always difficult to keep them up to date and to test when something changes.
>> Therefore I started to implement an E2E test framework with the help of Cypress to test a StreamPipes instance when it was started.
>> 
>> The code is currently in a separate branch [1]. I created a new folder in the ui, called cypress [2], containing all the test code.
>> I also added a short README.
>> 
>> The main idea is to test the workflows of the application as well as the adapters and processing elements.
>> 
>> I created some example adapter tests [3], both for specific and generic adapters. 
>> For them, it must be ensured that we have access to the corresponding data source and that this contains some sample data.
>> This will not work for all the adapters, but should be feasible for most of them. (e.g. for Kafka we can create a pipeline containing a Kafka sink and then read the data with the adapter).
>> It would be great if we had some simulators for the PLCs, because then we can include tests for them as well. Does anyone have an idea how to do that?
>> 
>> Additionally, I added two examples to test processing elements [4].
>> For a test definition, two csv files must be added to [5]. One contains the input data set and one contains the expected result.
>> For the test a data set adapter is started, then a pipeline is build with the processing element and the data lake sink is used to store the data.
>> Once the data is processed the result is exported from the data lake and compared to the file ‘expected.csv’.
>> Currently the configuration for the processor test must be set in a typescript file, but for the future I plan to add a configuration file. Then we only require the files with the test data and the configuration.
>> 
>> This is the first time I am implementing such a framework. So if any of you has more experience and ideas how we can improve it, please let me know.
>> 
>> [1] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380 <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380>
>> [2] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress>
>> [3] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/adapter <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/adapter>
>> [4] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/pipelineElement <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/pipelineElement>
>> [5] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/fixtures/pipelineElement <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/fixtures/pipelineElement>
>> 
>> 
>> 
>>

Re: Automated E2E Testing

Posted by Dominik Riemer <ri...@apache.org>.

Hi Philipp,

very cool! That's a very important addition and will ease our future release processes.
Maybe Chris has some idea concerning simulation of the PLC adapter?

Dominik


On 2021/06/23 11:04:19, Philipp Zehnder <ze...@apache.org> wrote: 
> Hi together,
> 
> since we have many different components, it is always difficult to keep them up to date and to test when something changes.
> Therefore I started to implement an E2E test framework with the help of Cypress to test a StreamPipes instance when it was started.
> 
> The code is currently in a separate branch [1]. I created a new folder in the ui, called cypress [2], containing all the test code.
> I also added a short README.
> 
> The main idea is to test the workflows of the application as well as the adapters and processing elements.
> 
> I created some example adapter tests [3], both for specific and generic adapters. 
> For them, it must be ensured that we have access to the corresponding data source and that this contains some sample data.
> This will not work for all the adapters, but should be feasible for most of them. (e.g. for Kafka we can create a pipeline containing a Kafka sink and then read the data with the adapter).
> It would be great if we had some simulators for the PLCs, because then we can include tests for them as well. Does anyone have an idea how to do that?
> 
> Additionally, I added two examples to test processing elements [4].
> For a test definition, two csv files must be added to [5]. One contains the input data set and one contains the expected result.
> For the test a data set adapter is started, then a pipeline is build with the processing element and the data lake sink is used to store the data.
> Once the data is processed the result is exported from the data lake and compared to the file ‘expected.csv’.
> Currently the configuration for the processor test must be set in a typescript file, but for the future I plan to add a configuration file. Then we only require the files with the test data and the configuration.
> 
> This is the first time I am implementing such a framework. So if any of you has more experience and ideas how we can improve it, please let me know.
> 
> [1] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380 <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380>
> [2] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress>
> [3] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/adapter <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/adapter>
> [4] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/pipelineElement <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/integration/pipelineElement>
> [5] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/fixtures/pipelineElement <https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-380/ui/cypress/fixtures/pipelineElement>
> 
> 
> 
>