[jira] [Updated] (MINIFICPP-967) Improve SITs

["Gábor Gyimesi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MINIFICPP-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gábor Gyimesi updated MINIFICPP-967:
------------------------------------
    Fix Version/s:     (was: 0.14.0)

> Improve SITs
> ------------
>
>                 Key: MINIFICPP-967
>                 URL: https://issues.apache.org/jira/browse/MINIFICPP-967
>             Project: Apache NiFi MiNiFi C++
>          Issue Type: Epic
>            Reporter: Marc Parisi
>            Assignee: Arpad Boda
>            Priority: Blocker
>
> I'm using this ticket to collect various other tickets ( closed and open ) for adding them to our docker test framework.
>  
> In my opinion this is the highest need in 0.7.0 in 0.8.0 and 0.9.0. Bugs will arise as a result of this; however, there have been a variety of tickets created and closed across this and elsewhere – so let's gather them into a single EPIC so we can all work on them together.
>  
> docker-verify
> We have several stages of testing as you have across any project: unit, integration, and SIT ( system integration testing ).
>  
> 1) V&V – Verify and validate interactions between internal and external components
>    In many cases, we can create all appropriate BV, EC, and introspective test cases in unit and integration tests, but there are many that we cannot capture without integration with a C2 server, NiFi Instance, Kafka server, or other type of server. We should not attempt to launch these services in unit or integration tests. External services and requirements for comms should be verified and validated in docker contianers.
> 2)  Regression Testing – we should use this for all releases and before merging. This may mean adding a targets if necessary, but the important thing is that we define expectations within this test framework ( and the DSL ) and adhere to them. This may mean that we are testing for memory leaks. We can do this in docker tests more easily than unit/integration tests. ASAN is a great goal to have for upcoming releases and I think docker tests are where we can build this functionality into so that we can test across versions of systems, glibc, and musl.
>  
> 3) We have many versions of software that we can build upon. While we can mitigate factors with including our own version of system libs, this won't solve all issues, and will not be something that all consumers want. So we can mitigate these desires by using docker tests to splay across platforms and build systems, executing docker tests.
>  
> 4) Fuzzing. Fuzzing can and often should be done at every level, but we can very easily define ways in the DSL to fuzz properties in processors. This mechanism should allow us to better test boundary values and equivalence classes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)