[incubator-hop-docs] branch asf-site updated: HOP-2341: make integration test documentation more complete

[ha...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

hansva pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-hop-docs.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new 5de59b7  HOP-2341: make integration test documentation more complete
     new 85b98a2  Merge pull request #24 from hansva/HOP-2341
5de59b7 is described below

commit 5de59b71964e12fee8e1682197b33693aef68d3e
Author: Hans Van Akelyen <ha...@gmail.com>
AuthorDate: Mon Dec 21 21:50:09 2020 +0100

    HOP-2341: make integration test documentation more complete
---
 .../ROOT/assets/images/integration-tests-001.png   | Bin 0 -> 106635 bytes
 .../ROOT/assets/images/integration-tests-002.png   | Bin 0 -> 87096 bytes
 .../ROOT/assets/images/integration-tests-003.png   | Bin 0 -> 12936 bytes
 .../ROOT/assets/images/integration-tests-004.png   | Bin 0 -> 20747 bytes
 .../modules/ROOT/pages/integration-testing.adoc    | 101 ++++++++++++++++++---
 5 files changed, 89 insertions(+), 12 deletions(-)

diff --git a/hop-dev-manual/modules/ROOT/assets/images/integration-tests-001.png b/hop-dev-manual/modules/ROOT/assets/images/integration-tests-001.png
new file mode 100644
index 0000000..b1a2fd6
Binary files /dev/null and b/hop-dev-manual/modules/ROOT/assets/images/integration-tests-001.png differ
diff --git a/hop-dev-manual/modules/ROOT/assets/images/integration-tests-002.png b/hop-dev-manual/modules/ROOT/assets/images/integration-tests-002.png
new file mode 100644
index 0000000..5945fbf
Binary files /dev/null and b/hop-dev-manual/modules/ROOT/assets/images/integration-tests-002.png differ
diff --git a/hop-dev-manual/modules/ROOT/assets/images/integration-tests-003.png b/hop-dev-manual/modules/ROOT/assets/images/integration-tests-003.png
new file mode 100644
index 0000000..363cffb
Binary files /dev/null and b/hop-dev-manual/modules/ROOT/assets/images/integration-tests-003.png differ
diff --git a/hop-dev-manual/modules/ROOT/assets/images/integration-tests-004.png b/hop-dev-manual/modules/ROOT/assets/images/integration-tests-004.png
new file mode 100644
index 0000000..70b4c8b
Binary files /dev/null and b/hop-dev-manual/modules/ROOT/assets/images/integration-tests-004.png differ
diff --git a/hop-dev-manual/modules/ROOT/pages/integration-testing.adoc b/hop-dev-manual/modules/ROOT/pages/integration-testing.adoc
index 876d655..2fb4206 100644
--- a/hop-dev-manual/modules/ROOT/pages/integration-testing.adoc
+++ b/hop-dev-manual/modules/ROOT/pages/integration-testing.adoc
@@ -4,22 +4,99 @@
 The Apache Hop team has created an integration testing framework that is being used to test key components of the software.
 The integration tests run as a daily build on Jenkins and can be found https://ci-builds.apache.org/job/Hop/job/Hop-integration-tests/[here].
 
-== Current Limitations
 
-* Only workflows are supported
-* No external sources to test against (no databases or other services available)
-* Only local engines will work (including the Apache Beam Direct Runners)
-* No extra parameters can be provided to hop-run see https://issues.apache.org/jira/browse/HOP-2271[HOP-2271]
-* No true negative test can be created see https://issues.apache.org/jira/browse/HOP-2272[HOP-2272]
+== Adding tests to an existing project
 
+There are two possible scenarios that you can follow, the first is adding a test to one of the existing projects containing tests. The second scenario is creating a new project.
 
-== Adding a test
+First start by creating a local clone of the Apache Hop repository, instruction to do this can be found xref:setup-dev-environment.adoc[here].
 
-The tests are stored in the source code repository and can be found in the https://github.com/apache/incubator-hop/tree/master/integration-tests[integration-tests] folder.
-Each folder represents an integration test that will be executed daily using the master branch version of the software.
+. Open the Hop gui and add one of the integration testing projects to your gui
++
+In this example we will be opening the `transforms` project and add an extra test in this project.
+When the UI asks to add an environment or lifecycle press no, using this is not supported 
++
+image::integration-tests-001.png[]
 
-Each folder contains a Hop project and all metadata needed for that project, they are dynamically loaded and the `main.hwf` is executed using the `local` workflow engine.
+. naming convention inside the project
++
+when taking a look inside the project you will see a distinct naming pattern is used.
+each test consists of one workflow and one or more pipelines to execute the test.
++
+The workflow *must* always start with `main-` followed by number and a logical explanation of what the test is checking. the pipelines linked to the workflow start with the same number the workflow uses to make a clear distinction which pipelines are linked to which test.
++
+image::integration-tests-002.png[]
++
+The workflows themselves can be straightforward, the exit code (success/abort) is used to determine if the workflow has passed or failed.
++
+note:: You can also make an integration test to test something that must fail. Then you simply link the false hop to a success and the true hop to an abort action.
++
+image::integration-tests-003.png[]
++
+image::integration-tests-004.png[]
++
+. create a unit test
++
+Now you can add your own workflow to the project folder. Make sure you have a Jira ticket you can refer to when creating a pull request.
 
-Adding a test is as simple as creating a new project that contains a `main.hwf` in the root folder and all test files should be self contained in the project folder.
-Please give each folder a HOP issue number or a self explaining foldername as this name is used for the test report in Jenkins.
 
+== Adding a new project
+
+You can also add a new project to the integration test folder. Give the project a meaningful name that explains the subject you will be testing.
+
+important:: the project should have a workflow engine named `local`
+
+== Special checks
+
+In some cases you want to check the logs to see if a specific value is present to mark your test as passed or failed. The script used in `main-0001-static-partitioning` can be helpful.
+
+
+Code used in JavaScript Action in the workflow:
+
+[source,java]
+----
+var txt = previous_result.getLogText();
+
+
+var ok = true;
+
+var expectedVariables = [ 
+   "partitioned.0 - Internal.Transform.Partition.ID = P1",
+   "partitioned.1 - Internal.Transform.Partition.ID = P2",
+   "partitioned.2 - Internal.Transform.Partition.ID = P3"
+ ];
+
+for (var i = 0 ; i<expectedVariables.length ; i++) {
+  var expectedVariable = expectedVariables[i];
+  if ( !txt.contains(expectedVariable)) {
+    ok = false;
+    log.logError("Expected variable expression '"+expectedVariable+"' was not logged at least once");
+  }
+}
+
+var expectedValues = [ 
+   "partitioned.1 - id = 1",
+   "partitioned.2 - id = 2",
+   "partitioned.0 - id = 3",
+   "partitioned.1 - id = 4",
+   "partitioned.2 - id = 5",
+   "partitioned.0 - id = 6",
+   "partitioned.1 - id = 7",
+   "partitioned.2 - id = 8",
+   "partitioned.0 - id = 9",
+   "partitioned.1 - id = 10",
+ ];
+
+for (var i = 0 ; i<expectedValues.length ; i++) {
+  var expectedValue = expectedValues[i];
+  if ( !txt.contains(expectedValue)) {
+    ok = false;
+    log.logError("Value logged as '"+expectedValue+"' was not logged at least once");
+  }
+}
+
+
+ok;
+----
+
+This script reads the log returned by the previous pipeline and parses it to search for values.