You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hop.apache.org by ha...@apache.org on 2023/03/15 09:59:35 UTC
[hop] branch master updated: initial version workflow logging samples, new and updated docs. fixes #2398
This is an automated email from the ASF dual-hosted git repository.
hansva pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/master by this push:
new 8fa2ac5e91 initial version workflow logging samples, new and updated docs. fixes #2398
new bdbdb32498 Merge pull request #2565 from bamaer/2398
8fa2ac5e91 is described below
commit 8fa2ac5e91ea40d51ea97144ec0cfdb4e78fafe1
Author: Bart Maertens <ba...@know.bi>
AuthorDate: Tue Mar 14 20:36:18 2023 +0100
initial version workflow logging samples, new and updated docs. fixes #2398
---
.../assets/images/metadata-types/workflow-log.png | Bin 0 -> 89710 bytes
docs/hop-user-manual/modules/ROOT/nav.adoc | 1 +
.../ROOT/pages/metadata-types/pipeline-log.adoc | 3 -
.../ROOT/pages/metadata-types/workflow-log.adoc | 19 +-
.../modules/ROOT/pages/pipeline/transforms.adoc | 1 +
.../pipeline/transforms/pipeline-logging.adoc | 19 +-
.../pipeline/transforms/workflow-logging.adoc | 94 +++
.../workflow/transform/WorkflowLoggingMeta.java | 2 +-
.../workflow-log/workflow-log-example.json | 13 +
.../samples/reflection/generate-fake-books.hwf | 149 ++++
.../samples/reflection/workflow-log-example.hpl | 775 +++++++++++++++++++++
11 files changed, 1066 insertions(+), 10 deletions(-)
diff --git a/docs/hop-user-manual/modules/ROOT/assets/images/metadata-types/workflow-log.png b/docs/hop-user-manual/modules/ROOT/assets/images/metadata-types/workflow-log.png
new file mode 100644
index 0000000000..fea8004ed9
Binary files /dev/null and b/docs/hop-user-manual/modules/ROOT/assets/images/metadata-types/workflow-log.png differ
diff --git a/docs/hop-user-manual/modules/ROOT/nav.adoc b/docs/hop-user-manual/modules/ROOT/nav.adoc
index fbf98fda94..65ec19721c 100644
--- a/docs/hop-user-manual/modules/ROOT/nav.adoc
+++ b/docs/hop-user-manual/modules/ROOT/nav.adoc
@@ -257,6 +257,7 @@ under the License.
*** xref:pipeline/transforms/verticabulkloader.adoc[Vertica Bulk Loader]
*** xref:pipeline/transforms/webservices.adoc[Web services lookup]
*** xref:pipeline/transforms/workflowexecutor.adoc[Workflow Executor]
+*** xref:pipeline/transforms/workflow-logging.adoc[Workflow Logging]
*** xref:pipeline/transforms/writetolog.adoc[Write to log]
*** xref:pipeline/transforms/xmlinputstream.adoc[XML Input Stream (StAX)]
*** xref:pipeline/transforms/xmljoin.adoc[XML Join]
diff --git a/docs/hop-user-manual/modules/ROOT/pages/metadata-types/pipeline-log.adoc b/docs/hop-user-manual/modules/ROOT/pages/metadata-types/pipeline-log.adoc
index 956957c563..6e63d54462 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/metadata-types/pipeline-log.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/metadata-types/pipeline-log.adoc
@@ -60,6 +60,3 @@ image:metadata-types/pipeline-log.png[Pipeline Log, width="75%"]
|Interval in seconds|30|if executed periodically, indicates the interval at which the pipeline log is executed
|===
-== Samples
-
-None
\ No newline at end of file
diff --git a/docs/hop-user-manual/modules/ROOT/pages/metadata-types/workflow-log.adoc b/docs/hop-user-manual/modules/ROOT/pages/metadata-types/workflow-log.adoc
index 51cfec7637..6461161bd1 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/metadata-types/workflow-log.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/metadata-types/workflow-log.adoc
@@ -17,6 +17,8 @@ under the License.
:imagesdir: ../../assets/images/
:page-pagination:
:description: Allows to log the activity of a workflow with a pipeline. A workflow log streams logging information from a running workflow to a pipeline. This receiving pipeline can then process this logging information and e.g. write to a relational or NoSQL database, a Kafka topic etc
+:openvar: ${
+:closevar: }
= Workflow Log
@@ -27,11 +29,21 @@ image:icons/workflow-log.svg[]
Allows to log the activity of a workflow with a pipeline.
A workflow log streams logging information from a running workflow to a pipeline.
-This receiving pipeline can then process this logging information and e.g. write to a relational or NoSQL database, a Kafka topic etc
+
+The only requirement for the receiving pipeline is that it starts with a xref:pipeline/transforms/workflow-logging.adoc[Workflow Logging] transform. Other than that, the logging pipeline is "just another pipeline".
+In this logging pipeline, you can process the logging information and e.g. write to a relational or NoSQL database, a Kafka topic etc
+
+== Samples
+
+The samples project comes with an example workflow log.
+
+Check the metadata perspective for the workflow log `workflow-log-example`. This workflow log is configured to send logging information for the workflow `{openvar}PROJECT_HOME{closevar}/reflection/generate-fake-books.hwf` to the logging pipeline `{openvar}PROJECT_HOME{closevar}/reflection/workflow-log-example.hpl`.
+
+image:metadata-types/workflow-log.png[Workflow Log, width="75%"]
== Related Plugins
-* Workflow Logging
+* xref:pipeline/transforms/workflow-logging.adoc[Workflow Logging]
== Options
@@ -48,6 +60,3 @@ This receiving pipeline can then process this logging information and e.g. write
|Interval in seconds|30|if executed periodically, indicates the interval at which the workflow log is executed
|===
-== Samples
-
-None
\ No newline at end of file
diff --git a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms.adoc b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms.adoc
index 137971bb4d..357ebcc072 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms.adoc
@@ -210,6 +210,7 @@ The pages nested under this topic contain information on how to use the transfor
* xref:pipeline/transforms/valuemapper.adoc[Value Mapper]
* xref:pipeline/transforms/webservices.adoc[Web services lookup]
* xref:pipeline/transforms/workflowexecutor.adoc[Workflow Executor]
+* xref:pipeline/transforms/workflow-logging.adoc[Workflow Logging]
* xref:pipeline/transforms/writetolog.adoc[Write to log]
* xref:pipeline/transforms/xmlinputstream.adoc[XML Input Stream (StAX)]
* xref:pipeline/transforms/xmljoin.adoc[XML Join]
diff --git a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pipeline-logging.adoc b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pipeline-logging.adoc
index 3adf06cc8d..1229ee8c7e 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pipeline-logging.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pipeline-logging.adoc
@@ -17,15 +17,32 @@ under the License.
:imagesdir: ../../assets/images/
:page-pagination:
:description: The pipeline logging transform gives you detailed logging information about a pipeline and its transforms.
+:openvar: ${
+:closevar: }
-= Pipeline Logging
+= image:icons/pipeline-log.svg[Pipeline logging transform Icon, role="image-doc-icon"] Pipeline Logging
+[%noheader,cols="3a,1a", role="table-no-borders" ]
+|===
+|
== Description
the Pipeline Logging transform receives its input from a xref:metadata-types/pipeline-log.adoc[Pipeline Log] metadata item.
The transform requires very little configuration, but provides a lot of information.
+
+|
+== Supported Engines
+[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
+!===
+!Hop Engine! image:check_mark.svg[Supported, 24]
+!Spark! image:question_mark.svg[Maybe Supported, 24]
+!Flink! image:question_mark.svg[Maybe Supported, 24]
+!Dataflow! image:question_mark.svg[Maybe Supported, 24]
+!===
+|===
+
== Related Plugins
* xref:metadata-types/pipeline-log.adoc[Pipeline Log]
diff --git a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/workflow-logging.adoc b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/workflow-logging.adoc
new file mode 100644
index 0000000000..35177d065b
--- /dev/null
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/workflow-logging.adoc
@@ -0,0 +1,94 @@
+////
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+////
+:imagesdir: ../../assets/images/
+:page-pagination:
+:description: The workflow logging transform gives you detailed logging information about a workflow and its actions.
+:openvar: ${
+:closevar: }
+
+= image:icons/workflow-log.svg[Workflow logging transform Icon, role="image-doc-icon"] Workflow logging
+
+
+[%noheader,cols="3a,1a", role="table-no-borders" ]
+|===
+|
+== Description
+
+the Workflow Logging transform receives its input from a xref:metadata-types/workflow-log.adoc[Workflow Log] metadata item.
+
+The transform requires very little configuration, but provides a lot of information.
+
+|
+== Supported Engines
+[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
+!===
+!Hop Engine! image:check_mark.svg[Supported, 24]
+!Spark! image:question_mark.svg[Maybe Supported, 24]
+!Flink! image:question_mark.svg[Maybe Supported, 24]
+!Dataflow! image:question_mark.svg[Maybe Supported, 24]
+!===
+|===
+
+== Related Plugins
+
+* xref:metadata-types/workflow-log.adoc[Workflow Log]
+
+== Options
+
+[%header, width="90%", cols="1,5"]
+|===
+|Option|Description
+|Transform name|Name of this transform
+|Also log action details a|
+
+* checked: the transform outputs both workflow and action logging and metrics. In this case, the logging will contain one line per action. For each action, the workflow logging and metrics information is repeated.
+* unchecked: the transform only outputs workflow logging and metrics.
+|===
+
+== Output Fields
+
+[%header, width="90%", cols="1,1,5"]
+|===
+ |Fieldname|Type|Description
+3+|**Workflow Logging Fields**
+|loggingDate|Date|the date and time this workflow was executed
+|loggingPhase|String|the phase this workflow is in at the time of logging (e.g. 'end')
+|workflowName|String|the logged workflow name
+|workflowFilename|String|the logged workflow file name
+|workflowStart|Date|execution start date for the logged workflow
+|workflowEnd|Date|execution end date for the logged workflow
+|workflowLogChannelId|String|logging channel id for the logged workflow
+|parentLogChannelId|String|parent logging channel id for the logged workflow (e.g. the channel id of the parent workflow)
+|workflowLogging|String|full logging text for the logged workflow
+|workflowErrorCount|Integer|number of errors that occurred in the logged workflow execution
+|workflowStatusDescription|String|status description for the logged workflow (e.g. 'Finished')
+3+|**Action Logging Fields**
+|actionName|String|the name of the action in the logged pipeline
+|actionNr|Integer|the number of the current action
+|actionResult|the result (exit code, Y/N) of the current action
+|actionLogChannelId|String|log channel id for the current action
+|actionLoggingText|String|logging text for the current action
+|actionErrors|Integer|number of errors for the current action
+|actionLogDate|String|status of the current action (e.g. 'Finished')
+|actionDuration|Integer|total duration for the current action
+|actionExitStatus|exit status of the current action
+|actionNrFilesRetrieved|the number of files retrieved by the current transform (if applicable)
+|actionFilename|the filename to use for the current action, e.g. the workflow or pipeline the current action runs.
+|actionComment|an optional comment added to the logs, e.g. `Workflow execution finished`
+|actionReason|an optional reason provided by the current action
+|===
+
diff --git a/plugins/misc/reflection/src/main/java/org/apache/hop/reflection/workflow/transform/WorkflowLoggingMeta.java b/plugins/misc/reflection/src/main/java/org/apache/hop/reflection/workflow/transform/WorkflowLoggingMeta.java
index e96f60a75e..9e33e7eba5 100644
--- a/plugins/misc/reflection/src/main/java/org/apache/hop/reflection/workflow/transform/WorkflowLoggingMeta.java
+++ b/plugins/misc/reflection/src/main/java/org/apache/hop/reflection/workflow/transform/WorkflowLoggingMeta.java
@@ -36,7 +36,7 @@ import org.apache.hop.pipeline.transform.TransformMeta;
name = "i18n::WorkflowLogging.Transform.Name",
description = "i18n::WorkflowLogging.Transform.Description",
categoryDescription = "i18n:org.apache.hop.pipeline.transform:BaseTransform.Category.Input",
- documentationUrl = "/logging/logging-reflection.html",
+ documentationUrl = "/pipeline/transforms/pipeline-logging.html",
image = "workflow-log.svg",
keywords = "i18n::WorkflowLoggingMeta.keyword")
public class WorkflowLoggingMeta extends BaseTransformMeta<WorkflowLogging, WorkflowLoggingData> {
diff --git a/plugins/misc/reflection/src/main/samples/metadata/workflow-log/workflow-log-example.json b/plugins/misc/reflection/src/main/samples/metadata/workflow-log/workflow-log-example.json
new file mode 100644
index 0000000000..95f784fad8
--- /dev/null
+++ b/plugins/misc/reflection/src/main/samples/metadata/workflow-log/workflow-log-example.json
@@ -0,0 +1,13 @@
+{
+ "loggingParentsOnly": true,
+ "name": "workflow-log-example",
+ "workflowToLog": [
+ "${PROJECT_HOME}/reflection/generate-fake-books.hwf"
+ ],
+ "intervalInSeconds": "30",
+ "executingPeriodically": false,
+ "pipelineFilename": "${PROJECT_HOME}/reflection/workflow-log-example.hpl",
+ "executingAtEnd": true,
+ "enabled": true,
+ "executingAtStart": true
+}
\ No newline at end of file
diff --git a/plugins/misc/reflection/src/main/samples/reflection/generate-fake-books.hwf b/plugins/misc/reflection/src/main/samples/reflection/generate-fake-books.hwf
new file mode 100644
index 0000000000..3acc7e3680
--- /dev/null
+++ b/plugins/misc/reflection/src/main/samples/reflection/generate-fake-books.hwf
@@ -0,0 +1,149 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+-->
+<workflow>
+ <name>generate-fake-books</name>
+ <name_sync_with_filename>Y</name_sync_with_filename>
+ <description/>
+ <extended_description/>
+ <workflow_version/>
+ <created_user>-</created_user>
+ <created_date>2023/03/14 19:32:36.099</created_date>
+ <modified_user>-</modified_user>
+ <modified_date>2023/03/14 19:32:36.099</modified_date>
+ <parameters>
+ </parameters>
+ <actions>
+ <action>
+ <name>Start</name>
+ <description/>
+ <type>SPECIAL</type>
+ <attributes/>
+ <DayOfMonth>1</DayOfMonth>
+ <hour>12</hour>
+ <intervalMinutes>60</intervalMinutes>
+ <intervalSeconds>0</intervalSeconds>
+ <minutes>0</minutes>
+ <repeat>N</repeat>
+ <schedulerType>0</schedulerType>
+ <weekDay>1</weekDay>
+ <parallel>N</parallel>
+ <xloc>50</xloc>
+ <yloc>80</yloc>
+ <attributes_hac/>
+ </action>
+ <action>
+ <name>Pipeline</name>
+ <description/>
+ <type>PIPELINE</type>
+ <attributes/>
+ <filename>${PROJECT_HOME}/reflection/generate-fake-books.hpl</filename>
+ <params_from_previous>N</params_from_previous>
+ <exec_per_row>N</exec_per_row>
+ <clear_rows>N</clear_rows>
+ <clear_files>N</clear_files>
+ <set_logfile>N</set_logfile>
+ <logfile/>
+ <logext/>
+ <add_date>N</add_date>
+ <add_time>N</add_time>
+ <loglevel>Basic</loglevel>
+ <set_append_logfile>N</set_append_logfile>
+ <wait_until_finished>Y</wait_until_finished>
+ <create_parent_folder>N</create_parent_folder>
+ <run_configuration>Spark</run_configuration>
+ <parameters>
+ <pass_all_parameters>Y</pass_all_parameters>
+ </parameters>
+ <parallel>N</parallel>
+ <xloc>209</xloc>
+ <yloc>80</yloc>
+ <attributes_hac/>
+ </action>
+ <action>
+ <name>Write to log</name>
+ <description/>
+ <type>WRITE_TO_LOG</type>
+ <attributes/>
+ <logmessage/>
+ <loglevel/>
+ <logsubject/>
+ <parallel>N</parallel>
+ <xloc>368</xloc>
+ <yloc>80</yloc>
+ <attributes_hac/>
+ </action>
+ <action>
+ <name>Dummy</name>
+ <description/>
+ <type>DUMMY</type>
+ <attributes/>
+ <parallel>N</parallel>
+ <xloc>527</xloc>
+ <yloc>80</yloc>
+ <attributes_hac/>
+ </action>
+ </actions>
+ <hops>
+ <hop>
+ <from>Start</from>
+ <to>Pipeline</to>
+ <enabled>Y</enabled>
+ <evaluation>Y</evaluation>
+ <unconditional>Y</unconditional>
+ </hop>
+ <hop>
+ <from>Pipeline</from>
+ <to>Write to log</to>
+ <enabled>Y</enabled>
+ <evaluation>Y</evaluation>
+ <unconditional>N</unconditional>
+ </hop>
+ <hop>
+ <from>Write to log</from>
+ <to>Dummy</to>
+ <enabled>Y</enabled>
+ <evaluation>Y</evaluation>
+ <unconditional>N</unconditional>
+ </hop>
+ </hops>
+ <notepads>
+ <notepad>
+ <backgroundcolorblue>251</backgroundcolorblue>
+ <backgroundcolorgreen>232</backgroundcolorgreen>
+ <backgroundcolorred>201</backgroundcolorred>
+ <bordercolorblue>90</bordercolorblue>
+ <bordercolorgreen>58</bordercolorgreen>
+ <bordercolorred>14</bordercolorred>
+ <fontbold>N</fontbold>
+ <fontcolorblue>90</fontcolorblue>
+ <fontcolorgreen>58</fontcolorgreen>
+ <fontcolorred>14</fontcolorred>
+ <fontitalic>N</fontitalic>
+ <fontname>Noto Sans</fontname>
+ <fontsize>10</fontsize>
+ <height>27</height>
+ <xloc>48</xloc>
+ <yloc>16</yloc>
+ <note>this workflow doesn't really do anything, it's mainly used to be executed by ${PROJECT_HOME}/reflection/workflow-log-example.hpl</note>
+ <width>729</width>
+ </notepad>
+ </notepads>
+ <attributes/>
+</workflow>
diff --git a/plugins/misc/reflection/src/main/samples/reflection/workflow-log-example.hpl b/plugins/misc/reflection/src/main/samples/reflection/workflow-log-example.hpl
new file mode 100644
index 0000000000..68ecfdc5db
--- /dev/null
+++ b/plugins/misc/reflection/src/main/samples/reflection/workflow-log-example.hpl
@@ -0,0 +1,775 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+-->
+<pipeline>
+ <info>
+ <name>workflow-log-example</name>
+ <name_sync_with_filename>Y</name_sync_with_filename>
+ <description/>
+ <extended_description/>
+ <pipeline_version/>
+ <pipeline_type>Normal</pipeline_type>
+ <parameters>
+ </parameters>
+ <capture_transform_performance>N</capture_transform_performance>
+ <transform_performance_capturing_delay>1000</transform_performance_capturing_delay>
+ <transform_performance_capturing_size_limit>100</transform_performance_capturing_size_limit>
+ <created_user>-</created_user>
+ <created_date>2023/03/14 19:31:44.374</created_date>
+ <modified_user>-</modified_user>
+ <modified_date>2023/03/14 19:31:44.374</modified_date>
+ </info>
+ <notepads>
+ </notepads>
+ <order>
+ <hop>
+ <from>workflow logs</from>
+ <to>unique workflow data</to>
+ <enabled>Y</enabled>
+ </hop>
+ <hop>
+ <from>action logs</from>
+ <to>workflow-log-action.csv out</to>
+ <enabled>Y</enabled>
+ </hop>
+ <hop>
+ <from>Workflow Logging</from>
+ <to>workflow logs</to>
+ <enabled>Y</enabled>
+ </hop>
+ <hop>
+ <from>Workflow Logging</from>
+ <to>action logs</to>
+ <enabled>Y</enabled>
+ </hop>
+ <hop>
+ <from>unique workflow data</from>
+ <to>workflow-log.csv out</to>
+ <enabled>Y</enabled>
+ </hop>
+ </order>
+ <transform>
+ <name>Workflow Logging</name>
+ <type>WorkflowLogging</type>
+ <description/>
+ <distribute>N</distribute>
+ <custom_distribution/>
+ <copies>1</copies>
+ <partitioning>
+ <method>none</method>
+ <schema_name/>
+ </partitioning>
+ <log_transforms>Y</log_transforms>
+ <attributes/>
+ <GUI>
+ <xloc>200</xloc>
+ <yloc>144</yloc>
+ </GUI>
+ </transform>
+ <transform>
+ <name>action logs</name>
+ <type>SelectValues</type>
+ <description/>
+ <distribute>Y</distribute>
+ <custom_distribution/>
+ <copies>1</copies>
+ <partitioning>
+ <method>none</method>
+ <schema_name/>
+ </partitioning>
+ <fields>
+ <field>
+ <name>loggingDate</name>
+ </field>
+ <field>
+ <name>workflowName</name>
+ </field>
+ <field>
+ <name>workflowFilename</name>
+ </field>
+ <field>
+ <name>workflowLogChannelId</name>
+ </field>
+ <field>
+ <name>workflowParentLogChannelId</name>
+ </field>
+ <field>
+ <name>actionName</name>
+ </field>
+ <field>
+ <name>actionNr</name>
+ </field>
+ <field>
+ <name>actionResult</name>
+ </field>
+ <field>
+ <name>actionLogChannelId</name>
+ </field>
+ <field>
+ <name>actionLoggingText</name>
+ </field>
+ <field>
+ <name>actionErrors</name>
+ </field>
+ <field>
+ <name>actionLogDate</name>
+ </field>
+ <field>
+ <name>actionDuration</name>
+ </field>
+ <field>
+ <name>actionExitStatus</name>
+ </field>
+ <field>
+ <name>actionNrFilesRetrieved</name>
+ </field>
+ <field>
+ <name>actionFilename</name>
+ </field>
+ <field>
+ <name>actionComment</name>
+ </field>
+ <field>
+ <name>actionReason</name>
+ </field>
+ <select_unspecified>N</select_unspecified>
+ </fields>
+ <attributes/>
+ <GUI>
+ <xloc>368</xloc>
+ <yloc>272</yloc>
+ </GUI>
+ </transform>
+ <transform>
+ <name>unique workflow data</name>
+ <type>SortRows</type>
+ <description/>
+ <distribute>Y</distribute>
+ <custom_distribution/>
+ <copies>1</copies>
+ <partitioning>
+ <method>none</method>
+ <schema_name/>
+ </partitioning>
+ <directory>${java.io.tmpdir}</directory>
+ <prefix>out</prefix>
+ <sort_size>1000000</sort_size>
+ <free_memory/>
+ <compress>N</compress>
+ <compress_variable/>
+ <unique_rows>Y</unique_rows>
+ <fields>
+ <field>
+ <name>loggingDate</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>loggingPhase</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>workflowName</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>workflowFilename</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>workflowStart</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>workflowEnd</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>workflowLogChannelId</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>workflowParentLogChannelId</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>workflowLogging</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>workflowErrorCount</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ <field>
+ <name>workflowStatusDescription</name>
+ <ascending>Y</ascending>
+ <case_sensitive>N</case_sensitive>
+ <collator_enabled>N</collator_enabled>
+ <collator_strength>0</collator_strength>
+ <presorted>N</presorted>
+ </field>
+ </fields>
+ <attributes/>
+ <GUI>
+ <xloc>544</xloc>
+ <yloc>144</yloc>
+ </GUI>
+ </transform>
+ <transform>
+ <name>workflow logs</name>
+ <type>SelectValues</type>
+ <description/>
+ <distribute>Y</distribute>
+ <custom_distribution/>
+ <copies>1</copies>
+ <partitioning>
+ <method>none</method>
+ <schema_name/>
+ </partitioning>
+ <fields>
+ <field>
+ <name>loggingDate</name>
+ </field>
+ <field>
+ <name>loggingPhase</name>
+ </field>
+ <field>
+ <name>workflowName</name>
+ </field>
+ <field>
+ <name>workflowFilename</name>
+ </field>
+ <field>
+ <name>workflowStart</name>
+ </field>
+ <field>
+ <name>workflowEnd</name>
+ </field>
+ <field>
+ <name>workflowLogChannelId</name>
+ </field>
+ <field>
+ <name>workflowParentLogChannelId</name>
+ </field>
+ <field>
+ <name>workflowLogging</name>
+ </field>
+ <field>
+ <name>workflowErrorCount</name>
+ </field>
+ <field>
+ <name>workflowStatusDescription</name>
+ </field>
+ <select_unspecified>N</select_unspecified>
+ </fields>
+ <attributes/>
+ <GUI>
+ <xloc>368</xloc>
+ <yloc>144</yloc>
+ </GUI>
+ </transform>
+ <transform>
+ <name>workflow-log-action.csv out</name>
+ <type>TextFileOutput</type>
+ <description/>
+ <distribute>Y</distribute>
+ <custom_distribution/>
+ <copies>1</copies>
+ <partitioning>
+ <method>none</method>
+ <schema_name/>
+ </partitioning>
+ <separator>;</separator>
+ <enclosure>"</enclosure>
+ <enclosure_forced>N</enclosure_forced>
+ <enclosure_fix_disabled>N</enclosure_fix_disabled>
+ <header>Y</header>
+ <footer>N</footer>
+ <format>DOS</format>
+ <compression>None</compression>
+ <encoding/>
+ <endedLine/>
+ <fileNameInField>N</fileNameInField>
+ <fileNameField/>
+ <create_parent_folder>Y</create_parent_folder>
+ <file>
+ <name>${PROJECT_HOME}/reflection/output/workflow-log-action</name>
+ <servlet_output>N</servlet_output>
+ <do_not_open_new_file_init>Y</do_not_open_new_file_init>
+ <extention>csv</extention>
+ <append>N</append>
+ <split>N</split>
+ <haspartno>N</haspartno>
+ <add_date>N</add_date>
+ <add_time>N</add_time>
+ <SpecifyFormat>N</SpecifyFormat>
+ <date_time_format/>
+ <add_to_result_filenames>Y</add_to_result_filenames>
+ <pad>N</pad>
+ <fast_dump>N</fast_dump>
+ <splitevery/>
+ </file>
+ <fields>
+ <field>
+ <name>loggingDate</name>
+ <type>Date</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowName</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowFilename</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowLogChannelId</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowParentLogChannelId</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionName</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionNr</name>
+ <type>Integer</type>
+ <format>0</format>
+ <currency>$</currency>
+ <decimal>.</decimal>
+ <group>,</group>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionResult</name>
+ <type>Boolean</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionLogChannelId</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionLoggingText</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionErrors</name>
+ <type>Integer</type>
+ <format>0</format>
+ <currency>$</currency>
+ <decimal>.</decimal>
+ <group>,</group>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionLogDate</name>
+ <type>Date</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionDuration</name>
+ <type>Integer</type>
+ <format>0</format>
+ <currency>$</currency>
+ <decimal>.</decimal>
+ <group>,</group>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionExitStatus</name>
+ <type>Integer</type>
+ <format>0</format>
+ <currency>$</currency>
+ <decimal>.</decimal>
+ <group>,</group>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionNrFilesRetrieved</name>
+ <type>Integer</type>
+ <format>0</format>
+ <currency>$</currency>
+ <decimal>.</decimal>
+ <group>,</group>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionFilename</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionComment</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>actionReason</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ </fields>
+ <attributes/>
+ <GUI>
+ <xloc>800</xloc>
+ <yloc>272</yloc>
+ </GUI>
+ </transform>
+ <transform>
+ <name>workflow-log.csv out</name>
+ <type>TextFileOutput</type>
+ <description/>
+ <distribute>Y</distribute>
+ <custom_distribution/>
+ <copies>1</copies>
+ <partitioning>
+ <method>none</method>
+ <schema_name/>
+ </partitioning>
+ <separator>;</separator>
+ <enclosure>"</enclosure>
+ <enclosure_forced>N</enclosure_forced>
+ <enclosure_fix_disabled>N</enclosure_fix_disabled>
+ <header>Y</header>
+ <footer>N</footer>
+ <format>DOS</format>
+ <compression>None</compression>
+ <encoding/>
+ <endedLine/>
+ <fileNameInField>N</fileNameInField>
+ <fileNameField/>
+ <create_parent_folder>Y</create_parent_folder>
+ <file>
+ <name>${PROJECT_HOME}/reflection/output/workflow-log</name>
+ <servlet_output>N</servlet_output>
+ <do_not_open_new_file_init>Y</do_not_open_new_file_init>
+ <extention>csv</extention>
+ <append>N</append>
+ <split>N</split>
+ <haspartno>N</haspartno>
+ <add_date>N</add_date>
+ <add_time>N</add_time>
+ <SpecifyFormat>N</SpecifyFormat>
+ <date_time_format/>
+ <add_to_result_filenames>Y</add_to_result_filenames>
+ <pad>N</pad>
+ <fast_dump>N</fast_dump>
+ <splitevery/>
+ </file>
+ <fields>
+ <field>
+ <name>loggingDate</name>
+ <type>Date</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>loggingPhase</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowName</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowFilename</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowStart</name>
+ <type>Date</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowEnd</name>
+ <type>Date</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowLogChannelId</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowParentLogChannelId</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowLogging</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowErrorCount</name>
+ <type>Integer</type>
+ <format>0</format>
+ <currency>$</currency>
+ <decimal>.</decimal>
+ <group>,</group>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ <field>
+ <name>workflowStatusDescription</name>
+ <type>String</type>
+ <format/>
+ <currency/>
+ <decimal/>
+ <group/>
+ <nullif/>
+ <trim_type>both</trim_type>
+ <length>-1</length>
+ <precision>-1</precision>
+ </field>
+ </fields>
+ <attributes/>
+ <GUI>
+ <xloc>800</xloc>
+ <yloc>144</yloc>
+ </GUI>
+ </transform>
+ <transform_error_handling>
+ </transform_error_handling>
+ <attributes/>
+</pipeline>