You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hop.apache.org by ha...@apache.org on 2021/03/23 07:39:48 UTC
[incubator-hop] branch master updated: HOP-2641 escape variable
references, heading updates
This is an automated email from the ASF dual-hosted git repository.
hansva pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-hop.git
The following commit(s) were added to refs/heads/master by this push:
new 2f487ca HOP-2641 escape variable references, heading updates
new 36c9067 Merge pull request #690 from bamaer/HOP-2641
2f487ca is described below
commit 2f487ca30bff2915c5e8ce4977cb40a594f2d68b
Author: Bart Maertens <ba...@know.bi>
AuthorDate: Tue Mar 23 08:28:10 2021 +0100
HOP-2641 escape variable references, heading updates
---
plugins/misc/projects/src/main/doc/projects.adoc | 2 +-
plugins/transforms/dropbox/src/main/doc/dropboxinput.adoc | 2 +-
plugins/transforms/dropbox/src/main/doc/dropboxoutput.adoc | 2 +-
plugins/transforms/dynamicsqlrow/src/main/doc/dynamicsqlrow.adoc | 2 +-
plugins/transforms/filterrows/src/main/doc/filterrows.adoc | 4 +++-
plugins/transforms/metainject/src/main/{ => doc}/metainject.adoc | 2 +-
plugins/transforms/sql/src/main/doc/sql.adoc | 2 +-
7 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/plugins/misc/projects/src/main/doc/projects.adoc b/plugins/misc/projects/src/main/doc/projects.adoc
index 062d8da..d10dd22 100644
--- a/plugins/misc/projects/src/main/doc/projects.adoc
+++ b/plugins/misc/projects/src/main/doc/projects.adoc
@@ -273,7 +273,7 @@ To execute an Apache Beam pipeline a lot of information and metadata is needed.
* By referencing the ```samples``` project Hop knows where the project is located (```config/projects/samples```)
* Since we know the location of the project, we can specify pipelines and workflows with a relative path
* The project knows where its metadata is stored (```config/projects/samples/metadata```) so it knows where to find the ```Direct``` pipeline run configuration (```config/projects/samples/metadata/pipeline-run-configuration/Direct.json```)
-* This run configuration defines its own pipeline engine specific variables, in this case the output folder : ```DATA_OUTPUT=${PROJECT_HOME}/beam/output/```
+* This run configuration defines its own pipeline engine specific variables, in this case the output folder : ```DATA_OUTPUT={openvar}PROJECT_HOME{closevar}/beam/output/```
* The output of the samples is as such written to ```config/projects/samples/beam/output```
To reference an environment you can execute using ```-e``` or ```--environment```. The only difference is that you'll have a number of extra environment variables set while executing.
diff --git a/plugins/transforms/dropbox/src/main/doc/dropboxinput.adoc b/plugins/transforms/dropbox/src/main/doc/dropboxinput.adoc
index 2d240fc..9c5e088 100644
--- a/plugins/transforms/dropbox/src/main/doc/dropboxinput.adoc
+++ b/plugins/transforms/dropbox/src/main/doc/dropboxinput.adoc
@@ -24,7 +24,7 @@ The dropbox input step reads information from a Dropbox storage account.
NOTE: create a Dropbox api key in the Dropbox https://www.dropbox.com/developers/apps/create[console].
-### Fields
+## Fields
[width="90%", options="header"]
|===
diff --git a/plugins/transforms/dropbox/src/main/doc/dropboxoutput.adoc b/plugins/transforms/dropbox/src/main/doc/dropboxoutput.adoc
index a5f5b62..847d063 100644
--- a/plugins/transforms/dropbox/src/main/doc/dropboxoutput.adoc
+++ b/plugins/transforms/dropbox/src/main/doc/dropboxoutput.adoc
@@ -24,7 +24,7 @@ The dropbox output step writes information to a Dropbox storage account.
NOTE: create a Dropbox api key in the Dropbox https://www.dropbox.com/developers/apps/create[console].
-### Fields
+## Fields
[width="90%", options="header"]
|===
diff --git a/plugins/transforms/dynamicsqlrow/src/main/doc/dynamicsqlrow.adoc b/plugins/transforms/dynamicsqlrow/src/main/doc/dynamicsqlrow.adoc
index 39efe14..9e47517 100644
--- a/plugins/transforms/dynamicsqlrow/src/main/doc/dynamicsqlrow.adoc
+++ b/plugins/transforms/dynamicsqlrow/src/main/doc/dynamicsqlrow.adoc
@@ -35,7 +35,7 @@ The Dynamic SQL row transform allows you to execute a SQL statement that is defi
|SQL field name|Specify the field that contains the SQL to execute
|Number of rows to return|Specify the number of records to return. 0 means, return all rows
|Outer Join|- false: don't return rows where nothing is found - true: at least return one source row, the rest is NULL
-|Replace variables|In case you want to use variables in the SQL, e.g. {openvart}able_name{closevar}, this option needs to be checked.
+|Replace variables|In case you want to use variables in the SQL, e.g. {openvar}table_name{closevar}, this option needs to be checked.
|Query only on parameters change|If your SQL statements do not change a lot, check this option to reduce the number of physical database queries.
|Template SQL|In Hop meta and data are separate so you have to specify the meta part in template SQL (field name and type).
I mean any statement that returns the same row structure.
diff --git a/plugins/transforms/filterrows/src/main/doc/filterrows.adoc b/plugins/transforms/filterrows/src/main/doc/filterrows.adoc
index 24cb7cb..00cbb15 100644
--- a/plugins/transforms/filterrows/src/main/doc/filterrows.adoc
+++ b/plugins/transforms/filterrows/src/main/doc/filterrows.adoc
@@ -17,6 +17,8 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:page-alternativeEditUrl: https://github.com/apache/incubator-hop/edit/master/plugins/transforms/filterrows/src/main/doc/filterrows.adoc
+:openvar: {
+:closevar: }
= Filter Rows
== Description
@@ -70,7 +72,7 @@ field1 = field3
=== Filtering special characters
-To filter special characters like explicit EOF (e.g. from old cobol files) Use a REGEX expression in the "filter row" transform with the syntax: "\x{1A}" where \x mean HEX representation and 1A into parenthesis is the EOF char to match in HEX.
+To filter special characters like explicit EOF (e.g. from old cobol files) Use a REGEX expression in the "filter row" transform with the syntax: "\x{openvar}1A{closevar}" where \x mean HEX representation and 1A into parenthesis is the EOF char to match in HEX.
== Metadata injection support
diff --git a/plugins/transforms/metainject/src/main/metainject.adoc b/plugins/transforms/metainject/src/main/doc/metainject.adoc
similarity index 99%
rename from plugins/transforms/metainject/src/main/metainject.adoc
rename to plugins/transforms/metainject/src/main/doc/metainject.adoc
index 3146a7f..9b38489 100644
--- a/plugins/transforms/metainject/src/main/metainject.adoc
+++ b/plugins/transforms/metainject/src/main/doc/metainject.adoc
@@ -73,7 +73,7 @@ To specify the source field as metadata to be injected, perform the following tr
3. Optionally, select Use constant value to specify a constant value for the injected metadata through one of the following actions:
- Manually entering a value.
- Using an internal variable to set the value ({openvar}Internal.transform.Unique.Count{closevar} for example).
- - Using a combination of manually specified values and parameter values ({openvarF}ILE_PREFIX{closevar}_{openvarF}ILE_DATE{closevar}.txt for example).
+ - Using a combination of manually specified values and parameter values ({openvar}FILE_PREFIX{closevar}_{openvar}FILE_DATE{closevar}.txt for example).
==== Injecting Metadata into the ETL Metadata Injection transform
diff --git a/plugins/transforms/sql/src/main/doc/sql.adoc b/plugins/transforms/sql/src/main/doc/sql.adoc
index 459ce2f..281fcda 100644
--- a/plugins/transforms/sql/src/main/doc/sql.adoc
+++ b/plugins/transforms/sql/src/main/doc/sql.adoc
@@ -35,7 +35,7 @@ You can execute SQL scripts with this transform, either once, during the initial
|SQL script to execute|Specify the SQL to execute. Separate statements by ; and use question marks as place holders for parameter.
|Execute for each row?|Select this option to execute the SQL for each incoming row. In this case paramters can be used. When this option is unchecked, the SQL statement is executed at the transform initialization phase.
|Execute as a single statement|This option does not split the statements by ; and will send the whole SQL to the database.
-|Variable substitution|In case you want to use variables in the SQL, e.g. {openvart}able_name{closevar}, this option needs to be checked.
+|Variable substitution|In case you want to use variables in the SQL, e.g. {openvar}table_name{closevar}, this option needs to be checked.
|Bind parameters?|Check this option to bind parameters using prepared statements, otherwise this transform will perform a literal string replacement of the parameters.
|Quote Strings?|This option adds quotes around the string according to the database dialect and also escapes special characters like CR, LF and the quote character itself.
|Parameters|The list of used parameters that will replace the question marks in the query in the given order. So the first question mark will be replaced by the first parameter, the second question mark by the second parameter etc.