You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hop.apache.org by gi...@apache.org on 2021/03/25 18:16:05 UTC
[incubator-hop-docs] branch asf-site updated: Documentation updated
to 03a9839
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-hop-docs.git
The following commit(s) were added to refs/heads/asf-site by this push:
new f5c454f Documentation updated to 03a9839
f5c454f is described below
commit f5c454f12c49d58f416ec18bfca8a9c602f2980f
Author: jenkins <bu...@apache.org>
AuthorDate: Thu Mar 25 18:16:00 2021 +0000
Documentation updated to 03a9839
---
.../cassandra/cassandra-connection.adoc | 64 ++++++++++++++++++
.../pages/pipeline/transforms/cassandra-input.adoc | 45 +++++++++++++
.../pipeline/transforms/cassandra-output.adoc | 75 ++++++++++++++++++++++
.../pages/pipeline/transforms/mongodbinput.adoc | 6 +-
...esultfilenames.adoc => cassandra-exec-cql.adoc} | 30 +++++----
.../workflow/actions/deleteresultfilenames.adoc | 6 +-
6 files changed, 208 insertions(+), 18 deletions(-)
diff --git a/hop-user-manual/modules/ROOT/pages/metadata-types/cassandra/cassandra-connection.adoc b/hop-user-manual/modules/ROOT/pages/metadata-types/cassandra/cassandra-connection.adoc
new file mode 100644
index 0000000..edcf895
--- /dev/null
+++ b/hop-user-manual/modules/ROOT/pages/metadata-types/cassandra/cassandra-connection.adoc
@@ -0,0 +1,64 @@
+////
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+////
+:documentationPath: /metadata-types/cassandra/
+:language: en_US
+
+= Cassandra Connection
+
+== Description
+
+Connections can be created and edited using the logo (click on it to edit), and the drop down menu (arrow down) which you can find on the right of every connection line on the various Cassandra action or transform dialogs.
+
+They can also be managed with the metadata perspective in the Hop GUI. Finally, please note that Cassandra Connections are serialized as JSON in the Hop metadata folder under
+
+`metadata/cassandra-connection/`
+
+== The Options
+
+|===
+|Option |Description
+
+|Hostname
+|Specify the host name(s) for the connection to the Cassandra server
+
+|Port
+|Specify the port number for the connection to the Cassandra server
+
+|Username
+|Specify the username of the target keyspace and/or table authentication details
+
+|Password
+|Specify the password of the target keyspace and/or table authentication details
+
+|Socket Timeout
+|Set an optional connection timeout period, specified in milliseconds.
+
+|Keyspace
+|Specify the keyspace (database) name. You can use the `Select Keyspace` button to select a keyspace. You can use the `Execute CQL` button to create one. For example:
+`CREATE KEYSPACE IF NOT EXISTS hop
+WITH replication = {'class':'SimpleStrategy', 'replication_factor' : 3} ;`
+
+|Schema hostname
+|For writes only: Specify the host name for the connection to the Cassandra schema (leave blank if not different from hostname)
+
+|Schema port
+|For writes only: Specify the port for the connection to the Cassandra schema (leave blank if not different from port)
+
+|Use compression
+|Select if you want the text of each BATCH INSERT statement compressed (with GZIP) before transmitting it to the node.
+
+|===
diff --git a/hop-user-manual/modules/ROOT/pages/pipeline/transforms/cassandra-input.adoc b/hop-user-manual/modules/ROOT/pages/pipeline/transforms/cassandra-input.adoc
new file mode 100644
index 0000000..65e8012
--- /dev/null
+++ b/hop-user-manual/modules/ROOT/pages/pipeline/transforms/cassandra-input.adoc
@@ -0,0 +1,45 @@
+////
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+////
+:documentationPath: /pipeline/transforms/
+:language: en_US
+:page-alternativeEditUrl: https://github.com/apache/incubator-hop/edit/master/plugins/tech/cassandra/src/main/doc/cassandra-input.adoc
+:openvar: ${
+:closevar: }
+
+= Cassandra Input
+
+The Cassandra Input transform reads data from a Cassandra table of an Apache Cassandra cluster using CQL (Cassandra Query Language) version 3.x.
+
+== Options
+
+|===
+|Option |Description
+
+|Cassandra connection
+|Here you specify the name of the connection to use either as a fixed value or as a variable expression. There are buttons to the right of the input field to hel you manage the metadata.
+
+|Transport max length
+|Set an optional maximum object size that can be sent. Leaving it blank means that the default cluster value is used. (typically 16384000)
+
+|Execute query for each incoming row?
+|This will cause the query to be executed repeatedly: once per input row.
+
+|CQL
+|The Cassandra Query Language statement you want to use. For more information on CQL, see https://docs.datastax.com/en/cql-oss/3.x/cql/cqlIntro.html[Introduction to Cassandra Query Language]
+
+|===
+
diff --git a/hop-user-manual/modules/ROOT/pages/pipeline/transforms/cassandra-output.adoc b/hop-user-manual/modules/ROOT/pages/pipeline/transforms/cassandra-output.adoc
new file mode 100644
index 0000000..493b88c
--- /dev/null
+++ b/hop-user-manual/modules/ROOT/pages/pipeline/transforms/cassandra-output.adoc
@@ -0,0 +1,75 @@
+////
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+////
+:documentationPath: /pipeline/transforms/
+:language: en_US
+:page-alternativeEditUrl: https://github.com/apache/incubator-hop/edit/master/plugins/tech/cassandra/src/main/doc/cassandra-output.adoc
+:openvar: ${
+:closevar: }
+
+= Cassandra Output
+
+The Cassandra Input transform writes data to a Cassandra table of an Apache Cassandra cluster using CQL (Cassandra Query Language) version 3.x.
+
+== Options
+
+|===
+|Option |Description
+
+|Cassandra connection
+|Here you specify the name of the connection to use either as a fixed value or as a variable expression. There are buttons to the right of the input field to hel you manage the metadata.
+
+|Table to write to
+|Specify the name of the table to write to.
+
+|Consistency level
+|Please see https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/dml/dmlConfigConsistency.html#Writeconsistencylevels[Cassandra Write consistency levels] for a list of possible values.
+
+|Commit batch size
+|The number of rows to update at once in each commit
+
+|Batch insert timeout
+|Specify the number of milliseconds to wait for a batch to completely insert before splitting into smaller sub-batches. You must specify a value lower than Socket timeout or leave empty for no timeout.
+
+|Sub batch size
+|Specify the sub-batch size (in number of rows) if the batch must be split because Batch insert timeout is reached.
+
+|Insert unlogged batches
+|Select if you want to use non-atomic batch writing. By default, batches are atomic (if any of the batch succeeds, all of it will succeed). Select this option to remove the atomic restriction.
+
+|Time to live (TTL)
+|Specify the amount of time in which to write a column. If the time expires, that column is deleted.
+
+|Incoming field to use as the key
+|Specify which incoming field to use as the key. You can use to specify the key from the names of incoming transform fields.
+
+|Create table
+|Select to create a named table (column family) if one does not already exist.
+
+|Table creation WITH clause
+|Specify additions to the table creation `WITH` clause.
+
+|Truncate table
+|Select if you want any existing data to be deleted from the named table before inserting incoming rows.
+
+|Update table metadata
+|Select if you want to update the table metadata with information on incoming fields not already present. If this option is not selected, any unknown incoming fields are ignored unless the Insert fields not in column meta data option is selected.
+
+|Insert fields not in column meta data
+|Select if want to insert the table metadata in any incoming fields not present, with respect to the default table validator. This option has no effect if Update table meta data is selected.
+
+|===
+
diff --git a/hop-user-manual/modules/ROOT/pages/pipeline/transforms/mongodbinput.adoc b/hop-user-manual/modules/ROOT/pages/pipeline/transforms/mongodbinput.adoc
index 0b43145..47b0daf 100644
--- a/hop-user-manual/modules/ROOT/pages/pipeline/transforms/mongodbinput.adoc
+++ b/hop-user-manual/modules/ROOT/pages/pipeline/transforms/mongodbinput.adoc
@@ -204,7 +204,7 @@ To display all array values, use the asterisk as the key, like this $.email[*].
|Indicates the data type.
|Indexed values
-|Specify a comma-separated list of legal values for String fields. When you specify values in this field, the Kettle indexed data type is applied to the data. If no values are specified, the String data type is applied. Usually, you will only need to modify this field if you are using Weka metadata for nominal fields.
+|Specify a comma-separated list of legal values for String fields. When you specify values in this field, the Hop indexed data type is applied to the data. If no values are specified, the String data type is applied. Usually, you will only need to modify this field if you are using Weka metadata for nominal fields.
|Sample: array min: max index
|Indicates minimum and maximum values for the index in the sampled documents.
@@ -213,7 +213,7 @@ To display all array values, use the asterisk as the key, like this $.email[*].
|Indicates how often the field occurs and the number of documents processed.
|Sample: disparate types
-|Indicates if different data types populate the same field in the sampled documents. When several documents are sampled and the same field contain different data types, the Sample: disparate types field is populated with a Y and the Type field displays the String data type. The Kettle type for the field is set to the String data type, for different output value types.
+|Indicates if different data types populate the same field in the sampled documents. When several documents are sampled and the same field contain different data types, the Sample: disparate types field is populated with a Y and the Type field displays the String data type. The Hop type for the field is set to the String data type, for different output value types.
|===
@@ -224,7 +224,7 @@ The following sections contain examples of query expressions and aggregate pipel
=== Query expression
MongoDB allows you to select and filter documents in a collection using specific fields and values.
-The http://docs.mongodb.org/manual/reference/mongodb-extended-json/[MongoDB Extended JSON] documentation details how to use queries. Pentaho supports only the features discussed on this page.
+The http://docs.mongodb.org/manual/reference/mongodb-extended-json/[MongoDB Extended JSON] documentation details how to use queries. Apache Hop supports only the features discussed on this page.
The following table displays some examples of the syntax and structure of the queries you can use to request data from MongoDB:
diff --git a/hop-user-manual/modules/ROOT/pages/workflow/actions/deleteresultfilenames.adoc b/hop-user-manual/modules/ROOT/pages/workflow/actions/cassandra-exec-cql.adoc
similarity index 52%
copy from hop-user-manual/modules/ROOT/pages/workflow/actions/deleteresultfilenames.adoc
copy to hop-user-manual/modules/ROOT/pages/workflow/actions/cassandra-exec-cql.adoc
index 748ab56..ece9e7e 100644
--- a/hop-user-manual/modules/ROOT/pages/workflow/actions/deleteresultfilenames.adoc
+++ b/hop-user-manual/modules/ROOT/pages/workflow/actions/cassandra-exec-cql.adoc
@@ -16,20 +16,26 @@ under the License.
////
:documentationPath: /workflow/actions/
:language: en_US
-:page-alternativeEditUrl: https://github.com/apache/incubator-hop/edit/master/workflow/actions/deleteresultfilenames/src/main/doc/deleteresultfilenames.adoc
-= Delete filenames from result
+:page-alternativeEditUrl: https://github.com/apache/incubator-hop/edit/master/plugins/tech/cassandra/src/main/doc/cassandra-exec-cql.adoc
+:openvar: ${
+:closevar: }
-== Description
+= Cassandra Exec CQL
-Use this action to delete all the filenames that are in the result files list of the last workflow action.
+This action allows you to execute one or more CassandraQL statements on a Cassandra cluster.
+You can use it to create or drop keyspaces, tables, indexes and so on.
+If all statements executed successfully the result will be true (success).
== Options
-[width="90%", options="header"]
-|===
-|Option|Description
-|Workflow action name|The name of the workflow action. *Note*: This name has to be unique in a single workflow. A workflow action can be placed several times on the canvas, however it will be the same workflow action.
-|Limit action to| Enable this feature if you want to limit the deletion to certain filenames in the result file list.
-|Wildcard|The regular expression to limit the files to delete
-|Exclude wildcard|The regular expression to exclude certain files from being deleted.
-|===
\ No newline at end of file
+* Cassandra Connection: You can specify the name of the connection to use either as a fixed value or as a variable expression. There are buttons to the right of the input field to hel you manage the metadata.
+* CQL Script: You can enter one or more CQL statements ending with a semi-colon on a new line.
+
+== Example:
+
+[source]
+----
+CREATE KEYSPACE IF NOT EXISTS hop
+WITH replication = {'class':'SimpleStrategy', 'replication_factor' : 3}
+;
+----
diff --git a/hop-user-manual/modules/ROOT/pages/workflow/actions/deleteresultfilenames.adoc b/hop-user-manual/modules/ROOT/pages/workflow/actions/deleteresultfilenames.adoc
index 748ab56..1998da3 100644
--- a/hop-user-manual/modules/ROOT/pages/workflow/actions/deleteresultfilenames.adoc
+++ b/hop-user-manual/modules/ROOT/pages/workflow/actions/deleteresultfilenames.adoc
@@ -28,8 +28,8 @@ Use this action to delete all the filenames that are in the result files list of
[width="90%", options="header"]
|===
|Option|Description
-|Workflow action name|The name of the workflow action. *Note*: This name has to be unique in a single workflow. A workflow action can be placed several times on the canvas, however it will be the same workflow action.
-|Limit action to| Enable this feature if you want to limit the deletion to certain filenames in the result file list.
+|Workflow action name|The name of the workflow action.
+|Limit action to|Enable this feature if you want to limit the deletion to certain filenames in the result file list.
|Wildcard|The regular expression to limit the files to delete
-|Exclude wildcard|The regular expression to exclude certain files from being deleted.
+|Exclude wildcard|The regular expression to exclude certain files from being deleted.
|===
\ No newline at end of file