You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by nf...@apache.org on 2020/09/08 09:38:33 UTC
[camel-k] branch master updated: chore(doc): document Kamelets and
fix small issues (#1690)
This is an automated email from the ASF dual-hosted git repository.
nferraro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel-k.git
The following commit(s) were added to refs/heads/master by this push:
new 929bd1b chore(doc): document Kamelets and fix small issues (#1690)
929bd1b is described below
commit 929bd1bf0d3e52c3dbd7feab50cd7a60077afb65
Author: Nicola Ferraro <ni...@gmail.com>
AuthorDate: Tue Sep 8 11:38:26 2020 +0200
chore(doc): document Kamelets and fix small issues (#1690)
---
docs/modules/ROOT/nav.adoc | 22 +-
.../modules/ROOT/pages/installation/minishift.adoc | 2 +
docs/modules/ROOT/pages/kamelets/kamelets.adoc | 508 +++++++++++++++++++++
docs/modules/ROOT/pages/tutorials/tutorials.adoc | 6 +-
4 files changed, 525 insertions(+), 13 deletions(-)
diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc
index c5afcc6..46bbab5 100644
--- a/docs/modules/ROOT/nav.adoc
+++ b/docs/modules/ROOT/nav.adoc
@@ -10,7 +10,6 @@
* xref:running/running.adoc[Running]
** xref:running/dev-mode.adoc[Dev Mode]
* xref:tutorials/tutorials.adoc[Tutorials]
-** xref:tutorials/tekton/tekton.adoc[Tekton Pipelines]
* xref:cli/cli.adoc[CLI]
** xref:cli/modeline.adoc[Modeline]
* xref:configuration/configuration.adoc[Configuration]
@@ -18,16 +17,6 @@
** xref:configuration/logging.adoc[Logging]
** xref:configuration/dependencies.adoc[Dependencies]
** xref:configuration/configmap-secret.adoc[ConfigMap/Secret]
-* xref:architecture/architecture.adoc[Architecture]
-** xref:architecture/operator.adoc[Operator]
-*** xref:architecture/cr/integration-platform.adoc[IntegrationPlatform]
-*** xref:architecture/cr/integration.adoc[Integration]
-*** xref:architecture/cr/integration-kit.adoc[IntegrationKit]
-*** xref:architecture/cr/build.adoc[Build]
-*** xref:architecture/cr/camel-catalog.adoc[CamelCatalog]
-** xref:architecture/runtime.adoc[Runtime]
-** xref:architecture/traits.adoc[Traits]
-** xref:architecture/sources.adoc[Sources]
* Observability
** xref:observability/monitoring.adoc[Monitoring]
* xref:traits:traits.adoc[Traits]
@@ -60,3 +49,14 @@
** xref:traits:service.adoc[Service]
** xref:traits:tracing.adoc[Tracing]
// End of autogenerated code - DO NOT EDIT! (trait-nav)
+* xref:kamelets/kamelets.adoc[Kamelets]
+* xref:architecture/architecture.adoc[Architecture]
+** xref:architecture/operator.adoc[Operator]
+*** xref:architecture/cr/integration-platform.adoc[IntegrationPlatform]
+*** xref:architecture/cr/integration.adoc[Integration]
+*** xref:architecture/cr/integration-kit.adoc[IntegrationKit]
+*** xref:architecture/cr/build.adoc[Build]
+*** xref:architecture/cr/camel-catalog.adoc[CamelCatalog]
+** xref:architecture/runtime.adoc[Runtime]
+** xref:architecture/traits.adoc[Traits]
+** xref:architecture/sources.adoc[Sources]
\ No newline at end of file
diff --git a/docs/modules/ROOT/pages/installation/minishift.adoc b/docs/modules/ROOT/pages/installation/minishift.adoc
index 8b1056f..6845ca5 100644
--- a/docs/modules/ROOT/pages/installation/minishift.adoc
+++ b/docs/modules/ROOT/pages/installation/minishift.adoc
@@ -1,6 +1,8 @@
[[installation-on-minishift]]
= Installing Camel K on Minishift
+WARNING: Minishift is no longer maintained and multiple users reported installation issues for Camel K, due to its old API version
+
You can run Camel K integrations on OpenShift using the Minishift cluster creation tool.
Follow the instructions in the https://github.com/minishift/minishift#getting-started[getting started guide] for the installation.
diff --git a/docs/modules/ROOT/pages/kamelets/kamelets.adoc b/docs/modules/ROOT/pages/kamelets/kamelets.adoc
new file mode 100644
index 0000000..37602c4
--- /dev/null
+++ b/docs/modules/ROOT/pages/kamelets/kamelets.adoc
@@ -0,0 +1,508 @@
+[[kamelets]]
+= Kamelets
+
+NOTE: Available since Camel K 1.2.0
+
+[[kamelets-introduction]]
+== Introduction
+
+Kamelets (**Kamel** route snipp**ets**) are a new concept introduced in Camel K that allow users to connect to external systems via a simplified interface,
+hiding all the low level details about how those connections are implemented.
+
+A Kamelet can act as *"source"* of data or alternatively as *"sink"*: a source allows to consume data from an external system,
+while a sink can let you send data to an external system or execute a particular action and get a result.
+
+For example, a "twitter-stream" Kamelet may allow a user to consume all tweets containing a specified keyword.
+Another Kamelet "twitter-post" may allow the user to publish some data as a tweet.
+Those are simple examples: experienced Camel developers know how to map those Kamelets into Camel endpoint URIs directly.
+But, in general, a Kamelet is not expected to map 1:1 a Camel component. Kamelets are **route templates**, so they can be much more complex.
+
+Being made of **pure Camel DSL**, Kamelets can embody all the logic that allows to consume or produce data from public SaaS or
+enterprise systems and only expose to the final users a clean interface that describes **the expected parameters, input and output**: just like an OpenAPI spec.
+
+For example, a "store-orders" Kamelet may be used to consume all events related to orders created in a customer enterprise system.
+The **Kamelet's interface** will define what parameters should be provided (e.g. filters on the type of order) and what's the expected datashape of the produced events.
+**Internally**, the Kamelet defines how such events will be generated: it may involve connections to multiple systems using different protocols, transformations and so on.
+But everything will be **hidden to the end user**.
+
+Kamelets are the fundamental **unit of abstraction** in the next-gen architecture of Apache Camel K.
+A system as a whole can be technically described as the set of operations that you can do with it:
+if you use the language of Kamelets to describe a specific system, then other users can have access to all those operations with ease,
+no matter how complicated is the internal logic underlying all those operations.
+Comparing it to the classical way of using Apache Camel, where developers used to write complex routes (containing both high-level and low-level details)
+to integrate systems, the Kamelet model tends to reduce the complexity by encapsulating low-level details into reusable components.
+
+Kamelets are also expected to be **rendered on visual tools** that will provide additional value to the end users.
+They are generic connectors that can be used in multiples ways, depending on the context, so each UIs can use them
+for its own purpose.
+
+[[kamelets-example]]
+== Kamelet Example
+
+Speaking technically, a Kamelet is a resource that can be installed on any Kubernetes cluster.
+The following is an example of Kamelet that we'll use to discuss the various parts:
+
+.telegram-text-source.kamelet.yaml
+[source,yaml]
+----
+apiVersion: camel.apache.org/v1alpha1
+kind: Kamelet
+metadata:
+ name: telegram-text-source # <1>
+ annotations: # <2>
+ camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PD94bW..."
+ labels: # <3>
+ camel.apache.org/kamelet.type: "source"
+spec:
+ definition: # <4>
+ title: "Telegram Text Source"
+ description: |-
+ Receive all text messages that people send to your telegram bot.
+
+ # Instructions
+ Description can include Markdown and guide the final user to configure the Kamelet parameters.
+ required:
+ - botToken
+ properties:
+ botToken:
+ title: Token
+ description: The token to access your bot on Telegram
+ type: string
+ x-descriptors:
+ - urn:alm:descriptor:com.tectonic.ui:password
+
+ types: # <5>
+ out:
+ mediaType: text/plain
+ # schema:
+ flow: # <6>
+ from:
+ uri: telegram:bots
+ parameters:
+ authorizationToken: "#property:botToken"
+ steps:
+ - convert-body-to:
+ type: "java.lang.String"
+ type-class: "java.lang.String"
+ charset: "UTF8"
+ - filter:
+ simple: "${body} != null"
+ - log: "${body}"
+ - to: "direct:#property:routeId"
+----
+<1> The Kamelet ID, to be used in integrations that want to leverage the Kamelet
+<2> Annotations such as icon provide additional display features to the Kamelet
+<3> Labels allow users to query Kamelets e.g. by kind ("source" vs. "sink")
+<4> Description of the Kamelets and parameters in JSON-schema specification format
+<5> The media type of the output (can include a schema)
+<6> The route template defining the behavior of the Kamelet
+
+At a high level (more details are provided later), a Kamelet resource describes:
+
+- A metadata section containing the ID (`metadata` -> `name`) of the Kamelet and other information, such as the type of Kamelet (`source` or `sink`)
+- A JSON-schema specification (`definition`) containing a set of parameters that you can use to configure the Kamelet
+- An optional section containing information about input and output expected by the Kamelet (`types`)
+- A Camel flow in YAML DSL containing the implementation of the Kamelet (`flow`)
+
+Once **installed on a Kubernetes namespace**, the Kamelet can be **used by any integration in that namespace**.
+
+Kamelets can be installed on a Kubernetes namespace with a simple command:
+
+[source,shell]
+----
+kubectl apply -f yourkamelet.kamelet.yaml
+----
+
+Kamelets are standard YAML files, but their common extension is `.kamelet.yaml` to help IDEs to recognize them and provide auto-completion (in the future).
+
+[[kamelets-usage-integration]]
+== Using Kamelets in Integrations
+
+Kamelets can be used in integrations **as if they were standard Camel components**. For example,
+suppose that you've created the `telegram-text-source` Kamelet in the `default` namespace on Kubernetes,
+then you can write the following integration to use the Kamelet:
+
+.example.groovy
+[source,groovy]
+----
+from('kamelet:telegram-text-source?botToken=XXXXYYYY')
+ .to('log:INFO')
+----
+
+NOTE: URI properties ("botToken") match the corresponding parameters in the Kamelet definition
+
+Kamelets can also be used multiple times in the same route definition. This happens usually with sink Kamelets.
+
+Suppose that you've defined a Kamelet named "my-company-log-sink" in your Kubernetes namespace, then you can write a route like this:
+
+.example.groovy
+[source,groovy]
+----
+from('kamelet:telegram-text-source?botToken=XXXXYYYY')
+ .to("kamelet:my-company-log-sink?bucket=general")
+ .filter().simple('${body} contains "Camel"')
+ .to("kamelet:my-company-log-sink?bucket=special")
+----
+
+The "my-company-log-sink" will obviously define what it means to write a log in the enterprise system and what is concretely a "bucket".
+
+=== Configuration
+
+When using a Kamelet, the instance parameters (e.g. "botToken", "bucket") can be passed explicitly in the URI or you can use properties. Properties can be also
+loaded implicitly by the operator from Kubernetes secrets (see below).
+
+==== 1. URI based configuration
+
+You can configure the Kamelet by passing directly the configuration parameters in the URI, as in:
+
+[source,groovy]
+----
+from("kamelet:telegram-text-source?botToken=the-token-value")
+// ...
+----
+
+In this case, "the-token-value" is passed explicitly in the URI (you can also pass a custom property placeholder as value).
+
+==== 2. Property based configuration
+
+An alternative way to configure the Kamelet is to provide configuration parameters as properties of the integration.
+
+Taking for example a different version of the integration above:
+
+[source,groovy]
+----
+from('kamelet:telegram-text-source')
+ .to("kamelet:my-company-log-sink")
+ .filter().simple('${body} contains "Camel"')
+ .to("kamelet:my-company-log-sink/mynamedconfig")
+----
+
+NOTE: The integration above does not contain URI query parameters and the last URI ("kamelet:my-company-log-sink/mynamedconfig") contains a path parameter with value "mynamedconfig"
+
+The integration above needs some configuration in order to run properly. The configuration can be provided in a property file:
+
+.example.properties
+[source,properties]
+----
+# Configuration for the Telegram source Kamelet
+camel.kamelet.telegram-text-source.botToken=the-token-value
+
+# General configuration for the Company Log Kamelet
+camel.kamelet.my-company-log-sink.bucket=general
+# camel.kamelet.my-company-log-sink.xxx=yyy
+
+# Specific configuration for the Company Log Kamelet corresponding to the named configuration "mynamedconfig"
+camel.kamelet.my-company-log-sink.mynamedconfig.bucket=special
+# When using "kamelet:my-company-log-sink/mynamedconfig", the bucket will be "special", not "general"
+----
+
+Then the integration can be run with the following command:
+
+[source,shell]
+----
+kamel run example.groovy --property-file example.properties
+----
+
+==== 3. Implicit configuration using secrets
+
+Property based configuration can also be used implicitly by creating secrets in the namespace that will be used to
+determine the Kamelets configuration.
+
+To use implicit configuration via secret, we first need to create a configuration file holding only the properties of a named configuration.
+
+.mynamedconfig.properties
+[source,properties]
+----
+# Only configuration related to the "mynamedconfig" named config
+camel.kamelet.my-company-log-sink.mynamedconfig.bucket=special
+# camel.kamelet.my-company-log-sink.mynamedconfig.xxx=yyy
+----
+
+We can create a secret from the file and label it so that it will be picked up automatically by the operator:
+
+[source,shell]
+----
+# Create the secret from the property file
+kubectl create secret generic my-company-log-sink.mynamedconfig --from-file=mynamedconfig.properties
+# Bind it to the named configuration "mynamedconfig" of the "my-company-log-sink" Kamelet
+kubectl label secret my-company-log-sink.mynamedconfig camel.apache.org/kamelet=my-company-log-sink camel.apache.org/kamelet.configuration=mynamedconfig
+----
+
+You can now write an integration that uses the Kamelet with the named configuration:
+
+.example.groovy
+[source,groovy]
+----
+from('timer:tick')
+ .setBody().constant('Hello')
+ .to('kamelet:my-company-log-sink/mynamedconfig')
+----
+
+You can run this integration without specifying other parameters, the Kamelet endpoint will be implicitly configured by the Camel K operator that will
+automatically mount the secret into the integration Pod.
+
+[[kamelets-usage-binding]]
+== Binding Kamelets
+
+In some contexts (for example **"serverless"**) users often want to leverage the power of Apache Camel to be able to connect to various sources/sinks, without
+doing additional processing (such as tranformations or other enterprise integration patterns).
+
+A common use case is that of **Knative Sources**, for which the Apache Camel developers maintain the https://knative.dev/docs/eventing/samples/apache-camel-source/[Knative CamelSources].
+Kamelets represent an **evolution** of the model proposed in CamelSources, but they allow using the same declarative style of binding, via a resource named **KameledBinding**.
+
+A KameletBinding allows to declaratively move data from a system described by a Kamelet towards a Knative destination (or other kind of destinations, in the future), or from
+a Knative channel/broker to another external system described by a Kamelet.
+
+For example, here's an example of binding:
+
+[source,yaml]
+----
+apiVersion: camel.apache.org/v1alpha1
+kind: KameletBinding
+metadata:
+ name: telegram-text-source-to-channel
+spec:
+ source: # <1>
+ ref:
+ kind: Kamelet
+ apiVersion: camel.apache.org/v1alpha1
+ name: telegram-text-source
+ properties:
+ botToken: the-token-here
+ sink: # <2>
+ ref:
+ kind: InMemoryChannel
+ apiVersion: messaging.knative.dev/v1
+ name: messages
+----
+<1> Reference to the source that provides data
+<1> Reference to the sink where data should be sent to
+
+This binding takes the `telegram-text-source` Kamelet, configures it using specific properties ("botToken") and
+makes sure that messages produced by the Kamelet are forwarded to the Knative **InMemoryChannel** named "messages".
+
+Note that source and sink are specified declaratively as standard **Kubernetes object references**.
+
+The example shows how we can reference the "telegram-text-source" resource in a KameletBinding. It's contained in the `source` section
+because it's a Kamelet of type "source".
+A Kamelet of type "sink", by contrast, can only be used in the `sink` section of a `KameletBinding`.
+
+**Under the covers, a KameletBinding creates an Integration** resource that implements the binding, but this is transparent to the end user.
+
+=== Binding to an explicit URI
+
+An alternative way to use a KameletBinding is to configure the source/sink to be an explicit Camel URI.
+For example, the following binding is allowed:
+
+[source,yaml]
+----
+apiVersion: camel.apache.org/v1alpha1
+kind: KameletBinding
+metadata:
+ name: telegram-text-source-to-channel
+spec:
+ source:
+ ref:
+ kind: Kamelet
+ apiVersion: camel.apache.org/v1alpha1
+ name: telegram-text-source
+ properties:
+ botToken: the-token-here
+ sink:
+ uri: https://mycompany.com/the-service # <1>
+----
+<1> KameletBinding with explicity URI
+
+This KameletBinding explicitly defines an URI where data is going to be pushed.
+
+NOTE: the `uri` option is also conventionally used in Knative to specify a non-kubernetes destination
+
+[[kamelets-specification]]
+== Kamelet Specification
+
+We're now going to describe the various parts of the Kamelet in more details.
+
+[[kamelets-specification-metadata]]
+=== Metadata
+
+The metadata section contains important information related to the Kamelet as Kubernetes resource.
+
+.Metadata Fields
+|===
+|name |Description |Type |Example
+
+|`name`
+|ID of the Kamelet, used to refer to the Kamelet in external routes
+|`string`
+|E.g. `telegram-text-source`
+
+|`namespace`
+|The Kubernetes namespace where the resource is installed
+|`string`
+|
+|===
+
+The following annotations and labels are also defined on the resource:
+
+.Annotations
+|===
+|name |Description |Type |Example
+
+|`camel.apache.org/kamelet.icon`
+|An optional icon for the Kamelet in URI data format
+|`string`
+|E.g. `data:image/svg+xml;base64,PD94bW...`
+|===
+
+.Labels
+|===
+|name |Description |Type |Example
+
+|label: `camel.apache.org/kamelet.type`
+|Indicates if the Kamelet can be used as source or sink
+|enum: `source`, `sink`
+|E.g. `source`
+|===
+
+[[kamelets-specification-definition]]
+=== Definition
+
+The definition part of a Kamelet contains a valid JSON-schema document describing general information about the
+Kamelet and all defined parameters.
+
+
+.Definition Fields
+|===
+|name |Description |Type |Example
+
+|`title`
+|Display name of the Kamelet
+|`string`
+|E.g. `Telegram Text Source`
+
+|`description`
+|A markdown description of the Kamelet
+|`string`
+|E.g. `Receive all text messages that people send to your telegram bot...`
+
+|`required`
+|List of required parameters (complies with JSON-schema spec)
+|array: `string`
+|
+
+|`properties`
+|Map of properties that can be configured on the Kamelet
+|map: `string` -> `schema`
+|
+
+|===
+
+Each property defined in the Kamelet has its own schema (normally a flat schema, containing only 1 level of properties).
+The following table lists some common fields allowed for each property.
+
+.Definition Parameters
+|===
+|name |Description |Type |Example
+
+|`title`
+|Display name of the property
+|`string`
+|E.g. `Token`
+
+|`description`
+|Simple text description of the property
+|`string`
+|E.g. `The token to access your bot on Telegram`
+
+|`type`
+|JSON-schema type of the property
+|`string`
+|E.g. `string`
+
+|`x-descriptors`
+|Specific aids for the visual tools
+|array: `string`
+|E.g. `- urn:alm:descriptor:com.tectonic.ui:password` displays the property as a password field in a tectonic-type form
+|===
+
+=== Data shapes
+
+Kamelets are designed to be plugged as sources or sinks in more general routes, so they can accept data as input and/or
+produce their own data. To help visual tools and applications to understand how to interact with the Kamelet, the
+specification of a Kamelet includes also information about type of data that it manages.
+
+[source,yaml]
+----
+# ...
+spec:
+ # ...
+ types:
+ out: # <1>
+ mediaType: application/json
+ schema: # <2>
+ properties:
+ # ...
+----
+<1> Defines the type of the `output`
+<2> JSON-schema definition of the output
+
+Data shape can be indicated for the following channels:
+
+- `in`: the input of the Kamelet, in case the Kamelet is of type `sink`
+- `out`: the output of the Kamelet, for both `source` and `sink` Kamelets
+- `error`: an optional error data shape, for both `source` and `sink` Kamelets
+
+Data shapes contain the following information:
+
+.Data Shape Options
+|===
+|name |Description |Type |Example
+
+|`mediaType`
+|The media type of the data
+|`string`
+|E.g. `application/json`
+
+|`schema`
+|An optional JSON-schema definition for the data
+|`object`
+|
+|===
+
+=== Flow
+
+Each Kamelet contains a YAML-based Camel DSL that provides the actual implementation of the connector.
+
+For example:
+
+
+[source,yaml]
+----
+spec:
+ # ...
+ flow:
+ from:
+ uri: telegram:bots
+ parameters:
+ authorizationToken: "#property:botToken"
+ steps:
+ - convert-body-to:
+ type: "java.lang.String"
+ type-class: "java.lang.String"
+ charset: "UTF8"
+ - filter:
+ simple: "${body} != null"
+ - log: "${body}"
+ - to: "direct:#property:routeId"
+----
+
+Source and sink flows will connect to the outside route via the `direct:#property:routeId` endpoint:
+- A source Kamelet must start **from** `direct:#property:routeId`
+- A sink Kamelet must contain a call **to** `direct:#property:routeId`
+
+Kamelets contain a **single route template** written in YAML DSL, as in the previous example.
+
+Kamelets, however, can also contain additional sources in the `spec` -> `sources` field. Those sources can be of any kind
+(not necessarily route templates) and will be added once to all the integrations where the Kamelet is used.
+They main role is to do advanced configuration of the integration context where the Kamelet is used, such as registering
+beans in the registry or adding customizers.
diff --git a/docs/modules/ROOT/pages/tutorials/tutorials.adoc b/docs/modules/ROOT/pages/tutorials/tutorials.adoc
index 3d1b4d9..8cffe64 100644
--- a/docs/modules/ROOT/pages/tutorials/tutorials.adoc
+++ b/docs/modules/ROOT/pages/tutorials/tutorials.adoc
@@ -1,6 +1,8 @@
[[tutorials]]
= Camel K Tutorials
-This section contains a collection of tutorials that dig inside specific Camel K features.
+The https://github.com/apache/camel-k-examples[apache/camel-k-examples] repository contains a curated collection of
+tutorials that drives you from the basics of Camel K to more advanced features.
-* xref:tutorials/tekton/tekton.adoc[Camel K in Tekton Pipelines]
+NOTE: The xref:tutorials/tekton/tekton.adoc[Tekton Pipelines] example is still available here and it's going to be migrated to
+the https://github.com/apache/camel-k-examples[apache/camel-k-examples] repository.