You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@streampipes.apache.org by ri...@apache.org on 2022/03/06 22:20:41 UTC
[incubator-streampipes-website] 03/03: update tutorial
This is an automated email from the ASF dual-hosted git repository.
riemer pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/incubator-streampipes-website.git
commit e5a7205cc7bb6b1c8e7a7223e3c307b57f6d1525
Author: Dominik Riemer <do...@gmail.com>
AuthorDate: Sun Mar 6 23:20:28 2022 +0100
update tutorial
---
documentation/docs/06_extend-sdk-event-model.md | 4 +-
.../docs/06_extend-sdk-stream-requirements.md | 2 +-
.../docs/06_extend-tutorial-data-processors.md | 132 ++++++++++-----------
3 files changed, 69 insertions(+), 69 deletions(-)
diff --git a/documentation/docs/06_extend-sdk-event-model.md b/documentation/docs/06_extend-sdk-event-model.md
index 8d85666..8e740f5 100644
--- a/documentation/docs/06_extend-sdk-event-model.md
+++ b/documentation/docs/06_extend-sdk-event-model.md
@@ -6,9 +6,9 @@ sidebar_label: SDK: Event Model
## Introduction
-The 0.61.0 release of StreamPipes introduces a new event model that replaces the ``Map`` representation of events at runtime. This guide explains the usage of the new event model to manipulate runtime events for data processors and data sink.
+This guide explains the usage of the event model to manipulate runtime events for data processors and data sink.
-## Reference
+## Prerequisites
This guide assumes that you are already familiar with the basic setup of [data processors](dev-guide-processor-sdk.md) and [data sinks](dev-guide-sink-sdk.md).
diff --git a/documentation/docs/06_extend-sdk-stream-requirements.md b/documentation/docs/06_extend-sdk-stream-requirements.md
index 87061b6..6a9d5c1 100644
--- a/documentation/docs/06_extend-sdk-stream-requirements.md
+++ b/documentation/docs/06_extend-sdk-stream-requirements.md
@@ -19,7 +19,7 @@ This guide covers the creation of stream requirements. Before reading this secti
## The StreamRequirementsBuilder
-Stream requirements can be defined in the ``Controller`` class of the pipeline element. Start with a method body like this:
+Stream requirements can be defined in the ``declareModel`` method of the pipeline element class. Start with a method body like this:
```java
diff --git a/documentation/docs/06_extend-tutorial-data-processors.md b/documentation/docs/06_extend-tutorial-data-processors.md
index c8c8894..543d046 100644
--- a/documentation/docs/06_extend-tutorial-data-processors.md
+++ b/documentation/docs/06_extend-tutorial-data-processors.md
@@ -4,10 +4,9 @@ title: Tutorial: Data Processors
sidebar_label: Tutorial: Data Processors
---
-In this tutorial, we will add a new data processor using the Apache Flink wrapper.
+In this tutorial, we will add a new data processor using the standalone wrapper.
-From an architectural point of view, we will create a self-contained service that includes the description of the data processor and a Flink-compatible implementation.
-Once a pipeline is started that uses this data processor, the implementation is submitted to an Apache Flink cluster.
+From an architectural point of view, we will create a self-contained service that includes the description of the data processor and a an implementation.
## Objective
@@ -21,25 +20,23 @@ The algorithm outputs every location event once the position has entered the geo
<div class="admonition-title">Note</div>
<p>The implementation in this tutorial is pretty simple - our processor will fire an event every time the GPS location is inside the geofence.
In a real-world application, you would probably want to define a pattern that recognizes the _first_ event a vehicle enters the geofence.<br/>
- This can be easily done using a CEP library, e.g., Apache Flink CEP.</p>
+ This can be easily done using a CEP library.</p>
</div>
## Project setup
-To create new projects from scratch, several Maven archetypes exist to start developing.
-Enter the following command to create a new project based on the StreamPipes ``Processors-Flink`` archetype:
+Instead of creating a new project from scratch, we recommend to use the Maven archetype to create a new project skeleton (streampipes-archetype-extensions-jvm).
+Enter the following command in a command line of your choice (Apache Maven needs to be installed):
```
-mvn archetype:generate -DarchetypeGroupId=org.apache.streampipes \
--DarchetypeArtifactId=streampipes-archetype-pe-processors-flink -DarchetypeVersion=0.68.0 \
--DgroupId=org.streampipes.tutorial -DartifactId=geofence-tutorial -DclassNamePrefix=Geofencing -DpackageName=geofencing
+mvn archetype:generate \
+-DarchetypeGroupId=org.apache.streampipes -DarchetypeArtifactId=streampipes-archetype-extensions-jvm \
+-DarchetypeVersion=0.69.0 -DgroupId=my.groupId \
+-DartifactId=my-example -DclassNamePrefix=MyExample -DpackageName=mypackagename
```
-Once you've imported the generated project, the project structure should look as follows:
-
-<img src="/docs/img/tutorial-processors/project-structure-processor.PNG" alt="Project Structure Data Processor">
-
+You will see a project structure similar to the structure shown in the [archetypes](06_extend-archetypes.md) section.
<div class="admonition tip">
<div class="admonition-title">Tip</div>
@@ -51,65 +48,68 @@ Now you're ready to create your first data processor for StreamPipes!
## Adding data processor requirements
First, we will add a new stream requirement.
-Open the class `GeofencingController` which should look as follows:
+Create a new class `GeofencingProcessor` which should look as follows:
```java
-package org.streampipes.tutorial.pe.processor.geofencing;
-
-import org.streampipes.tutorial.config.Config;
-
-import org.streampipes.model.DataProcessorType;
-import org.streampipes.model.graph.DataProcessorDescription;
-import org.streampipes.model.graph.DataProcessorInvocation;
-import org.streampipes.sdk.builder.ProcessingElementBuilder;
-import org.streampipes.sdk.builder.StreamRequirementsBuilder;
-import org.streampipes.sdk.extractor.ProcessingElementParameterExtractor;
-import org.streampipes.sdk.helpers.EpRequirements;
-import org.streampipes.sdk.helpers.Labels;
-import org.streampipes.sdk.helpers.OutputStrategies;
-import org.streampipes.sdk.helpers.SupportedFormats;
-import org.streampipes.sdk.helpers.SupportedProtocols;
-import org.streampipes.wrapper.flink.FlinkDataProcessorDeclarer;
-import org.streampipes.wrapper.flink.FlinkDataProcessorRuntime;
-
-public class GeofencingController extends
- FlinkDataProcessorDeclarer<GeofencingParameters> {
-
- private static final String EXAMPLE_KEY = "example-key";
-
- @Override
- public DataProcessorDescription declareModel() {
- return ProcessingElementBuilder.create("org.streampipes.tutorial-geofencing")
- .category(DataProcessorType.ENRICH)
- .withAssets(Assets.DOCUMENTATION, Assets.ICON)
- .withLocales(Locales.EN)
- .requiredStream(StreamRequirementsBuilder
- .create()
- .requiredProperty(EpRequirements.anyProperty())
- .build())
- .outputStrategy(OutputStrategies.keep())
- .requiredTextParameter(Labels.from(EXAMPLE_KEY, "Example Text Parameter", "Example " +
- "Text Parameter Description"))
- .build();
- }
-
- @Override
- public FlinkDataProcessorRuntime<GeofencingParameters> getRuntime(DataProcessorInvocation
- graph, ProcessingElementParameterExtractor extractor) {
-
- String exampleString = extractor.singleValueParameter(EXAMPLE_KEY, String.class);
-
- GeofencingParameters params = new GeofencingParameters(graph, exampleString);
-
- return new GeofencingProgram(params, Config.INSTANCE.getDebug());
- }
-
+package org.apache.streampipes.pe.example;
+
+import org.apache.streampipes.commons.exceptions.SpRuntimeException;
+import org.apache.streampipes.model.DataProcessorType;
+import org.apache.streampipes.model.graph.DataProcessorDescription;
+import org.apache.streampipes.model.runtime.Event;
+import org.apache.streampipes.sdk.builder.ProcessingElementBuilder;
+import org.apache.streampipes.sdk.builder.StreamRequirementsBuilder;
+import org.apache.streampipes.sdk.helpers.EpRequirements;
+import org.apache.streampipes.sdk.helpers.Labels;
+import org.apache.streampipes.sdk.helpers.Locales;
+import org.apache.streampipes.sdk.helpers.OutputStrategies;
+import org.apache.streampipes.sdk.utils.Assets;
+import org.apache.streampipes.wrapper.context.EventProcessorRuntimeContext;
+import org.apache.streampipes.wrapper.routing.SpOutputCollector;
+import org.apache.streampipes.wrapper.standalone.ProcessorParams;
+import org.apache.streampipes.wrapper.standalone.StreamPipesDataProcessor;
+
+public class GeofencingProcessor extends StreamPipesDataProcessor {
+
+ private static final String EXAMPLE_KEY = "example-key";
+
+ @Override
+ public DataProcessorDescription declareModel() {
+ return ProcessingElementBuilder.create("org.streampipes.tutorial-geofencing")
+ .category(DataProcessorType.ENRICH)
+ .withAssets(Assets.DOCUMENTATION, Assets.ICON)
+ .withLocales(Locales.EN)
+ .requiredStream(StreamRequirementsBuilder
+ .create()
+ .requiredProperty(EpRequirements.anyProperty())
+ .build())
+ .outputStrategy(OutputStrategies.keep())
+ .requiredTextParameter(Labels.from(EXAMPLE_KEY, "Example Text Parameter", "Example " +
+ "Text Parameter Description"))
+ .build();
+ }
+
+ @Override
+ public void onInvocation(ProcessorParams parameters, SpOutputCollector spOutputCollector, EventProcessorRuntimeContext runtimeContext) throws SpRuntimeException {
+
+ }
+
+ @Override
+ public void onEvent(Event event, SpOutputCollector collector) throws SpRuntimeException {
+
+ }
+
+ @Override
+ public void onDetach() throws SpRuntimeException {
+
+ }
}
+
```
-In this class, we need to implement two methods: The `declareModel` method is used to define abstract stream requirements such as event properties that must be present in any input stream that is later connected to the element using the StreamPipes UI.
-The second method, `getRuntime` is used to create and deploy the parameterized Flink program once a pipeline using this element is started.
+In this class, we need to implement three methods: The `declareModel` method is used to define abstract stream requirements such as event properties that must be present in any input stream that is later connected to the element using the StreamPipes UI.
+The second method, `onInvocation` is triggered once a pipeline is started. Finally, the `onEvent` method
Similar to data sources, the SDK provides a builder class to generate the description for data processors.
Delete the content within the ``declareModel`` method and add the following lines to the `declareModel` method: