You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by GitBox <gi...@apache.org> on 2022/11/28 19:47:17 UTC
[GitHub] [streampipes-website] dominikriemer opened a new pull request, #36: docs: Add documentation for functions (apache/streampipes#751)
dominikriemer opened a new pull request, #36:
URL: https://github.com/apache/streampipes-website/pull/36
<!--
Thanks for contributing! Here are some tips you can follow to help us incorporate your contribution quickly and easily:
1. If this is your first time, please read our contributor guidelines:
- https://streampipes.apache.org/getinvolved.html
- https://cwiki.apache.org/confluence/display/STREAMPIPES/Getting+Started
2. Make sure the PR title is formatted like: `[STREAMPIPES-<Jira issue #>] PR title ...`
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., `[WIP][STREAMPIPES-<Jira issue #>] PR title ...`.
4. Please write your PR title to summarize what this PR proposes/fixes.
5. Be sure to keep the PR description updated to reflect all changes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. Make sure tests pass via `mvn clean install`.
8. (Optional) If the contribution is large, please file an Apache ICLA
- http://apache.org/licenses/icla.pdf
-->
### Purpose
<!--
Please clarify what changes you are proposing and describe how those changes will address the issue.
-->
Add basic documentation on how to use StreamPipes functions.
### Approach
<!--
Describe how you are implementing the solutions along with the design details.
-->
### Samples
<!--
Provide high-level details about the samples related to this feature.
-->
### Remarks
<!--
List related issues/PRs, link to discussions in the mailing list, todo items, or any other notes related to the PR.
-->
Fixes: <Jira issue link>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [streampipes-website] dominikriemer commented on a diff in pull request #36: docs: Add documentation for functions (apache/streampipes#751)
Posted by GitBox <gi...@apache.org>.
dominikriemer commented on code in PR #36:
URL: https://github.com/apache/streampipes-website/pull/36#discussion_r1035208889
##########
documentation/docs/06_extend-sdk-functions.md:
##########
@@ -0,0 +1,125 @@
+---
+id: extend-sdk-functions
+title: SDK Guide: Functions
+sidebar_label: SDK: Functions
+---
+
+## Introduction
+
+Pipeline elements such as data processors and data sinks are a great way
+to create _reusable_ components that can be part of pipelines.
+However, creating a pipeline element is not always the best choice:
+
+* The behaviour of a data processor is bound to a specific input stream _and_
+* A data processor doesn't contain any user-defined configuration _and_
+* The intended action is fixed or known at build time and the data processor shouldn't be available in the pipeline editor.
+
+To cover such use cases, we provide _StreamPipes Functions_. Functions
+are a great way to define custom processing logic based on previously
+connected data streams.
+
+Functions can be registered in a similar way to pipeline elements, but define expected input
+streams at startup time. Functions are started once the corresponding _extensions service_ starts
+and run until the service is stopped.
+
+## Writing a function
+
+<div class="admonition warning">
+ <div class="admonition-title">Work in Progress</div>
+ <p>Functions are currently in preview mode and are not yet recommended for production usage.
+APIs are subject to change in a future version.</p>
+</div>
+
+To define a function, create a new extensions service using the [Maven Archetypes](06_extend-archetypes.md) or use an already existing service.
+
+### Skeleton
+
+Functions can be defined by creating a new class which extends the ``StreamPipesFunction`` class.
+
+The basic skeleton looks like this:
+
+```java
+public class StreamPipesFunctionExample extends StreamPipesFunction {
+
+ @Override
+ public FunctionId getFunctionId() {
+ return FunctionId.from("my-function-id", 1);
+ }
+
+ @Override
+ public List<String> requiredStreamIds() {
+ return List.of("<id of the required stream>");
+ }
+
+ @Override
+ public void onServiceStarted(FunctionContext context) {
+ // called when the service is started
+ }
+
+ @Override
+ public void onEvent(Event event, String streamId) {
+ // called when an event arrives
+ }
+
+ @Override
+ public void onServiceStopped() {
+ // called when the service is stopped
+ }
+}
+
+```
+
+The structure of a function class is easy to understand:
+* _getFunctionId_ requires an identifier in form of a ``FunctionId``, which defines the id itself along with a version number that can be freely chosen.
+* _requiredStreamIds_ expects a list of references to data streams that are already available in StreamPipes. See below to learn how to find the id of a stream in StreamPipes.
+* _onServiceStarted_ is called once the extensions service is started and can be used to initialize the function.
+* _onEvent_ is called every time a new event arrives and provides a ``streamId`` as a reference to the corresponding stream, which is useful in case multiple data streams are received by the function.
+* _onServiceStopped_ is called when the extensions service is stopped and can be used to perform any required cleanup.
+
+### Getting a stream ID
+
+Functions require a reference to all data streams that should be retrieved by the function.
+Currently, the only way to get the ID of a function is by navigating to the ``Asset Management`` view in the StreamPipes UI.
+Create a new asset, click on ``Edit Asset`` and open ``Add Link`` in the _Linked Resources_ panel.
+Choose ``Data Source`` as link type, select one of the available sources, copy the ``Resource ID`` and provide this ID in the ``requiredStreamIds`` method.
Review Comment:
Good idea! I'll update this once the next release is on its way, there are also some older screenshots in the user manual that need to be replaced
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [streampipes-website] dominikriemer merged pull request #36: docs: Add documentation for functions (apache/streampipes#751)
Posted by GitBox <gi...@apache.org>.
dominikriemer merged PR #36:
URL: https://github.com/apache/streampipes-website/pull/36
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [streampipes-website] bossenti commented on a diff in pull request #36: docs: Add documentation for functions (apache/streampipes#751)
Posted by GitBox <gi...@apache.org>.
bossenti commented on code in PR #36:
URL: https://github.com/apache/streampipes-website/pull/36#discussion_r1034008455
##########
documentation/docs/06_extend-sdk-functions.md:
##########
@@ -0,0 +1,125 @@
+---
+id: extend-sdk-functions
+title: SDK Guide: Functions
+sidebar_label: SDK: Functions
+---
+
+## Introduction
+
+Pipeline elements such as data processors and data sinks are a great way
+to create _reusable_ components that can be part of pipelines.
+However, creating a pipeline element is not always the best choice:
+
+* The behaviour of a data processor is bound to a specific input stream _and_
+* A data processor doesn't contain any user-defined configuration _and_
+* The intended action is fixed or known at build time and the data processor shouldn't be available in the pipeline editor.
+
+To cover such use cases, we provide _StreamPipes Functions_. Functions
+are a great way to define custom processing logic based on previously
+connected data streams.
+
+Functions can be registered in a similar way to pipeline elements, but define expected input
+streams at startup time. Functions are started once the corresponding _extensions service_ starts
+and run until the service is stopped.
+
+## Writing a function
+
+<div class="admonition warning">
+ <div class="admonition-title">Work in Progress</div>
+ <p>Functions are currently in preview mode and are not yet recommended for production usage.
+APIs are subject to change in a future version.</p>
+</div>
+
+To define a function, create a new extensions service using the [Maven Archetypes](06_extend-archetypes.md) or use an already existing service.
+
+### Skeleton
+
+Functions can be defined by creating a new class which extends the ``StreamPipesFunction`` class.
+
+The basic skeleton looks like this:
+
+```java
+public class StreamPipesFunctionExample extends StreamPipesFunction {
+
+ @Override
+ public FunctionId getFunctionId() {
+ return FunctionId.from("my-function-id", 1);
+ }
+
+ @Override
+ public List<String> requiredStreamIds() {
+ return List.of("<id of the required stream>");
+ }
+
+ @Override
+ public void onServiceStarted(FunctionContext context) {
+ // called when the service is started
+ }
+
+ @Override
+ public void onEvent(Event event, String streamId) {
+ // called when an event arrives
+ }
+
+ @Override
+ public void onServiceStopped() {
+ // called when the service is stopped
+ }
+}
+
+```
+
+The structure of a function class is easy to understand:
+* _getFunctionId_ requires an identifier in form of a ``FunctionId``, which defines the id itself along with a version number that can be freely chosen.
+* _requiredStreamIds_ expects a list of references to data streams that are already available in StreamPipes. See below to learn how to find the id of a stream in StreamPipes.
+* _onServiceStarted_ is called once the extensions service is started and can be used to initialize the function.
+* _onEvent_ is called every time a new event arrives and provides a ``streamId`` as a reference to the corresponding stream, which is useful in case multiple data streams are received by the function.
+* _onServiceStopped_ is called when the extensions service is stopped and can be used to perform any required cleanup.
+
+### Getting a stream ID
+
+Functions require a reference to all data streams that should be retrieved by the function.
+Currently, the only way to get the ID of a function is by navigating to the ``Asset Management`` view in the StreamPipes UI.
+Create a new asset, click on ``Edit Asset`` and open ``Add Link`` in the _Linked Resources_ panel.
+Choose ``Data Source`` as link type, select one of the available sources, copy the ``Resource ID`` and provide this ID in the ``requiredStreamIds`` method.
Review Comment:
If you want to, you could add a GIF for easier reproducibility
##########
documentation/docs/06_extend-sdk-functions.md:
##########
@@ -0,0 +1,125 @@
+---
+id: extend-sdk-functions
+title: SDK Guide: Functions
+sidebar_label: SDK: Functions
+---
+
+## Introduction
+
+Pipeline elements such as data processors and data sinks are a great way
+to create _reusable_ components that can be part of pipelines.
+However, creating a pipeline element is not always the best choice:
+
+* The behaviour of a data processor is bound to a specific input stream _and_
+* A data processor doesn't contain any user-defined configuration _and_
+* The intended action is fixed or known at build time and the data processor shouldn't be available in the pipeline editor.
+
+To cover such use cases, we provide _StreamPipes Functions_. Functions
+are a great way to define custom processing logic based on previously
+connected data streams.
+
+Functions can be registered in a similar way to pipeline elements, but define expected input
+streams at startup time. Functions are started once the corresponding _extensions service_ starts
+and run until the service is stopped.
+
+## Writing a function
+
+<div class="admonition warning">
+ <div class="admonition-title">Work in Progress</div>
+ <p>Functions are currently in preview mode and are not yet recommended for production usage.
+APIs are subject to change in a future version.</p>
+</div>
+
+To define a function, create a new extensions service using the [Maven Archetypes](06_extend-archetypes.md) or use an already existing service.
+
+### Skeleton
+
+Functions can be defined by creating a new class which extends the ``StreamPipesFunction`` class.
+
+The basic skeleton looks like this:
+
+```java
+public class StreamPipesFunctionExample extends StreamPipesFunction {
+
+ @Override
+ public FunctionId getFunctionId() {
+ return FunctionId.from("my-function-id", 1);
+ }
+
+ @Override
+ public List<String> requiredStreamIds() {
+ return List.of("<id of the required stream>");
+ }
+
+ @Override
+ public void onServiceStarted(FunctionContext context) {
+ // called when the service is started
+ }
+
+ @Override
+ public void onEvent(Event event, String streamId) {
+ // called when an event arrives
+ }
+
+ @Override
+ public void onServiceStopped() {
+ // called when the service is stopped
+ }
+}
+
+```
+
+The structure of a function class is easy to understand:
+* _getFunctionId_ requires an identifier in form of a ``FunctionId``, which defines the id itself along with a version number that can be freely chosen.
+* _requiredStreamIds_ expects a list of references to data streams that are already available in StreamPipes. See below to learn how to find the id of a stream in StreamPipes.
+* _onServiceStarted_ is called once the extensions service is started and can be used to initialize the function.
+* _onEvent_ is called every time a new event arrives and provides a ``streamId`` as a reference to the corresponding stream, which is useful in case multiple data streams are received by the function.
+* _onServiceStopped_ is called when the extensions service is stopped and can be used to perform any required cleanup.
+
+### Getting a stream ID
+
+Functions require a reference to all data streams that should be retrieved by the function.
+Currently, the only way to get the ID of a function is by navigating to the ``Asset Management`` view in the StreamPipes UI.
Review Comment:
The other one would be via the API, but not sure whether we want to "advertise" this here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org