You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@streampipes.apache.org by ri...@apache.org on 2023/04/15 08:21:58 UTC
[streampipes-website] branch dev updated: Provice blog post for relase 0.91.0 (#72)
This is an automated email from the ASF dual-hosted git repository.
riemer pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/streampipes-website.git
The following commit(s) were added to refs/heads/dev by this push:
new a3ac1a08 Provice blog post for relase 0.91.0 (#72)
a3ac1a08 is described below
commit a3ac1a08a388b02436268350d30043cafac1290b
Author: Tim <50...@users.noreply.github.com>
AuthorDate: Sat Apr 15 10:21:52 2023 +0200
Provice blog post for relase 0.91.0 (#72)
* blog(release): provide blogpost to announce StreamPipes 0.91.0
* Fix minor typos
* Add migration info
* Re-release 0.91.0 docs version
---------
Co-authored-by: Dominik Riemer <do...@gmail.com>
---
documentation/docs/01_try-installation.md | 7 +
.../docs/05_deploy-environment-variables.md | 0
.../website/blog/2023-04-12-release-091.md | 169 +++++++++++++++++++++
.../blog/assets/2023-04-12/adapter-running.gif | Bin 0 -> 2601261 bytes
.../blog/assets/2023-04-12/how-to-get-api-key.gif | Bin 0 -> 196505 bytes
documentation/website/i18n/en.json | 3 +
.../version-0.91.0}/01_try-installation.md | 10 +-
.../version-0.91.0/01_try-overview.md | 129 ++++++++++++++++
website/partials/_download_link_to_github.ejs | 3 +-
9 files changed, 319 insertions(+), 2 deletions(-)
diff --git a/documentation/docs/01_try-installation.md b/documentation/docs/01_try-installation.md
index 2a68e037..5c838539 100644
--- a/documentation/docs/01_try-installation.md
+++ b/documentation/docs/01_try-installation.md
@@ -32,6 +32,13 @@ best experience), Firefox or Edge.
## Install StreamPipes
+<div class="admonition info">
+ <div class="admonition-title">Migration from 0.90.0 to 0.91.0</div>
+ <p>When upgrading an existing installation from 0.90.0 to 0.91.0, set the environment variable
+<b>SP_INFLUX_INIT_MODE</b> to <b>upgrade</b> instead of <b>setup</b> due to an upgrade of the underlying time-series storage. </p>
+<p>Depending on the installation type, this needs to be set in the <b>.env</b> or the <b>values.yml</b> file.</p>
+</div>
+
<ul style="padding-left:0">
<li class="installation-step" style="margin-top:20px;">
<div class="wrapper-container" style="">
diff --git a/documentation/docs/05_deploy-environment-variables.md b/documentation/docs/05_deploy-environment-variables.md
new file mode 100644
index 00000000..e69de29b
diff --git a/documentation/website/blog/2023-04-12-release-091.md b/documentation/website/blog/2023-04-12-release-091.md
new file mode 100644
index 00000000..fdbd2f29
--- /dev/null
+++ b/documentation/website/blog/2023-04-12-release-091.md
@@ -0,0 +1,169 @@
+---
+title: Apache StreamPipes release 0.91.0
+author: Tim Bossenmaier
+authorURL: https://github.com/bossenti
+authorImageURL: /docs/img/bossenmaier.png
+---
+**<div style="float: left; padding-right: 40px;">14 minutes to read</div>**
+<br>
+
+The Apache StreamPipes community is delighted to announce the release of Apache StreamPipes version 0.91.0!
+The biggest highlight of this release is the birth of our official StreamPipes Python library.
+Apart from that, we have made a lot of improvements under the hood, especially in terms of improving the development experience (e.g., by introducing and enforcing code style rules).
+Nevertheless, our new version includes more than 90 improvements and bug fixes in total.
+
+<!--truncate-->
+
+The current release can be downloaded <a href="https://streampipes.apache.org/download.html">here</a>.
+
+## StreamPipes Python
+[Apache StreamPipes](https://streampipes.apache.org/) aims to enable non-technical users to connect and analyze IoT data streams.
+To this end, it provides an easy-to-use and convenient user interface that allows one to connect to an IoT data source and create some visual
+graphs within a few minutes. <br>
+Although this is the main use case of Apache StreamPipes, it can also provide great value for people who are eager to work on data analysis or data science with IoT data, but don't want to get in touch with all the hassle associated with extracting data from devices in a suitable format.
+In this scenario, StreamPipes helps you connect to your data source and extract the data for you.
+You then can make the data available outside StreamPipes by writing it into an external source, such as a database, Kafka, etc.
+While this requires another component, you can also extract your data directly from StreamPipes programmatically using the StreamPipes API.
+For convenience, we also provide you with a StreamPipes client both available for Java and Python.
+Specifically with StreamPipes Python, we want to address the amazing data analytics and data science community in Python and benefit from the great universe of Python libraries out there.
+
+### Getting started
+Besides our official version, which you can access [here](https://streampipes.apache.org/download.html),
+we provide Apache StreamPipes as a convenient release via PyPI.
+Therefore, you can easily install it via the following command
+```bash
+pip install streampipes
+```
+All documentation related to StreamPipes Python can be found [here](https://streampipes.apache.org/docs/docs/python/latest/).
+
+But now enough words are written, let's just start with a few examples.
+As a first step, we want to establish a connection to a running StreamPipes instance.
+```python
+from streampipes.client import StreamPipesClient
+from streampipes.client.config import StreamPipesClientConfig
+from streampipes.client.credential_provider import StreamPipesApiKeyCredentials
+
+config = StreamPipesClientConfig(
+ credential_provider = StreamPipesApiKeyCredentials(
+ username = "test@streampipes.apache.org",
+ api_key = "DEMO-KEY",
+ ),
+ host_address = "localhost",
+ https_disabled = True,
+ port = 80
+)
+
+client = StreamPipesClient(client_config=config)
+```
+
+`client.describe()` gives you an overview about what can be discovered via the Python client:
+```bash
+Hi there!
+You are connected to a StreamPipes instance running at http://localhost:80.
+The following StreamPipes resources are available with this client:
+1x DataLakeMeasures
+1x DataStreams
+```
+
+The required API key can be retrieved from StreamPipes via the UI:
+
+<img src="/docs/blog/assets/2023-04-12/how-to-get-api-key.gif" alt="demonstration how to get an API key in form of a gif">
+
+### Query Data from StreamPipes data lake
+As a first step, we can see which data sets are available in the data lake:
+```python
+client.dataLakeMeasureApi.all().to_pandas()
+```
+
+To query data for a specific data lake measure, the following command takes you there:
+```python
+client.dataLakeMeasureApi.get(identifier="flow-rate").to_pandas()
+```
+There are some parameters that allow you to query data more specifically, visit the [docs](https://streampipes.apache.org/docs/docs/python/latest/reference/endpoint/api/data_lake_measure/#streampipes.endpoint.api.data_lake_measure.MeasurementGetQueryConfig) for more detailed information.
+
+If you are curious which features are available for StreamPipes Python, have a look at our [tutorial section](https://streampipes.apache.org/docs/docs/python/latest/tutorials/1-introduction-to-streampipes-python-client/).
+
+We plan to further extend the Python library in the next releases - we welcome your ideas on any features you'd like to see!
+
+## Create Adapter without starting
+
+Until now, when creating an adapter in StreamPipes, the adapter was started immediately after completion.
+From now on you can choose via the adapter overview in the last dialog whether the adapter should be started immediately or later (this is then possible in the adapter overview).
+
+<img src="/docs/blog/assets/2023-04-12/adapter-running.gif" alt="demonstration how to disable automatic start of an adapter in form of a gif">
+
+## Redirect after Login
+
+This is another small user interface improvement: when a user opens a StreamPipes link even though they are not logged in, e.g.,
+http://streampipes.demo/#/connect, StreamPipes displays the login dialog and now redirects the user directly to the target view
+(in this case the connection overview).
+
+## Support Output Streams for Standalone Functions
+Within the last release, [0.90.0](/docs/blog/2023/01/17/release-090.html), we introduced StreamPipes Functions as a lightweight alternative to processing elements.
+StreamPipes Functions do now support having output streams, which means that they allow not only to consume but to send data to StreamPipes as well.
+
+Below you can see a simple example function that demonstrates how events can be pushed back to StreamPipes again.
+```java
+public class FunctionPublishExample extends StreamPipesFunction {
+
+ private static final Logger LOG = LoggerFactory.getLogger(FunctionPublishExample.class);
+
+ private static final String FUNCTION_ID = "org.apache.streampipes.example.function.publish";
+ private static final String STREAM_APP_ID = "example-output-stream-1";
+
+ private SpOutputCollector outputCollector;
+
+ @Override
+ public List<String> requiredStreamIds() {
+ return List.of("urn:streampipes.apache.org:eventstream:EtMUkN");
+ }
+
+ @Override
+ public void onServiceStarted(FunctionContext context) {
+ LOG.info("Service started");
+ this.outputCollector = context.getOutputCollectors().get(STREAM_APP_ID);
+ }
+
+ @Override
+ public void onEvent(Event event, String streamId) {
+ LOG.info("on event");
+ var ev = new Event();
+ ev.addField("timestamp", System.currentTimeMillis());
+ ev.addField("example-property", "abc");
+ this.outputCollector.collect(ev);
+ }
+
+ @Override
+ public void onServiceStopped() {
+ LOG.info("service stopped");
+ }
+
+ @Override
+ public FunctionConfig getFunctionConfig() {
+
+ return FunctionConfigBuilder
+ .create(FunctionId.from(FUNCTION_ID, 1))
+ .withOutputStream(DataStreamBuilder.create(STREAM_APP_ID, "My Function Stream", "")
+ .property(EpProperties.timestampProperty("timestamp"))
+ .property(EpProperties.stringEp(
+ Labels.from("my-example-property", "test", "test"),
+ "example-property",
+ SO.TEXT,
+ PropertyScope.MEASUREMENT_PROPERTY))
+ .format(Formats.jsonFormat())
+ .protocol(Protocols.kafka("localhost", 9094, STREAM_APP_ID))
+ .build())
+ .build();
+ }
+}
+```
+
+This functionality is also already supported in StreamPipes Python! 🚀
+
+This means that you can write a StreamPipes function in Python that allows you to consume streams of data from StreamPipes,
+perform any data manipulation in Python, and send the data back to StreamPipes, where it can be further processed.
+
+
+<br>
+To get a comprehensive overview of all changes released within Apache StreamPipes 0.91.0
+please visit our [release notes](https://github.com/apache/incubator-streampipes/blob/release/0.91.0/RELEASE_NOTES.md#0910).
diff --git a/documentation/website/blog/assets/2023-04-12/adapter-running.gif b/documentation/website/blog/assets/2023-04-12/adapter-running.gif
new file mode 100644
index 00000000..9655e613
Binary files /dev/null and b/documentation/website/blog/assets/2023-04-12/adapter-running.gif differ
diff --git a/documentation/website/blog/assets/2023-04-12/how-to-get-api-key.gif b/documentation/website/blog/assets/2023-04-12/how-to-get-api-key.gif
new file mode 100644
index 00000000..879ec8ad
Binary files /dev/null and b/documentation/website/blog/assets/2023-04-12/how-to-get-api-key.gif differ
diff --git a/documentation/website/i18n/en.json b/documentation/website/i18n/en.json
index 520cf9c2..d5ba9198 100644
--- a/documentation/website/i18n/en.json
+++ b/documentation/website/i18n/en.json
@@ -73,6 +73,9 @@
"title": "Docker Deployment",
"sidebar_label": "Docker Deployment"
},
+ "05_deploy-environment-variables": {
+ "title": "05_deploy-environment-variables"
+ },
"deploy-kubernetes": {
"title": "Kubernetes Deployment",
"sidebar_label": "Kubernetes Deployment"
diff --git a/documentation/docs/01_try-installation.md b/documentation/website/versioned_docs/version-0.91.0/01_try-installation.md
similarity index 92%
copy from documentation/docs/01_try-installation.md
copy to documentation/website/versioned_docs/version-0.91.0/01_try-installation.md
index 2a68e037..265908d2 100644
--- a/documentation/docs/01_try-installation.md
+++ b/documentation/website/versioned_docs/version-0.91.0/01_try-installation.md
@@ -1,7 +1,8 @@
---
-id: try-installation
+id: version-0.91.0-try-installation
title: Installation
sidebar_label: Installation
+original_id: try-installation
---
The easiest way to install StreamPipes is our Docker-based installation. For production-grade deployments, we also
@@ -32,6 +33,13 @@ best experience), Firefox or Edge.
## Install StreamPipes
+<div class="admonition info">
+ <div class="admonition-title">Migration from 0.90.0 to 0.91.0</div>
+ <p>When upgrading an existing installation from 0.90.0 to 0.91.0, set the environment variable
+<b>SP_INFLUX_INIT_MODE</b> to <b>upgrade</b> instead of <b>setup</b> due to an upgrade of the underlying time-series storage. </p>
+<p>Depending on the installation type, this needs to be set in the <b>.env</b> or the <b>values.yml</b> file.</p>
+</div>
+
<ul style="padding-left:0">
<li class="installation-step" style="margin-top:20px;">
<div class="wrapper-container" style="">
diff --git a/documentation/website/versioned_docs/version-0.91.0/01_try-overview.md b/documentation/website/versioned_docs/version-0.91.0/01_try-overview.md
new file mode 100644
index 00000000..e4174ac0
--- /dev/null
+++ b/documentation/website/versioned_docs/version-0.91.0/01_try-overview.md
@@ -0,0 +1,129 @@
+---
+id: version-0.91.0-user-guide-introduction
+title: Apache StreamPipes Documentation
+sidebar_label: Overview
+original_id: user-guide-introduction
+---
+
+This is the documentation of Apache StreamPipes.
+
+<img class="docs-image docs-image-small docs-image-no-shadow" style="padding: 10px;" src="/docs/img/01_try-overview/01_streampipes-overview.png" alt="StreamPipes Overview">
+
+
+<div class="container grid col-3">
+ <div class="column">
+ <div class="toc-box">
+ <div class="toc-header border-thin">
+ 🚀 Try
+ </div>
+ <div class="toc-content">
+ <div class="toc-section-header"><b>Your first steps with Apache StreamPipes:</b></div>
+ <a href="try-installation.html">Install StreamPipes 🔗</a>
+ </div>
+ </div>
+ </div>
+ <div class="column">
+ <div class="toc-box">
+ <div class="toc-header border-thin">
+ 💡 Concepts
+ </div>
+ <div class="toc-content">
+ <div class="toc-section-header"><b>Learn about some general concepts of StreamPipes:</b></div>
+ <a href="concepts-overview.html">Overview 🔗</a>
+ </div>
+ </div>
+ </div>
+ <div class="column">
+ <div class="toc-box">
+ <div class="toc-header border-thin">
+ 🎓 Use
+ </div>
+ <div class="toc-content">
+ <div class="toc-section-header"><b>Learn how to use the various modules of StreamPipes:</b></div>
+ <a href="use-connect.html">StreamPipes Connect 🔗</a>, <a href="use-pipeline-editor.html">Pipeline Editor 🔗</a>,
+ <a href="use-managing-pipelines.html">Managing Pipelines 🔗</a>,
+ <a href="use-live-dashboard.html">Live Dashboard 🔗</a>, <a href="use-data-explorer.html">Data Explorer 🔗</a>,
+ <a href="use-notifications.html">Notifications 🔗</a>
+ </div>
+ </div>
+ </div>
+ <div class="column">
+ <div class="toc-box">
+ <div class="toc-header border-thin">
+ 📚 Pipeline Elements
+ </div>
+ <div class="toc-content">
+ <div class="toc-section-header"><b>Available pipeline elements in StreamPipes:</b></div>
+ <a href="pe/org.apache.streampipes.connect.protocol.stream.kafka.html">Adapters 🔗</a>,
+ <a href="pe/org.apache.streampipes.processors.aggregation.flink.aggregation.html">Data Processors 🔗</a>,
+ <a href="pe/org.apache.streampipes.sinks.databases.jvm.couchdb.html">Data Sinks 🔗</a>
+ </div>
+ </div>
+ </div>
+ <div class="column">
+ <div class="toc-box">
+ <div class="toc-header border-thin">
+ ⚡ Deploy
+ </div>
+ <div class="toc-content">
+ <div class="toc-section-header"><b>How to set up StreamPipes in test and production environments:</b></div>
+ <a href="deploy-docker.html">Docker 🔗</a>, <a href="deploy-kubernetes.html">Kubernetes 🔗</a>, <a href="../deploy
+ -use-ssl">Use SSL 🔗</a>
+ </div>
+ </div>
+ </div>
+ <div class="column">
+ <div class="toc-box">
+ <div class="toc-header border-thin">
+ 💻 Extend
+ </div>
+ <div class="toc-content">
+ <div class="toc-section-header"><b>Write your own pipeline elements for StreamPipes:</b></div>
+ <a href="extend-setup.html">Development Setup 🔗</a>, <a href="extend-cli.html">CLI 🔗</a>, <a href="../extend
+ -archetypes">Maven Archetypes 🔗</a>,
+ <a href="extend-tutorial-data-sources.html">Tutorial Data Sources 🔗</a>, <a href="../extend-tutorial-data
+ -processors">Tutorial Data Processors 🔗</a>, <a href="extend-tutorial-data-sinks.html">Tutorial Data Sinks 🔗
+ </a>,
+ <a href="extend-sdk-event-model.html">Event Model 🔗</a>, <a href="extend-sdk-stream-requirements.html">Stream
+ Requirements 🔗</a>, <a href="extend-sdk-static-properties.html">Static Properties 🔗</a>,
+ <a href="extend-sdk-output-strategies.html">Output Strategies 🔗</a>
+ </div>
+ </div>
+ </div>
+ <div class="column">
+ <div class="toc-box">
+ <div class="toc-header border-thin">
+ 🔧 Technicals
+ </div>
+ <div class="toc-content">
+ <div class="toc-section-header"><b>Learn about technical concepts behind the curtain:</b></div>
+ <a href="technicals-architecture.html">Architecture 🔗</a>, <a href="technicals-user-guidance.html">User
+ Guidance 🔗
+ </a>, <a href="technicals-runtime-wrappers.html">Runtime Wrappers 🔗</a>,
+ <a href="technicals-messaging.html">Messaging 🔗</a>, <a href="technicals-configuration.html">Configuration 🔗</a>
+ </div>
+ </div>
+ </div>
+ <div class="column">
+ <div class="toc-box">
+ <div class="toc-header border-thin">
+ 👪 Community
+ </div>
+ <div class="toc-content">
+ <div class="toc-section-header"><b>Get support and learn how to contribute to StreamPipes:</b></div>
+ <a href="community-get-help.html">Get Help 🔗</a>, <a href="community-contribute.html">Contribute 🔗</a>
+ </div>
+ </div>
+ </div>
+ <div class="column">
+ <div class="toc-box">
+ <div class="toc-header border-thin">
+ 🐍 StreamPipes Python
+ </div>
+ <div class="toc-content">
+ <div class="toc-section-header"><b>Discover what we offer for the Python world:</b></div>
+ <a href="/docs/docs/python/latest">Python Documentation 🔗</a>
+ </div>
+ </div>
+ </div>
+</div>
diff --git a/website/partials/_download_link_to_github.ejs b/website/partials/_download_link_to_github.ejs
index 3688e5f2..bcdfa522 100644
--- a/website/partials/_download_link_to_github.ejs
+++ b/website/partials/_download_link_to_github.ejs
@@ -7,7 +7,8 @@
</span>
</div>
<div class="col-md-11 col-9">
- For a detailed description of the installer go to the <a href="https://github.com/apache/streampipes/tree/dev/installer">GitHub repository</a>
+ For a detailed description of the installer and migration guides, read the <a href="https://streampipes.apache.org/docs/docs/try-installation.html">installation guide</a>
+ or learn about advanced <a href="https://streampipes.apache.org/docs/docs/deploy-docker.html">Docker</a> or <a href="https://streampipes.apache.org/docs/docs/deploy-kubernetes.html">K8s</a> deployment setups.
</div>
</div>
</li>