You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by Patrick Wiener <wi...@apache.org> on 2020/02/24 17:37:32 UTC
Multi-arch docker images for StreamPipes
Hi,
as a result of a PoC described in a blog post (https://streampipes.apache.org/docs/blog/2020/01/23/streampipes_on_rpi/ <https://streampipes.apache.org/docs/blog/2020/01/23/streampipes_on_rpi/>) I was working on bringing this multi-arch feature officially
into the StreamPipes community (https://issues.apache.org/jira/browse/STREAMPIPES-66 <https://issues.apache.org/jira/browse/STREAMPIPES-66>). I finished the extension of StreamPipes related Docker images to support ARM-based CPU
architectures - currently arm32v7 (e.g. Raspberry Pi) as well as arm64v8 (e.g. Jetson Nano).
As of now, this includes all core StreamPipes services + mandatory external services for running StreamPipes lite:
ui
backend
connect (master + worker)
all pipeline elements
activemq
consul
couchdb
kafka
zookeeper
influxdb
We currently still rely on an external kafka-rest service for live preview of data source in the pipeline editor. This external component is not provided as an multi-arch image since
we plan to implement this feature by ourselves to minimize dependencies. So live preview is currently not supported in ARM-based setup.
I updated the cli, installer and helm deployment within https://github.com/apache/incubator-streampipes-installer/tree/dev <https://github.com/apache/incubator-streampipes-installer/tree/dev> accordingly and tested the different alternatives.
amd64/x86: tested on server and local Macbook Pro
arm32v7: tested on Raspberry Pi Model 4, 4Gb RAM
arm64v8: tested on Jetson Nano, 4Gb RAM
Cheers,
Patrick
Re: Multi-arch docker images for StreamPipes
Posted by Patrick Wiener <wi...@apache.org>.
Good point. Definitely worth noting in the docs. In addition, I plan to do another blog post
as an extension to running StreamPipes on Raspberry Pi, where I explain how to use our helm
chart in order to deploy SP over a set of heterogeneous (edge) nodes :P
Patrick
> Am 24.02.2020 um 23:57 schrieb Dominik Riemer <ri...@apache.org>:
>
> Hi Patrick,
>
> wow, this is very cool!
> After our next (or first ASF) release, I think it would be great to also
> mention this in the documentation, e.g., by adding a short notice in the
> installation manual that explains how to run StreamPipes on a Pi or Nano.
>
> Dominik
>
> -----Original Message-----
> From: Patrick Wiener <wi...@apache.org>
> Sent: Monday, February 24, 2020 6:38 PM
> To: dev@streampipes.apache.org
> Subject: Multi-arch docker images for StreamPipes
>
> Hi,
>
> as a result of a PoC described in a blog post
> (https://streampipes.apache.org/docs/blog/2020/01/23/streampipes_on_rpi/
> <https://streampipes.apache.org/docs/blog/2020/01/23/streampipes_on_rpi/>) I
> was working on bringing this multi-arch feature officially into the
> StreamPipes community (https://issues.apache.org/jira/browse/STREAMPIPES-66
> <https://issues.apache.org/jira/browse/STREAMPIPES-66>). I finished the
> extension of StreamPipes related Docker images to support ARM-based CPU
> architectures - currently arm32v7 (e.g. Raspberry Pi) as well as arm64v8
> (e.g. Jetson Nano).
>
> As of now, this includes all core StreamPipes services + mandatory external
> services for running StreamPipes lite:
>
> ui
> backend
> connect (master + worker)
> all pipeline elements
> activemq
> consul
> couchdb
> kafka
> zookeeper
> influxdb
>
> We currently still rely on an external kafka-rest service for live preview
> of data source in the pipeline editor. This external component is not
> provided as an multi-arch image since we plan to implement this feature by
> ourselves to minimize dependencies. So live preview is currently not
> supported in ARM-based setup.
>
> I updated the cli, installer and helm deployment within
> https://github.com/apache/incubator-streampipes-installer/tree/dev
> <https://github.com/apache/incubator-streampipes-installer/tree/dev>
> accordingly and tested the different alternatives.
>
> amd64/x86: tested on server and local Macbook Pro
> arm32v7: tested on Raspberry Pi Model 4, 4Gb RAM
> arm64v8: tested on Jetson Nano, 4Gb RAM
>
>
>
> Cheers,
> Patrick
>
RE: Multi-arch docker images for StreamPipes
Posted by Dominik Riemer <ri...@apache.org>.
Hi Patrick,
wow, this is very cool!
After our next (or first ASF) release, I think it would be great to also
mention this in the documentation, e.g., by adding a short notice in the
installation manual that explains how to run StreamPipes on a Pi or Nano.
Dominik
-----Original Message-----
From: Patrick Wiener <wi...@apache.org>
Sent: Monday, February 24, 2020 6:38 PM
To: dev@streampipes.apache.org
Subject: Multi-arch docker images for StreamPipes
Hi,
as a result of a PoC described in a blog post
(https://streampipes.apache.org/docs/blog/2020/01/23/streampipes_on_rpi/
<https://streampipes.apache.org/docs/blog/2020/01/23/streampipes_on_rpi/>) I
was working on bringing this multi-arch feature officially into the
StreamPipes community (https://issues.apache.org/jira/browse/STREAMPIPES-66
<https://issues.apache.org/jira/browse/STREAMPIPES-66>). I finished the
extension of StreamPipes related Docker images to support ARM-based CPU
architectures - currently arm32v7 (e.g. Raspberry Pi) as well as arm64v8
(e.g. Jetson Nano).
As of now, this includes all core StreamPipes services + mandatory external
services for running StreamPipes lite:
ui
backend
connect (master + worker)
all pipeline elements
activemq
consul
couchdb
kafka
zookeeper
influxdb
We currently still rely on an external kafka-rest service for live preview
of data source in the pipeline editor. This external component is not
provided as an multi-arch image since we plan to implement this feature by
ourselves to minimize dependencies. So live preview is currently not
supported in ARM-based setup.
I updated the cli, installer and helm deployment within
https://github.com/apache/incubator-streampipes-installer/tree/dev
<https://github.com/apache/incubator-streampipes-installer/tree/dev>
accordingly and tested the different alternatives.
amd64/x86: tested on server and local Macbook Pro
arm32v7: tested on Raspberry Pi Model 4, 4Gb RAM
arm64v8: tested on Jetson Nano, 4Gb RAM
Cheers,
Patrick