You are viewing a plain text version of this content. The canonical link for it is here.
Posted to iot@apache.org by Christofer Dutz <ch...@c-ware.de> on 2022/01/21 17:11:50 UTC
How about an Apache HOP to StreamPipes connector?
Hi all,
I had been thinking .. in industrial IoT we need to do a lot of data collection on premise and on the edge as connecting a PLC to a normal corporate network is generally a bad idea.
Now I know Streampipes is a tad overweight for small edge computers (But not to blame, as it's not built for that)... But HOP seems to be super lightweight.
So, I was thinking of how it would be to use HOP for the Edge processing and then forwarding the data to StreamPipes?
What do you folks think?
Chris
PS: Only CCed Philip as he was the only email, I had in Outlook from the other StreamPipes folks ;-)
RE: How about an Apache HOP to StreamPipes connector?
Posted by Dominik Riemer <ri...@fzi.de>.
Hi Hans,
sounds interesting! I'll play around a little bit with hop to see how an integration could look like.
Dominik
-----Original Message-----
From: Hans Van Akelyen <ha...@gmail.com>
Sent: Tuesday, January 25, 2022 10:50 PM
To: iot@apache.org
Cc: zehnder@apache.org
Subject: Re: How about an Apache HOP to StreamPipes connector?
Hi there,
We have not yet had edge compute cases as far as I know, but it is an area where we would definitely like to help.
If you look at hop you can basically strip it down to only contain the features needed. With the core libraries without any plugins you will end up at around 50MB. You will probably need some transforms to extract the data from the edge system eg (PLC4X transforms when they are available). Or a CSV/text file input to grab some local files you would like to parse and them but this shouldn't add too much bloat.
Then we have options to move those files to AWS/GCP/Azure storage or send them to some message broker.
The main thing is that we tried to make as much as possible pluggable so the platform can be tailored to specific needs and not carry too much bloat.
Cheers,
Hans
On Tue, 25 Jan 2022 at 21:43, Philipp Zehnder <ze...@apache.org> wrote:
> Hi Chris,
>
> thanks for the mail. I also think that this could be very interesting.
>
> What exactly are you envisioning here and how would the communication
> between the edge and the cloud look like?
> Our goal is also to connect data in a more lightweight way, for
> example by only running the adapters on the edge and the other services in the cloud.
>
> How could such an integration look like?
>
> Philipp
>
>
>
> Am 21.01.22, 18:12 schrieb "Christofer Dutz" <ch...@c-ware.de>:
>
> Hi all,
>
> I had been thinking .. in industrial IoT we need to do a lot of
> data collection on premise and on the edge as connecting a PLC to a
> normal corporate network is generally a bad idea.
> Now I know Streampipes is a tad overweight for small edge
> computers (But not to blame, as it's not built for that)... But HOP
> seems to be super lightweight.
>
> So, I was thinking of how it would be to use HOP for the Edge
> processing and then forwarding the data to StreamPipes?
>
> What do you folks think?
>
> Chris
>
> PS: Only CCed Philip as he was the only email, I had in Outlook
> from the other StreamPipes folks ;-)
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: iot-unsubscribe@apache.org For additional
> commands, e-mail: iot-help@apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: iot-unsubscribe@apache.org
For additional commands, e-mail: iot-help@apache.org
Re: How about an Apache HOP to StreamPipes connector?
Posted by Hans Van Akelyen <ha...@gmail.com>.
Hi there,
We have not yet had edge compute cases as far as I know, but it is an area
where we would definitely like to help.
If you look at hop you can basically strip it down to only contain the
features needed. With the core libraries without any plugins you will end
up at around 50MB. You will probably need some transforms to extract the
data from the edge system eg (PLC4X transforms when they are available). Or
a CSV/text file input to grab some local files you would like to parse and
them but this shouldn't add too much bloat.
Then we have options to move those files to AWS/GCP/Azure storage or send
them to some message broker.
The main thing is that we tried to make as much as possible pluggable so
the platform can be tailored to specific needs and not carry too much bloat.
Cheers,
Hans
On Tue, 25 Jan 2022 at 21:43, Philipp Zehnder <ze...@apache.org> wrote:
> Hi Chris,
>
> thanks for the mail. I also think that this could be very interesting.
>
> What exactly are you envisioning here and how would the communication
> between the edge and the cloud look like?
> Our goal is also to connect data in a more lightweight way, for example by
> only running the adapters on the edge and the other services in the cloud.
>
> How could such an integration look like?
>
> Philipp
>
>
>
> Am 21.01.22, 18:12 schrieb "Christofer Dutz" <ch...@c-ware.de>:
>
> Hi all,
>
> I had been thinking .. in industrial IoT we need to do a lot of data
> collection on premise and on the edge as connecting a PLC to a normal
> corporate network is generally a bad idea.
> Now I know Streampipes is a tad overweight for small edge computers
> (But not to blame, as it's not built for that)... But HOP seems to be super
> lightweight.
>
> So, I was thinking of how it would be to use HOP for the Edge
> processing and then forwarding the data to StreamPipes?
>
> What do you folks think?
>
> Chris
>
> PS: Only CCed Philip as he was the only email, I had in Outlook from
> the other StreamPipes folks ;-)
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: iot-unsubscribe@apache.org
> For additional commands, e-mail: iot-help@apache.org
>
>
Re: How about an Apache HOP to StreamPipes connector?
Posted by Philipp Zehnder <ze...@apache.org>.
Hi Chris,
thanks for the mail. I also think that this could be very interesting.
What exactly are you envisioning here and how would the communication between the edge and the cloud look like?
Our goal is also to connect data in a more lightweight way, for example by only running the adapters on the edge and the other services in the cloud.
How could such an integration look like?
Philipp
Am 21.01.22, 18:12 schrieb "Christofer Dutz" <ch...@c-ware.de>:
Hi all,
I had been thinking .. in industrial IoT we need to do a lot of data collection on premise and on the edge as connecting a PLC to a normal corporate network is generally a bad idea.
Now I know Streampipes is a tad overweight for small edge computers (But not to blame, as it's not built for that)... But HOP seems to be super lightweight.
So, I was thinking of how it would be to use HOP for the Edge processing and then forwarding the data to StreamPipes?
What do you folks think?
Chris
PS: Only CCed Philip as he was the only email, I had in Outlook from the other StreamPipes folks ;-)
---------------------------------------------------------------------
To unsubscribe, e-mail: iot-unsubscribe@apache.org
For additional commands, e-mail: iot-help@apache.org